Student retention through online faculty and student learning: Documenting and measuring impact on Hispanic students
General description of the project
Student retention at Sacramento State is addressed at a systemic level through faculty professional development, course redesign, and student programs that ensure each student has access and support to effectively use technology for learning.
The practice underlying all these projects is the integration of evidence based professional development for faculty, data informed course redesign, and institution level support for student access to technology.
Sacramento State has developed an online Faculty Learning Community (FLC) program that provides faculty with knowledge and skills to implement evidence based and culturally responsive teaching strategies. These FLCs utilize curriculum from Quality Matters, ACUE (Association of College and University Educators), and ESCALA Education to provide faculty professional development at scale, impacting over 500 faculty over the past 3 years. These online curricula are embedded in a yearlong FLC program that includes classroom “Observe and Analyze” sessions, teaching and learning portfolio creation, and a culminating event where implemented course design and teaching strategies are disseminated to the campus community.
Sacramento State’s course redesign programs are integrated with the faculty professional development programs and include institutional data informed assessment, and an online course quality assurance process. Evidence of success includes examples from STEM course redesign projects. A team redesign of an introductory Physics course utilizing ACUE and ESCALA strategies has reduced the failure rate and narrowed the equity gap for Hispanic Students. Redesign of an introductory engineering course utilizing ACUE active learning and equitable learning strategies has created accessible microlectures, formative assessments, and a higher level of student engagement using digital tools. An upper division thermodynamics course has been redesigned to earn Quality Matters Certification for online courses, meeting 23 essential standards for best practice. The redesign of these courses has been supported by one or more of our ACUE microcredential programs, Summer Teaching Institutes, Summer Course Compression Grant, or Online Course Services Programs, all sponsored through Sacramento State’s Center for Teaching and Learning (CTL).
Student programs related to technology that support retention include a culturally responsive computer loan program and an online learning tutorial, Hornet Learning Online 101. These programs ensure students have the tools, connectivity, and skills to access and utilize technology mediated instruction. In addition to a campus wide long term computer loan program and reduced cost internet options, a specialized loan program was created for student groups that were found to be disproportionately impacted by the move to emergency remote instruction during the COVID-19 pandemic. One group experiencing a large fall in student retention during this time was the College Assistance Migrant Program (CAMP). Quantitative institutional data and qualitative CAMP program data revealed the need and special requirements for a computer loan program customized to the needs of the CAMP students and staff. This program has been implemented this Fall 2023 and data is pending.
Before March of 2020 Sacramento State had few online courses and little support for online students. To support the transition to remote online teaching and learning for the 2020/2021 academic year, an online learning tutorial, “Hornet Learning Online 101” was launched. Since the Fall of 2020 it has enrolled over 6000 students annually. The one-hour tutorial contains topics including basic academic technology literacy, nuts and bolts learning management tutorials, and emerging AI learning. Students receive a completion badge at the end of the tutorial that can be submitted to course instructors for credit. Of specific importance has been the “online learning readiness” survey that students complete as part of the tutorial. Because this survey can be linked to other institutional data sets, analysis of survey results in Spring of 2021 provided a snapshot of the discrepancy between the number and demographics of students who expressed a need for computers and high-speed internet access and the number who actually checked out devices.
Evidence of success or cost effectiveness:
The online FLC program has been analyzed for its impact on student grades. A difference-in-difference analysis of student grades comparing FLC faculty and faculty non-participants shows a +0.20 grade point unit difference and a higher course pass rate for students in courses taught by FLC participants. Additional evidence from the FLC teaching portfolios, faculty surveys and student surveys have also been analyzed for evidence of instructional change including implementation of equity minded best practices. Specific analysis of ESCALA FLC portfolios will be available at the Culminating event 2023
Evidence of increasing pass rates and closing equity gaps for redesigned courses will be presented along with samples of digital curriculum and assessments. The successful use of linked data sets from the Hornet Learning Online 101 tutorial, institutional research, and information technology demonstrates how specific characteristics of the digital divide can be revealed and addressed. Data from the Fall 2023 CAMP Computer Loan Program is pending.
Technologies
Technologies for teaching, learning, research, and project evaluation:
Technologies used for the Online FLCs included the CANVAS Learning Management System (LMS), zoom video conferencing, and digital video recordings. Curriculum providers include ACUE, ESCALA, and Quality Matters. Zoom was used for synchronous Observe and Analyze, Portfolio making, and Culminating Event sessions. LMS, Qualtrics survey, and single sign on technology were used to collect faculty and student survey data. Technology used for the difference-in-difference study included use of R coding, SPSS, and excel for data extraction, analysis, and visualization. The use of a secure data repository remotely accessibly by a team of education researchers, statisticians, and researchers was essential to obtaining institution level measures of impact and success of the projects. All these technologies are judiciously used to ensure that the redesigned courses are accessible, meet quality standards, and yield equitable teaching and learning outcomes across both in-person and online modalities.
Technologies for advising:
A Central Management System (CMS) is to check grades and student progress, facilitating decisions on interventions and determining the necessary support for students. Additionally, we employ EAB for check-ins, follow-ups, and interventions. CAMP students are required to complete five study hours per week, and this software aids in tracking their study hours. In instances where EAB is not operational, we have developed Excel spreadsheets to monitor student activities and ensure accountability. CAMP students are mandated to participate in three check-ins during the semester with the retention specialist. When a student is underperforming, an intervention plan is created with the student to ensure their active involvement in the plan. The plan encompasses time management, tutoring sessions, study hours, and a weekly progress check-in to ensure the student is following the plan. If no progress is observed, additional tutoring time is assigned to ensure the student receives the necessary support. Check-ins with the retention specialist are required in person, while tutoring sessions are provided either in person or online. The majority of students choose the online option. We have just been awarded a technology grant to expand CAMP retention services. We plan to use technology to translate CAMP websites into Spanish and the create an AI supported 2nd and 3rd year retention program.
Explain project results
Sacramento State is a Hispanic Serving Institution, with approximately 38% of our enrollment composed of Hispanic students. As such, reducing the equity gaps, course fail rates and improving performance metrics for our student body has a significant positive impact on our Hispanic students.
The project and the activities described in this proposal identify faculty as agents of systemic and sustainable change. All the FLCs and professional development promote evidence-based strategies that build faculty’s cultural competence skills and their ability to engage students in active learning in online environments. The ESCALA curriculum in particular focuses on collectivist cultural values that can be in opposition to the individualist cultural norms of higher education. By implementing more inclusive language and course policies and designing assessments and projects that have high utility value, faculty change learning outcomes for Hispanic students. Major accomplishments to improve teaching and learning through an equity lens over the past 3 years include:
1. Faculty have earned over 308 ACUE microcredentials and implemented at least one evidence based and/or equity minded teaching strategy in the course(s) that they teach. The microcredentials were in the following areas:
a. Designing Student-Centered Courses
b. Inclusive Teaching for Equitable Learning
c. Promoting Active Learning Online
d. Creating an Inclusive and Supportive Learning Environment
e. Inspiring Inquiry and Preparing Lifelong Learners
2. A total of 221 Quality Matters completion certifications were earned by 126 faculty in one or more of the following areas:
a. Applying the QM rubric (APPQMR)
b. Improving Your Online Course (IYOC)
c. Higher Ed Peer Reviewer Course (PRC)
3. A total of 64 courses have been redesigned for fully asynchronous online delivery in summer 6-week format, which have been shown to increase retention rates and reduce time to degree for students. Many of these redesigned courses have been selected for California State University (CSU) system’s Chancellor’s Office Online Course program, which opens these courses to not just Sacramento State students, but for students across all the 22 campuses in the CSU system.
4. A total of 51 faculty either already earned ECSALA certification for Culturally Responsive Pedagogies in STEM for HSIs or are currently pursuing the certificate. The Hispanic student-centered pedagogical techniques implemented, and cultural competencies gained through ESCALA include:
a. A deep self-reflection about faculty’s own cultural values in the context of high and low context cultures in order to eschew deficit mind-set
b. A recognition of Hispanic students’ cultural advantages/ventajas
c. Intentionality in validating Latinx students’ STEM identity through growth mindset and validating language that teaches to the cultural strengths of Latinx students
d. Incorporating Intrinsic, Situational and Attainment Utility Values into assignments in order to enhance Latinx students’ engagement in “doing” STEM
Why it should be considered best practice?
1. The project identifies and supports faculty as the prime agents of enduring cultural change at a systemic level.
2. Through sustained faculty professional development in research and evidence based pedagogical approaches and cultural competence, this project demonstrates that there is a significant increase (0.2 grade point units) in student performance and thereby in their retention.
3. This model of Faculty Learning Communities and Communities of Practice have been shown in literature to act as catalyst for institutional change at a Hispanic Serving Institution.
4. The courses redesigned by participating faculty can be easily adopted to fully asynchronous online format.
5. The multi-dimensional professional development opportunities include a suite of practices for academic rigor, rigor in online course delivery, inclusivity, and culturally responsive instruction.
6. Many practices reflect the latest from published research and are based on sound theoretical principles. Hence, the success of this project and its model of Online FLCs can be easily replicated with high fidelity for repeatable results.
Highlights of your proposed presentation
1. The course redesigns and professional development activities implemented in this project are funded through competitive federal grants that focus on Hispanic student retention and evidence-based strategies.
2. Rigorous quantitative and qualitative evaluation of project components has demonstrated significant positive impact in increasing students’ grades and thereby in progress to degree and retention.
3. Purposeful and intentional use of technology is incorporated to enable superior quality courses to be offered in asynchronous fully online format.
4. Particular attention is paid to ensure that cultural competence of faculty is built through ESCALA and ACUE credentials, so that Hispanic and other students of color experience an inclusive learning environment.
5. 308 ACUE microcredentials have been earned, 221 faculty certifications in QM have been completed, and 51 in ESCALA.
6. A total of 64 courses are redesigned through the Summer Online Grant program.
The Evaluation Committee will evaluate submitted proposals based on the following criteria. Each area will be rated on a scale from 1 to 7 (1= non-satisfactory; 7 =outstanding), for a maximum of 63 points.