Research Showcase at the 2021 ED Games Expo

2021 Ed Games Expo
The ED Games Expo is an annual showcase of game-changing innovations in education technology that were developed through more than 30 programs at the US Department of Education and across the federal government. The full agenda of the Ed Games Expo is available here. Follow the ED Games Expo on twitter @USEDgov @SBIRgov and #EDGamesExpoAbout the ED Games Expo
The ED Games Expo is an annual showcase of game-changing innovations in education technology that were developed through more than 30 programs at the US Department of Education and across the federal government. Follow the ED Games Expo on twitter @USEDgov @SBIRgov and #EDGamesExpo

Check out the recording our presentation here.

Event at the 2021 Meeting of the American Educational Research Association (AERA)

Summary Poster of the Event

Experimental Study of the Revised Video Recommendation Algorithm – February 3rd to May 31st, 2020

We recruited 74 teachers from Broward, Palm Beach and Duval school districts to study the improved video recommendation system for the Algebra Nation platform. We randomly assigned the ~14,000 students of the participating teachers to treatment or control conditions. The treatment group received video recommendations using the revised algorithm, while the control group was recommended to watch the next video in the Algebra Nation sequence. Due t COVID-19, the study had two periods: School-based instruction from February 3rd to March 17th; Online-based instruction from March 18th to May 31st. Preliminary results show that students who received the treatment condition during the first period of the study performed better in mathematics tests in Algebra Nation than students in the control group.

Designing Online Platforms to Enable Research Event

Supplemental Study

Through a partnership with Dr. Danielle McNamara at Arizona State University, 100 students in Broward County will be provided high quality, 1:1 tutoring in Algebra 1. Transcripts and EOC grades will be studied to understand what qualities create effective tutoring experiences.

Virtual Learning Lab Panel at the Annual Meeting of the American Educational Research Association

Personalization Algorithm Experimental Study 1

This semester, students taking Algebra 1 in Broward County Public Schools will experience the new and improved Video Recommendation Systems. Students will receive suggestions for the next video they should view, similar to the user experience in YouTube. One group will receive recommendations in the standard order, and one will receive recommendations based on their past performance. We’ll examine their EOC scores to see which system is more effective.

Personalization Algorithm Experimental Study 1

This semester, students taking Algebra 1 in Broward County Public Schools will experience the new and improved Video Recommendation Systems. Students will receive suggestions for the next video they should view, similar to the user experience in YouTube. One group will receive recommendations in the standard order, and one will receive recommendations based on their past performance. We’ll examine their EOC scores to see which system is more effective.

FERA Presentation

Quasi-experimental Evaluation of Usage Patterns of Virtual Learning Environments with Multilevel Mixture Models and Propensity Score Weighting
Walter is awarded Educational Researcher of the Year!

New PI

Dr. Walter Leite assumes the role of PI for the rest of the project.

Panel

A panel was held in conjunction with the annual American Education Research Association exploring opportunities and challenges of personalized learning in education settings.

AERA Conference Re-taker Study poster presentation

The analysis examines the impact of Algebra Nation on the performance of students who failed the end-of-course (EOC) exam the previous year. Results indicate that higher use of Algebra Nation is associated with greater EOC gains, particularly for students most at risk for repeated failure.

Initial Feasibility Study Launched

The recommendation version of Algebra Nation was pilot-tested in Spring 2018 in Alachua County Schools located in Gainesville FL. Students (N = 2,171) used the platform once per week during the spring semester. Teachers completed weekly implementation logs that were compared to usage reports and completed a post-study survey.

IES Principle Investigator Meeting

VLL team hosts a panel on personalized learning, discussing positive and negative components. Topics included the impact on teachers, the costs of providing expanding content, etc.

Measurement Study 2 Progress

Large-scale affect surveys were collected from Algebra 1 students from September-December 2017 (N= 25,629). Each survey prompt included a random question on one of 18 affective states in a non-obtrusive, light-touch approach. The data were used to train neural networks to predict students’ reports of their affective states from their activity with Algebra Nation in the previous 5 minutes. Performance was significantly above random baselines for 10 affective states: confusion, contentment, disappointment, engagement, frustration, happiness, hopefulness, pleasantness, pride and relief.

Completion of Measurement Study 1

This analysis focused on the impact of Algebra Nation use on student’s scores on the end of course Algebra 1 exam (EOC). For the 2015-2016 school year, data from more than 80,000 students indicated that Algebra Nation users had better EOC performance than non-users.

First Graduate Student Training Workshop

Students spent four full days on campus working with project members on topics such as a comparison of urban, suburban and rural schools, and an investigation of video tutors’ popularity as a function of gender, age and ethnicity.

Measurement Study 2 Initiated

The team at CU leads research to develop sensor-free engagement estimates that can be used to assess the impact of personalized instruction. In addition, student reports of cognitive and emotional states are aligned with log files of on-task and off-task behaviors, with the goal of identifying “trigger events” that could be leveraged to promote engagement in an adaptive version of the system.

Measurement Study 1 Initiated

The first study was initiated to identify a) usage patterns at the student level that are linked to EoC outcomes that can be used to drive a recommendation engine and b) usage patterns at the classroom level that are linked to EoC outcomes that can be used to design improved professional development.

Build a Data Archive

The first step in the focused program of research was to build an archive of data from prior users to identify patterns of use that are linked to successful outcomes, using Algebra 1 End of Course exam scores, provided by the Florida Department of Education, as well as user data from the Algebra Nation platform.

University of Florida

Dr. Carole Beal (UF), supported by Dr. George Michailidis (UF), Dr. Donald Pemberton (UF), Dr. Sidney D’Mello (CU), and Walter Leite (UF), secured $8.9M from the Institute of Education Sciences to support their research titled “Precision Education: The Virtual Learning Lab.”