The problem
In a recorded lecture it's easy to drift off without noticing, and the person teaching never learns where attention dropped. We wanted the player itself to notice and help you catch up, without sending video of your face to a server.
What I built
- I trained the engagement classifier on MediaPipe face-landmark features and ran it in the browser with ONNX Runtime Web, so the classification happens on the viewer's device.
- When engagement drops, the video pauses and asks Gemini-generated questions about what you missed. I built the Flask back end on the Gemini API for these quizzes and for summaries, packaged with Docker for Google Cloud.
- React features: the ONNX monitor, the groups dashboard and the summaries view.
Results
- Best Poster Award at ACM SYSTOR 2025 for "FocusFlow: A Real-Time Engagement System Enabled by Client-Side Inference".
- 1st place in the research and infrastructure track of MTA's 2025 final-project exhibition (about 70 projects).
- Demo paper at AIED 2025 in Italy.