Gaze, physics and cognitive insight
An interactive version of our poster for the ACM GreekCHI-CYCHI Symposium 2026.
Poster · ACM GreekCHI-CYCHI Symposium 2026
Applying Physics-Informed Neural Networks and Eye-Tracking Data to Gain Cognitive Insights in Human-Centred Applications
Combining the equations governing the dynamics of an eye with eye-tracking data, we created an algorithm that essentially replicates the function of eye-tracking software. Along the way we compared different networks on the problem, and looked at what the model's least accurate moments have in common.
Move your pointer, or tap: the eye turns to follow it.
Objectives
How the model learns
A physics-informed neural network (PINN) is a network whose loss combines a standard data term with a physics term, derived from the equations governing the system. Pick a step to see what it does.
The physics weight λ = 1.4 was tuned.
The guiding equation
The physics term encodes the eye's rotational dynamics on Donders' surface. Tap a symbol to see what it stands for.
Does it work?
The pure PINN achieves the lowest error on both axes, beating the feed-forward, GNN and TCN baselines and every PINN hybrid.
Mean absolute error by model
Lower is better. All differences vs. the PINN are significant (Wilcoxon, p < 0.01).
| Model | MAE X | MAE Y |
|---|
Predicted vs. actual gaze points
Colour shows the Euclidean error of each prediction, from blue (low) to red (high).
What's with the deviations?
We trained a classifier to predict when the physics model misses, using standard ocular features.
An AUC of 0.768 is well above chance, which establishes that the ocular features carry real joint information about when the physics model fails. It is also far from perfect, and it should be. The value of the residual is precisely that it captures something the standard features do not fully explain.
- Blink countlargest positive weight
- Saccade velocitypositive
- Timesmall, positive
- Pupilsmall, negative
- Fixation durationlargest negative weight
Conclusions
Physics-informed constraints give the most accurate gaze model we tested.
Adding graph or temporal layers to the PINN increased error variance, which suggests the physics prior alone captures gaze dynamics most cleanly.
The approach is interpretable and data-efficient.
The same signals are a promising substrate for cognitive insight: attention, fatigue, comprehension.