MIT Media Lab
Personal project note, June 2026. Not peer reviewed. Documentation written with Claude AI. The classifier described here runs on this site.
This is a hobby experiment. The site has a signal strip at the bottom of each page that visualises cursor activity in real time, and the natural question was whether the same behavioural data could support a coarse label — something like "this person is reading carefully" versus "they are scanning". The Attentive Cursor Dataset provides a reasonably sized corpus with labelled cursor sessions, so it offered a convenient starting point.
The goal was not to produce a production-quality attention detector. It was to (a) practise the full pipeline from raw behavioural data to a deployed browser model, (b) understand where cursor-based attention inference actually sits relative to the literature, and (c) end up with something honest that can sit on the site without overclaiming what it does.
The Attentive Cursor Dataset (Leiva & Arapakis, Front. Hum. Neurosci., 2020) contains mouse-tracking logs from a crowdsourced study where participants browsed search engine result pages (SERPs) containing display advertisements. Ground truth is a self-reported attention score on a 1–5 Likert scale, with 1 = no attention.
We binarise the label at threshold 3: attention ≥ 3 is mapped to class 1 (attentive), attention < 3 to class 0 (not attentive). After dropping sessions with missing or near-empty logs, the working dataset consists of 2,714 sessions: 1,985 attentive (73.1%) and 729 not attentive (26.9%). The class imbalance is substantial and is handled by class-balanced sample weighting during training.
| Property | Value |
|---|---|
| Sessions (after filtering) | 2,714 |
| Sessions dropped (missing logs) | 195 |
| Attentive (label 1) | 1,985 (73.1%) |
| Not attentive (label 0) | 729 (26.9%) |
| Median events per session | 29 |
| Range of events per session | 3 – 397 |
We extract 17 scalar features from the sequence of mousemove events in each session. These are chosen because they are computable from a raw (timestamp, x, y) sequence with no additional data, which means the same computation can run in real time in a browser. No temporal or sequence models are used; all features are summary statistics over the session window.
| Feature | Description |
|---|---|
n_moves | Total number of mousemove events |
duration_ms | Session length (ms) |
total_dist | Total cursor path length (px) |
mean_vel | Mean cursor speed (px/ms) |
std_vel | Standard deviation of speed |
max_vel | Peak speed |
median_vel | Median speed |
mean_angle_diff | Mean direction change between consecutive events (rad) |
dir_changes | Count of direction changes exceeding 45° |
n_pauses | Count of intervals with speed < 0.1 px/ms |
pause_ratio | Fraction of intervals counted as pauses |
x_range | Horizontal extent of cursor path (px) |
y_range | Vertical extent of cursor path (px) |
bbox_area | Bounding box area (x_range × y_range) |
std_x | Standard deviation of x positions |
std_y | Standard deviation of y positions |
n_clicks | Total click count |
Three classifiers are compared: logistic regression, SVM with an RBF kernel, and XGBoost. All are wrapped in a standardisation pipeline (zero mean, unit variance). Logistic regression and SVM use class_weight='balanced'; XGBoost uses scale_pos_weight set to the inverse class ratio (2.72). Hyperparameters are selected by grid search on 5-fold stratified cross-validation with AUC as the criterion.
| Model | Grid |
|---|---|
| Logistic Regression | C ∈ {0.01, 0.1, 1, 10}; penalty = L2 |
| SVM (RBF) | C ∈ {0.1, 1, 10}; γ ∈ {scale, auto} |
| XGBoost | depth ∈ {3, 4, 6}; estimators ∈ {100, 200}; lr ∈ {0.05, 0.1} |
Table 4 shows cross-validated performance. All three models achieve AUC close to 0.55, which is modestly above chance but weak in absolute terms. This is consistent with prior findings: Leiva & Arapakis (2020) report similar AUC values for summary-statistic approaches on the same dataset, with substantial improvements only when using deep sequence representations of the full cursor trajectory.
| Model | AUC | F1 | Accuracy | Avg. Prec. |
|---|---|---|---|---|
| XGBoost | 0.559 ± 0.016 | 0.741 ± 0.010 | 0.631 ± 0.015 | 0.753 ± 0.011 |
| Logistic Regression | 0.554 ± 0.021 | 0.691 ± 0.014 | 0.583 ± 0.017 | 0.762 ± 0.020 |
| SVM (RBF) | 0.550 ± 0.034 | 0.724 ± 0.021 | 0.612 ± 0.024 | 0.762 ± 0.022 |
Table 5 shows the classification report on a held-out 20% split (n = 543) for the best model (XGBoost, tuned). Precision on the minority class (not attentive) is 0.30, indicating substantial false positives even with class balancing. The model predominantly assigns the majority class.
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Not attentive | 0.30 | 0.38 | 0.33 | 146 |
| Attentive | 0.75 | 0.66 | 0.70 | 397 |
| Macro avg | 0.52 | 0.52 | 0.52 | 543 |
Confusion matrix (held-out, n = 543):
| Pred: Not att. | Pred: Att. | |
| True: Not att. | 56 | 90 |
| True: Att. | 133 | 264 |
The top-ranked features by XGBoost gain are std_y, y_range, total_dist, max_vel, and duration_ms. The dominance of vertical-extent features may reflect that attentive SERP sessions involve more deliberate vertical scanning of the page.
The XGBoost model (100 trees, max depth 3, lr = 0.05) is exported as a tree dump in JSON format and loaded client-side by cursor-classifier.js. The inference pipeline standardises features using training-set statistics, traverses all trees recursively, applies a sigmoid to the summed leaf values, and produces a probability. At 8-second intervals, the label is updated to one of three states based on fixed probability thresholds: focused (P ≥ 0.60), engaged (0.47 ≤ P < 0.60), or scanning (P < 0.47). Model size is approximately 124 KB. No data is sent to any server.
The results are weak. AUC of 0.56 means the classifier is doing very little beyond guessing the majority class more often, which the class-balanced training partially corrects for. Several compounding problems make it unlikely that a substantially better result is achievable with this feature set alone.
Domain mismatch. The training data comes from SERP browsing under a specific crowdsourcing protocol; a personal portfolio site differs in pace, layout, and task. Cursor behaviour that was predictive of ad attention may not transfer to engagement with academic content. Whether the model generalises to this site at all is unknown.
Label validity. Self-reported attention to a web advertisement, collected post-hoc on a 5-point scale by crowdsourced workers, is a noisy ground truth. The label threshold (attention ≥ 3) is an arbitrary cut that conflates moderate and high attention.
Feature coverage. Summary statistics over a short session lose temporal structure. Prior work (Arapakis & Leiva, SIGIR 2020) demonstrates that deep representations of cursor trajectories substantially outperform handcrafted features. The 17 features here omit potentially informative signals such as reading pauses (hover dwell time), scroll velocity, and section-level position.
Ground truth for this site. There is no labelled cursor data from this site, so the model is applied out of distribution. A proper calibration on this domain would require self-report labels from actual visitors, which is impractical for a personal page.
Leiva, L. A., & Arapakis, I. (2020). The Attentive Cursor Dataset. Frontiers in Human Neuroscience, 14. doi:10.3389/fnhum.2020.565664
Arapakis, I., & Leiva, L. A. (2020). Learning efficient representations of mouse movements to predict user attention. Proc. SIGIR 2020.