Visual Saliency Evaluation for ASD and Typically Developing Groups

The problem

Saliency models are trained on fixation data pooled from typically developing viewers. The implicit assumption is that one attention prior generalizes. Autistic viewers are known to attend differently to faces, text, and social content, so the question worth asking is narrow and testable: do published saliency models predict ASD fixation maps as well as they predict TD fixation maps, and do the standard evaluation metrics even agree on the answer?

I evaluated three models against fixation maps from both groups: FES, a classical hand-designed feature approach; SalFBNet, a feedback-recurrent convolutional network; and TranSalNet, a transformer-augmented encoder-decoder.

Approach

I scored every model against both groups using four metrics that measure different things.

AUC-Borji is location-based. It treats saliency prediction as binary classification over pixel locations, sweeping a threshold and drawing an ROC curve, with negatives sampled from fixation locations in other images rather than uniformly. That sampling choice matters: it corrects for center bias, because the negatives inherit the same spatial prior as the positives.

CC (Pearson correlation coefficient) is distribution-based and symmetric in its two arguments. It measures how well the shape of the predicted map matches the shape of the ground-truth density, and it penalizes false positives and false negatives equally.

KL divergence is distribution-based and asymmetric. It punishes a model heavily for putting near-zero probability mass where the ground truth has fixations. Missing a fixated region is expensive; spreading extra mass over unfixated regions is comparatively cheap.

NSS is the mean of the normalized saliency map sampled at fixated points. It is location-based and rewards a model for high, sharply peaked response exactly at fixations.

The pipeline ran in PyTorch for inference and MATLAB for parts of the metric implementation, orchestrated through Colab so the whole thing could be re-run from a clean runtime. I kept the map preprocessing explicit: normalization, blur, and histogram handling change these scores more than people expect, and unreported preprocessing is where most saliency comparisons quietly diverge.

What I found

The metrics disagree, and the disagreement is the result. A model can rank first under NSS and lose under KL, because NSS rewards peaked confident maps while KL punishes the same peakedness whenever a real fixation cluster falls outside the peak. CC sits between the two and tends to favor smoother maps. Ranking three models by “saliency accuracy” is therefore not a well-posed question without naming the metric first.

The deliverable was a reproducible set of quantitative summaries plus side-by-side visual comparisons of predicted maps against per-group fixation maps, so the numeric gaps could be checked against what the maps actually look like.

Limitations

The datasets involved are small, and group-level fixation maps average over real variance between individuals. Any difference I measured is a difference in aggregate, not evidence about any particular viewer. I also did not retrain or fine-tune the models on group-specific data, so this is an evaluation of transfer, not an attempt to build a better ASD-specific predictor. That would be the natural next step, and it needs more data than I had.