Experience Stern | Faculty & Research

Window to the soul? Maybe, but the eyes are also a flashing neon sign for a new artificial intelligence-based system that can read them to predict what you’ll do next.

A new paper entitled, “Predicting Consumer Choice From Raw Eye-Movement Data Using the RETINA Deep Learning Architecture,” published by Data Mining and Knowledge Discovery and co-authored by NYU Stern Professor Alexander Tuzhilin, alongside professors Michel Wedel (University of Maryland’s Robert H. Smith School of Business) and Moshe Unger (Tel Aviv University), utilized eye-tracking technology and a new deep-learning AI algorithm to predict study participants’ choices while they viewed a comparison website with rows and columns of products and their features.

The algorithm, known as RETINA (Raw Eye Tracking and Image Ncoder Architecture), could accurately and effectively zero in on the areas of interest of the website crucial for making appropriate selection choices before people had even made their decisions. With their advanced machine-learning method, the research team could use the full scope of raw data from the eye-tracking rather than the snippets current methods record. Unusually, the algorithm is able to incorporate raw eye movement data from each eye.

The RETINA algorithm could be applied in many settings by all types of companies. For example, a retailer could use it to enhance the virtual shopping experiences they are developing in the metaverse, a shared, virtual online world. Many of the VR devices people will use to explore the metaverse will have built-in eye tracking to help better render the virtual environment. With this algorithm, retailers could tailor the mix of products on display in their virtual store to what a person will likely choose, based on their initial eye movements.

RETINA has applications outside of marketing as eye tracking becomes more ubiquitous in many other fields, including medicine, psychology and psychiatry, usability and design, arts, reading, finance, accounting—anything where people are making decisions based on some kind of visual assessment.

The biggest players in tech, including Apple, Meta, and Google, have recently acquired eye-tracking companies and are considering a range of applications. With front-facing cameras, it is now possible to track people’s eye movements from a personal smartphone, tablet, computer, or a VR device.

Eye tracking will become available at a very large scale, predict the authors, who also note that privacy concerns remain top of mind: “The processing of the eye movement data typically has been very laborious. With this algorithm, we side-step a lot of that, so there may be many applications that we haven’t even thought about. The excitement of future applications, however, is tempered by questions of consent and data security when using eye-tracking data, which urgently need to be addressed.”

______

This article was adapted from a research summary published by Maryland Today. See the original publication here.