DFKI Project: Eye Tracking Inside the Oculus Rift

What are the possibilities for using eye tracking technology in cognitive computing, specifically in research on applications for persons with dementia? This is one of the central questions which Dr. Daniel Sonntag and his team at DFKI pursue in their KOGNIT research project. To do so, they use the SMI Eye Tracking Upgrade for the Oculus Rift DK2 HMD - complemented with the SMI Unity plug-in - for integrating a person’s gaze with real and virtual environments.

KOGNIT Project on Dementia

Different scenarios are being set up within the ongoing KOGNIT research project financed by the German Ministry for Education and Research. With the help of eye tracking, eye and gaze data of patients would be recorded in virtual representations of every-day situations, such as crossing the street or getting breakfast ready. One purpose is to learn more about the gaze behavior of persons with certain cognitive deficits like dementia. Another is to find out how eye tracking can assist dementia patients with memory loss in these every-day situations.

Episodic Memory Visualizations

In one particular approach, the team has created an application where episodic memory visualizations from a digital data base are shown to patients to help them with every-day tasks, such as

- Recalling names of persons and objects

- Navigation, orientation

- Searching for objects (e.g. pair of glasses, book)

- Reminding (to drink, take pills, etc.)

- Games for entertainment and memory training

Making Human Eyes a Tool for Cognitive Computing

The DFKI research program explores how cognitive computing systems can interact with people in real time, in order to extend what neither person or machine could do on their own, for example fill in the gaps caused by memory loss. Besides the SMI Eye Tracking Upgrade for the Oculus Rift DK2, DFKI researchers are using SMI Eye Tracking Glasses combined with propietary DFKI object recognition software.