Pupil Labs - Neon¶

Setting Up the Eye Tracker¶

1. Follow Neon’s Getting Started guide to setup the headset and companion device.

Setting Up PsychoPy^®¶

1. Install the Pupil Labs plugin using the Plugin Manager and restart Builder.

2. Open experiment settings in the Builder Window (cog icon in top panel)

3. Open the Eyetracking tab

4. Modify the properties as follows:

   • Select Pupil Labs (Neon) from the Eyetracker Device drop down menu

   • Companion address / Companion port - Defines how to connect to the Companion Device. These values can be found in the Neon Companion app by clicking the Stream button in the top-left corner of the app.

   • Recording enabled - Enable this option to create a recording on the Companion device.

5. Add AprilTag components to the routines that require eyetracking. - Three tags is generally considered a bare minimum, but more tags will yield more robust detection and more accurate mapping. - All the tags which are visible together must each have a unique ID. - Tags can be placed anywhere on the screen as long as they are fully visible and do not overlap.

A sample experiment is available for reference.

EyeTracker Class¶

class psychopy.iohub.devices.eyetracker.hw.pupil_labs.pupil_core.EyeTracker(*args, **kwargs)[source]¶

Bases: EyeTrackerDevice

Implementation of the Common Eye Tracker Interface for the Pupil Core headset.

Uses ioHub’s polling method to process data from Pupil Capture’s Network API.

To synchronize time between Pupil Capture and PsychoPy, the integration estimates the offset between their clocks and applies it to the incoming data. This step effectively transforms time between the two softwares while taking the transmission delay into account. For details, see this real-time time-sync tutorial.

This class operates in two modes, depending on the pupillometry_only runtime setting:

1. Pupillometry-only mode

   If the pupillometry_only setting is to True, the integration will only receive eye-camera based metrics, e.g. pupil size, its location in eye camera coordinates, etc. The advatage of this mode is that it does not require calibrating the eye tracker or setting up AprilTag markers for the AoI tracking. To receive gaze data in PsychoPy screen coordinates, see the Pupillometry+Gaze mode below.

   Internally, this is implemented by subscribing to the pupil. data topic.

2. Pupillometry+Gaze mode

   If the Pupillometry only setting is set to False, the integration will receive positional data in addition to the pupillometry data mentioned above. For this to work, one has to setup Pupil Capture’s built-in AoI tracking system and perform a calibration for each subject.

   The integration takes care of translating the spatial coordinates to PsychoPy display coordinates.

   Internally, this mode is implemented by subscribing to the gaze.3d. and the corresponding surface name data topics.only

Note

Only one instance of EyeTracker can be created within an experiment. Attempting to create > 1 instance will raise an exception.

getLastGazePosition() → Tuple[float, float] | None[source]¶

The getLastGazePosition method returns the most recent eye gaze position received from the Eye Tracker. This is the position on the calibrated 2D surface that the eye tracker is reporting as the current eye position. The units are in the units in use by the ioHub Display device.

If binocular recording is being performed, the average position of both eyes is returned.

If no samples have been received from the eye tracker, or the eye tracker is not currently recording data, None is returned.

Returns:

• None:

  If the eye tracker is not currently recording data or no eye samples have been received.

• tuple:

  Latest (gaze_x,gaze_y) position of the eye(s)

getLastSample() → None | psychopy.iohub.devices.eyetracker.MonocularEyeSampleEvent | psychopy.iohub.devices.eyetracker.BinocularEyeSampleEvent[source]¶

The getLastSample method returns the most recent eye sample received from the Eye Tracker. The Eye Tracker must be in a recording state for a sample event to be returned, otherwise None is returned.

Returns:

• MonocularEyeSampleEvent:

  Gaze mapping result from a single pupil detection. Only emitted if a second eye camera is not being operated or the confidence of the pupil detection was insufficient for a binocular pair. See also this high-level overview of the Pupil Capture Data Matching algorithm

• BinocularEyeSample:

  Gaze mapping result from two combined pupil detections

• None:

  If the eye tracker is not currently recording data.

isConnected() → bool[source]¶

isConnected returns whether the ioHub EyeTracker Device is connected to Pupil Capture or not. A Pupil Core headset must be connected and working properly for any of the Common Eye Tracker Interface functionality to work.

Parameters:

None –

Returns:

bool: True = the eye tracking hardware is connected. False otherwise.

isRecordingEnabled() → bool[source]¶

The isRecordingEnabled method indicates if the eye tracker device is currently recording data.

Returns:

True == the device is recording data; False == Recording is not occurring

setConnectionState(enable: bool) → None[source]¶

setConnectionState either connects (setConnectionState(True)) or disables (setConnectionState(False)) active communication between the ioHub and Pupil Capture.

Note

A connection to the Eye Tracker is automatically established when the ioHub Process is initialized (based on the device settings in the iohub_config.yaml), so there is no need to explicitly call this method in the experiment script.

Note

Connecting an Eye Tracker to the ioHub does not necessarily collect and send eye sample data to the ioHub Process. To start actual data collection, use the Eye Tracker method setRecordingState(bool) or the ioHub Device method (device type independent) enableEventRecording(bool).

Parameters:

enable (bool) – True = enable the connection, False = disable the connection.

Returns:

bool: indicates the current connection state to the eye tracking hardware.

setRecordingState(should_be_recording: bool) → bool[source]¶

The setRecordingState method is used to start or stop the recording and transmission of eye data from the eye tracking device to the ioHub Process.

If the pupil_capture_recording.enabled runtime setting is set to True, a corresponding raw recording within Pupil Capture will be started or stopped.

should_be_recording will also be passed to EyeTrackerDevice.enableEventReporting().

Parameters:

recording (bool) – if True, the eye tracker will start recordng data.; false = stop recording data.

Returns:

bool: the current recording state of the eye tracking device

property surface_topic: str¶

Read-ony Pupil Capture subscription topic to receive data from the configured surface

trackerSec() → float[source]¶

Returns EyeTracker.trackerTime()

Returns:

The eye tracker hardware’s reported current time in sec.msec-usec format.

trackerTime() → float[source]¶

Returns the current time reported by the eye tracker device.

Implementation measures the current time in PsychoPy time and applies the estimated clock offset to transform the measurement into tracker time.

Returns:

The eye tracker hardware’s reported current time.

Supported Event Types¶

The Neon–PsychoPy^® integration provides real-time access to BinocularEyeSampleEvents events. The supported fields are described below.

class psychopy.iohub.devices.eyetracker.BinocularEyeSampleEvent(*args, **kwargs)[source]¶

The BinocularEyeSampleEvent event represents the eye position and eye attribute data collected from one frame or reading of an eye tracker device that is recording both eyes of a participant.

Event Type ID: EventConstants.BINOCULAR_EYE_SAMPLE

Event Type String: ‘BINOCULAR_EYE_SAMPLE’

device_time: float¶

time of gaze measurement, in sec.msec format, using Pupil Capture clock

logged_time: float¶

time at which the sample was received in PsychoPy, in sec.msec format, using PsychoPy clock

time: float¶

time of gaze measurement, in sec.msec format, using PsychoPy clock

delay: float¶

The difference between logged_time and time, in sec.msec format

gaze_x: float¶

x component of gaze location in display coordinates. Set to float("nan") in pupillometry-only mode.

gaze_y: float¶

y component of gaze location in display coordinates. Set to float("nan") in pupillometry-only mode.

eyetracker.hw.pupil_labs.neon.EyeTracker:
    # Indicates if the device should actually be loaded at experiment runtime.
    enable: True

    # The variable name of the device that will be used to access the ioHub Device class
    # during experiment run-time, via the devices.[name] attribute of the ioHub
    # connection or experiment runtime class.
    name: tracker

    device_number: 0

    #####

    model_name: Neon

    model_number: "1"

    serial_number: N/A

    manufacturer_name: Pupil Labs (Neon)

    software_version: N/A

    hardware_version: N/A

    firmware_version: N/A

    #####

    monitor_event_types: [BinocularEyeSampleEvent]

    # Should eye tracker events be saved to the ioHub DataStore file when the device
    # is recording data ?
    save_events: True

    # Should eye tracker events be sent to the Experiment process when the device
    # is recording data ?
    stream_events: True

    # How many eye events (including samples) should be saved in the ioHub event buffer before
    # old eye events start being replaced by new events. When the event buffer reaches
    # the maximum event length of the buffer defined here, older events will start to be dropped.
    event_buffer_length: 1024

    # Do not change this value.
    auto_report_events: False

    device_timer:
        interval: 0.005

    #####

    runtime_settings:
        companion_address: neon.local
        companion_port: 8080
        recording_enabled: True
        camera_calibration: scene_camera.json