[ENH] BEP 020 Eye Tracking #1128

mszinte · 2022-06-15T12:09:03Z

Here is the specifications of the BEP 020 about eye tracking.

it follows the main discussion initiated on a google document.
it includes the different modification the group of maintainers suggested to us during our zoom meeting.
it includes the macros as used in other modality specific extensions
it includes links toward dataset examples.

Note

We meet regularly and everyone is welcome :
Next meeting October 3Rd 2024 4pm CET on zoom.
Note that if you consider joining but this time or day doesn't suits you, reach me (@mszinte) and I will arrange another appointment.

Notes of last meeting

Chat and discussions also happening on matrix

We are currently drafing a companion paper for this BEP, feel free to participate (GoogleDoc)

Tip

HTML preview of this BEP

Issues for:

general todo list: To do list mszinte/bids-specification#1
conversion discussion: Converter roadmap mszinte/bids-specification#4
list datasets: eyetracking dataset examples mszinte/bids-specification#3

src/04-modality-specific-files/10-eye-tracking.md

correction of text

sappelhoff · 2022-07-25T09:45:48Z

(NOTE: I'll cross-post this message across several BEP threads)

Hi there, just a quick notification that we have just merged #918 and it may be interesting to look at the implications for this BEP.

We are introducing "BIDS URIs", which unify the way we refer to and point to files in BIDS datasets (as opposed to "dataset-relative" or "subject-relative" or "file-relative" links).

If the diff and discussion in the PR is unclear, you can also read the rendered version: https://bids-specification.readthedocs.io/en/latest/02-common-principles.html#bids-uri

Perhaps there are things in the BEP that need adjusting now, but perhaps also not -- in any case it's good to be aware of this new feature!

Let me know if there are any questions, comments, or concerns.

mkdocs.yml

src/04-modality-specific-files/10-eye-tracking.md

src/schema/metadata/AOIDefinition.yaml

src/04-modality-specific-files/10-eye-tracking.md

effigies · 2024-05-30T15:33:31Z

@oesteban #1128 (comment):

Should be doable. Opened a validator issue for it. bids-standard/bids-validator#1948

Remi-Gau · 2024-06-27T13:54:07Z

src/schema/objects/metadata.yaml

+    The number of calibrations corresponding to this run.
+  type: integer
+  minimum: 0
+CalibrationPosition:


assumes the same position for all calibrations / validations

julia-pfarr · 2024-07-08T09:53:19Z

src/modality-specific-files/physiological-recordings.md

+}
+```
+
+Content of `sub-01_task-VisualSearch_events.json`:


Is this really the _events.json file or is it the _physioevents.json?

This is correct, it annotates the events.tsv file, as opposed to the new _physioevents.json.

So basically in the _physioevents.json file I only have the column names plus their description? And do I need a _events.json file if I do not have an _events.tsv? Because Martins dataset on Open Neuro does not have it.

Okay, gotcha -- you've actually hit an edge case we do not currently cover:

We say that some metadata belonging in the _events.json file becomes mandatory if eye tracking is present

However, BIDS allows task-rest to dismiss the events.tsv file. This BEP could make it mandatory for rest, even if the events.tsv file is just empty (only the header row).

WDYT @effigies @Remi-Gau ?

(dataset: https://doi.org/10.18112/openneuro.ds004158.v2.0.2)

Or include it in the physioevents.json? (if this does not disturb too much other stuff) Because that would be easy to include in the converter as we are already asking users to put some metadata manually in themetadata.yml which is read in by our code. The current code actually does that because I understood it wrong, cf my initial question...

Not sure that is a good idea. If I were to move the metadata from events.json, I would think it'd be better in the physio.json file rather than physioevents.json.

These metadata are about stimulus presentation, so they are likely relevant beyond eye tracking exclusively. Perhaps a reasonable alternative would be to update the specs so these metadata are encoded within stim.json files instead of the events.json files. Either way, they need to allow a json be defined without the corresponding .tsv[.gz] file, or allow empty tables (okay for .tsv but dangerous for .tsv.gz).

Ok, I'm of course fine with the solution that fits best to BIDS in general but I agree that we should keep this information somehow.

allow a json be defined without the corresponding .tsv[.gz] file

would probably the best solution for this particular BEP as we agreed on .tsv.gz

julia-pfarr · 2024-07-08T09:58:49Z

src/modality-specific-files/physiological-recordings.md

+```JSON
+{
+    "DeviceSerialNumber": "17535483",
+    "Columns": ["timestamp", "x_coordinate", "y_coordinate", "pupil_size"],


In the src/schema/objects/columns.yaml and src/schema/rules/tabular_data/physio.yaml it says "et_timestamp", here it says "timestamp", which is correct?

Both are correct, these are two different things:

src/schema/objects/columns.yaml, src/schema/rules/tabular_data/physio.yaml define a et_timestamp column name. This is so to allow the timestamp be different from the already existing timestamp column. This identifier is internal to the schema, so it does not propagate into actual column names.

the line 672 you commented on is just the necessary specification of the "Columns" metadata, the column name is arbitrary, and in this case it is named "timestamp" but other names would be equally supported.

src/modality-specific-files/physiological-recordings.md

Remi-Gau · 2024-07-27T07:15:04Z

src/schema/rules/tabular_data/physio.yaml

+  selectors:
+    - suffix == "physioevents"
+  columns:
+    onset: required


we need new definitions for onset and duration because now the one used here assumed that the unit is in seconds.

src/modality-specific-files/physiological-recordings.md

Remi-Gau · 2024-08-01T15:29:24Z

src/modality-specific-files/physiological-recordings.md

+`sub-01_task-visualSearch_recording-eye1_physio.json` sidecar
+could read:
+
+```JSON


insert in example here

What example is missing here?

Remi-Gau · 2024-08-02T13:43:28Z

Discussed previously during some meetings with @oesteban @mszinte

Note that if the source data for single run was acquired by turning the eyetracker only during trials instead of keeping the eyetracking recording for the whole duration of the run, this will lead to discontinuous timestamps.

@julia-pfarr and I are encountering the issue in some of the datasets we are converting.

The decision for now is to pad the output files with rows of NaNs for the missing time points.
Note that this technically "creates" timepoints that were not recorded and that it may inflate the size of the output file.

Wonder if this consequence of using physio data for this kind of eyetracking acquisition should be mentioned somewhere in the spec, or if this is more a converter implementation detail + best practice recommendations for data acquistion...

scott-huberty · 2024-08-02T14:39:58Z

Hi everyone,

Note that if the source data for single run was acquired by turning the eyetracker only during trials instead of keeping the eyetracking recording for the whole duration of the run, this will lead to discontinuous timestamps.

Frustratingly, this is at least somewhat common in EyeLink eytrackers. Eyelink will also stop recording any time you enter a calibration sequence.

The decision for now is to pad the output files with rows of NaNs for the missing time points. Note that this technically "creates" timepoints that were not recorded and that it may inflate the size of the output file.

This is exactly what we did in the eyelink reader in MNE. I was also unsure if it was the right thing to do.. It makes me feel a little better about that decision, seeing you all arrive to the same conclusion independently 🙂

Wonder if this consequence of using physio data for this kind of eyetracking acquisition should be mentioned somewhere in the spec, or if this is more a converter implementation detail + best practice recommendations for data acquistion...

Just a heads up that our decision to pad with NaNs in MNE has caused its fair share of headaches, especially with signal processing routines (e.g. if you filter your pupil size signal, one of those NaN's could obliterate the whole signal!).

Remi-Gau · 2024-08-11T10:07:40Z

src/schema/rules/sidecars/continuous.yaml

+  fields:
+    EnvironmentCoordinates: required
+    RecordedEye: required
+    SampleCoordinateUnits: required


Note that the unit may also described in the description of each column of the physio file so so this may conflict with also having it here, no?

qian-chu · 2024-08-22T09:50:36Z

Hi everyone thanks for the hard work for pushing this forward! I have a question/minor suggestion: when timestamp is among the columns, shouldn't we also encourage users to provide metadata about it? For example the unit (ms for Eyelink, ns for Pupil Labs) and the reference frame (UNIX time or time since system startup). The current example doesn't provide such info:

bids-specification/src/modality-specific-files/physiological-recordings.md

Lines 670 to 691 in 26c463a

    
           ```JSON 
        
           { 
        
               "DeviceSerialNumber": "17535483", 
        
               "Columns": ["timestamp", "x_coordinate", "y_coordinate", "pupil_size"], 
        
               "EnvironmentCoordinates": "top-left", 
        
               "Manufacturer": "SR-Research", 
        
               "ManufacturersModelName": "EYELINK II CL v4.56 Aug 18 2010", 
        
               "RecordedEye": "right", 
        
               "SampleCoordinateSystem": "gaze-on-screen", 
        
               "SampleCoordinateUnits": "pixel", 
        
               "SamplingFrequency": 1000, 
        
               "SoftwareVersion": "SREB1.10.1630 WIN32 LID:F2AE011 Mod:2017.04.21 15:19 CEST", 
        
               "ScreenAOIDefinition": [ 
        
                   "square", 
        
                   [100, 150, 300, 350] 
        
               ], 
        
               "pupil_size": { 
        
                   "Description": "Pupil area of the recorded eye as calculated by the eye-tracker in arbitrary units (see EyeLink's documentation for conversion).", 
        
                   "Units": "a.u." 
        
               } 
        
           } 
        
           ```

src/schema/objects/metadata.yaml

for more information, see https://pre-commit.ci

Remi-Gau · 2024-09-05T14:23:38Z

src/schema/objects/metadata.yaml

 EncodingTechnique:
  name: EncodingTechnique
  display_name: Encoding Technique
  description: |
    The encoding technique used during readout.
    For example, `"Cartesian"`, `"EPSI"`, `"Spiral"`,
    or `"Density-weighted concentric ring trajectory"`.
-  type: string


typo due to merge conflict resolution

Sourav-Kulkarni

Inputs from previous meeting for inserting details about Eyetracking setup 'geometry'

src/schema/objects/metadata.yaml

…ance

for more information, see https://pre-commit.ci

oesteban · 2024-10-03T11:24:48Z

The decision for now is to pad the output files with rows of NaNs for the missing time points.
Note that this technically "creates" timepoints that were not recorded and that it may inflate the size of the output file.

Yes, I would defer that to seeing the outcomes of Fix context.columns typescript issues legacy-validator#1796 and take the currently standard approach (n/a-filled rows)

mszinte

Good to me, thanks

arnodelorme · 2024-10-11T21:55:01Z

We are eager to release our HBN dataset (2000+ subjects with eye tracking). I have provided several candidates on another thread matching the proposed format. When can we expect the format finalized? Tx

mszinte · 2024-10-12T12:33:46Z

We are eager to release our HBN dataset (2000+ subjects with eye tracking). I have provided several candidates on another thread matching the proposed format. When can we expect the format finalized? Tx

To my opinion the BEP is ready, we are finishing examples and testing deeply our converter.
We hope very soon that we will ask for BIDS community review and go through the final steps before finalization.

I would then suggest you to take it as it is and keep an eye on this thread in case some reviewer makes important changes when we will open it to review.

Best,

Sourav-Kulkarni · 2024-11-14T10:49:52Z

src/schema/objects/columns.yaml

+  name: pupil_size
+  display_name: Pupil size
+  description: |
+    Pupil size or area of the recorded eye, in the


Are we recording the particular type of pupil size that's being recorded here ('diameter' v/s 'area')?

The data that comes out of Eyelink is always in 'arbitrary units' (Eyelink documentation: page 106, 4.4.3) and the actual data type is stored in the settings (Eyelink documentation: page 25 'Pupil Size Data').

chloepasturel and others added 10 commits October 1, 2021 12:15

create branch bep020

a73a9ea

Add datatype metadata and suffixes for eye tracking

b43e979

bep020 md

ac14d18

Update 10-eye-tracking.md

3316ef5

minor update md

033a948

minor updates md 2

971ac2a

final changes and working version with mkdocs

6236f83

add forgotten example

1d71719

final changes

131b75d

latest changes

b7f4d5d

mszinte requested a review from tsalo as a code owner June 15, 2022 12:09

guiomar reviewed Jun 15, 2022

View reviewed changes