Vendor Config YAML or Reconfigure Existing RecordLoaders

Vendor PSG systems (Alice 6, Domino, Noxturnal, Profusion, Sonata, etc.) use proprietary formats. Some vendors require extra steps to prepare the data for example export EDF files or annotations. Specific pre requisites and steps should be described in the specific record loader documentation. This guide describes how to generate a vendor config that specifies file paths, extensions, and record ID patterns, signal mapping and how to add it to your project configuration.

Same principles apply to reconfiguring existing record loaders. For example, if you have another nsrr dataset that uses the same file structure as an existing record loader.

Generate a vendor config

phases create --type vendor-config

The wizard asks:

  1. Which vendor (select from list) or type in custom recordloader id (e.g. nox or mesa)
  2. Dataset / config key (identifier for example my-nox or my-mesa)
  3. Database name (for example MyStudy)
  4. Path to your recordings
  5. Is the dataset complete (no new data is expected, for example if data collection is still ongoing, select False, this will trigger getRecordList to check for new records every time)
  6. Record ID pattern (regex) to extract record ID from file path (for example ".*/(.+)\\.edf" to extract recordId from /path/to/recordId.edf)

It writes configs/<vendor>.yml.

Config structure

All vendor configs follow the same pattern, this is an example of a custom alice vendor config:

loader:
  my-alice:
    baseloader: alice
    dataBase: MY-ALICE
    dataBaseVersion: 0.1.0

    filePath: data/MY-ALICE/
    dataIsFinal: True # no new data is expected (False if data is still being recorded)

    dataset:
      dataHandler:
        listFilter: null # (e.g. "acq" if all files contain acq in the name)
        idPattern: .*/(.*[^-T]).edf

    # the channels that should be extracted from the signal files
    # if a channel is not found a ChannelNotFoundError is raised, unless optional: True is set
    sourceChannels:
    - name: EEG F3-A2
      type: eeg
      alias: [F3-A2, F3:A2]  # alternative channel names to look for
      optional: true
    - name: EEG C3-A2
      type: eeg
      optional: true

    # derive channels, if some recordings have two separately recorded electrodes
    # combineChannels:
    #   - name: F3:A2
    #     combineType: derived
    #     type: eeg
    #     channels: [EEG F3, EEG A2]

alice-path: data/MY-ALICE/
useLoader: my-alice

Mapping channels

The sourceChannels parameter tells the loader which signals to extract from each recording file and how to normalise them across recordings. This is the main harmonization mechanism, enabling a complete dataset to expose the same channel names and types, regardless of the setup used.

Each entry supports:

  • name — canonical channel name exposed to the rest of the pipeline.
  • aliases — list of alternative names to search for in the file (e.g. [F3-A2, F3:A2]). The first match is used.
  • type — signal category: eeg, ecg, eog, emg, effort, spo2, airflow, etc. Used downstream for type-wise channel selection and semantic data creation.
  • optional — when true, recordings that lack this channel are still loaded. When false (the default), any recording missing the channel is skipped with a ChannelNotFoundError.
sourceChannels:
  - name: EEG C3-A2
    type: eeg
    aliases: [C3-A2, C3:A2]
  - name: EOG Left
    type: eog
    aliases: [EOG(L), EOG L, LEOG]
  - name: SpO2
    type: spo2
    optional: true # not all recordings have pulse oximetry

Combine channels

Sometimes a signal is split across two electrodes in some recordings (e.g. F3 and A2 recorded separately instead of F3-A2 as a differential). combineChannels lets you synthesize a new channel from existing ones so the rest of the pipeline sees a consistent signal.

Supported combineType values:

Type Description
derived Subtract second channel from first (ch1 - ch2). Typical use: re-reference EEG.
sum Add all listed channels together.
mean Average all listed channels.
select Select a channel from a list of channels
copy Copy a single channel into a new one
selectByQuality Select a channel based on its quality (signal.quality needs to filled) 
combineChannels:
  - name: EEG F3-A2
    combineType: derived          # F3 - A2
    type: eeg
    channels: [EEG F3, EEG A2]
  - name: Effort Total
    combineType: sum
    type: effort
    channels: [Effort Thorax, Effort Abdomen]
  - name: EOG Mean
    combineType: mean
    type: eog
    channels: [EOG Left, EOG Right]

Combined channels are created after alias resolution, so you can reference the canonical name from sourceChannels.

Adding to project.yaml

plugins:
  - pyPhasesRecordloader
  - pyPhasesRecordloaderAlice   # or whichever vendor
  - SleepHarmonizer

importAfter:
  - configs/alice.yml           # your generated vendor config

config:
  useLoader: my-alice
  alice-path: /recordings

Flat workflow

from phases import DefaultProject

project = DefaultProject.create("configs/alice.yml", [
    "pyPhasesRecordloader",
    "pyPhasesRecordloaderAlice",
    "SleepHarmonizer",
])
project.setConfig("alice-path", "/recordings")