Note
Go to the end to download the full example code.
Quickstart: Synthetic EEG Pipeline¶
This example walks through a complete NeuroDAGs pipeline using synthetically generated EEG data — no real dataset required.
We will:
Generate a synthetic multi-subject BrainVision dataset.
Define a pipeline in Python (preprocessing → spectral → band power).
Inspect the plan with a dry run.
Execute the pipeline.
Assemble results into a dataframe.
Plot band power across subjects.
Setup¶
Standard imports and a temporary working directory.
import tempfile
from pathlib import Path
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import yaml
from neurodags.datasets import generate_dummy_dataset
from neurodags.orchestrators import build_derivative_dataframe, iterate_derivative_pipeline, run_pipeline
WORKDIR = Path(tempfile.mkdtemp(prefix="neurodags_quickstart_"))
DATA_DIR = WORKDIR / "rawdata"
OUT_DIR = WORKDIR / "derivatives"
OUT_DIR.mkdir(parents=True, exist_ok=True)
print(f"Working directory: {WORKDIR}")
2026-05-15 18:28:04 [debug ] Registered derivative name=CheckLineFrequency
2026-05-15 18:28:04 [debug ] Registered derivative name=BasicPrep1
2026-05-15 18:28:04 [debug ] Registered derivative name=InterDerivativeDependence
2026-05-15 18:28:04 [debug ] Registered derivative name=SpectrumArrayWelch
2026-05-15 18:28:04 [debug ] Registered derivative name=SpectrumArrayMultitaper
2026-05-15 18:28:04 [debug ] Registered derivative name=MEEGMetadata
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/spectral.py:14: DeprecationWarning:
The `fooof` package is being deprecated and replaced by the `specparam` (spectral parameterization) package.
This version of `fooof` (1.1) is fully functional, but will not be further updated.
New projects are recommended to update to using `specparam` (see Changelog for details).
from fooof import FOOOF
Working directory: /tmp/neurodags_quickstart_mk8j_x5n
Step 1 — Generate Synthetic Dataset¶
generate_dummy_dataset() creates BrainVision trios
(.vhdr / .vmrk / .eeg) using 1/f^α (pink) noise to mimic realistic
EEG spectral characteristics.
We generate 3 subjects × 1 session at 200 Hz, 30 seconds each.
generate_dummy_dataset(
data_params={
"DATASET": "quickstart",
"PATTERN": "sub-%subject%/ses-%session%/sub-%subject%_ses-%session%_task-rest",
"NSUBS": 3,
"NSESSIONS": 1,
"NTASKS": 1,
"NACQS": 1,
"NRUNS": 1,
"PREFIXES": {
"subject": "S",
"session": "SE",
"task": "T",
"acquisition": "A",
"run": "R",
},
"ROOT": str(DATA_DIR),
},
generation_args={
"NCHANNELS": 8,
"SFREQ": 200.0,
"STOP": 30.0,
"NUMEVENTS": 10,
"random_state": 0,
},
)
source_files = sorted(DATA_DIR.rglob("*.vhdr"))
print(f"Generated {len(source_files)} source file(s):")
for f in source_files:
print(f" {f.relative_to(WORKDIR)}")
Creating RawArray with float64 data, n_channels=8, n_times=6000
Range : 0 ... 5999 = 0.000 ... 29.995 secs
Ready.
/home/runner/work/neurodags/neurodags/src/neurodags/datasets.py:703: RuntimeWarning: Encountered data in 'double' format. Converting to float32.
export_raw(fname=str(vhdr_path), raw=raw, fmt="brainvision", overwrite=True)
2026-05-15 18:28:11 [debug ] get_num_digits: called method=safe n=3
2026-05-15 18:28:11 [debug ] get_num_digits: computed (safe) digits=1 n=3
2026-05-15 18:28:11 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:11 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:11 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:11 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:11 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:11 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:11 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:11 [debug ] get_num_digits: computed (safe) digits=1 n=1
Generated 3 source file(s):
rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr
rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr
rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr
Step 2 — Datasets config as YAML¶
In real workflows, save this string to datasets.yml and point
load_configuration at that file for version-controlled, reproducible runs.
Paths are injected from Python so the notebook remains runnable.
DATASETS_YAML = f"""\
quickstart:
name: Quickstart
file_pattern: "{DATA_DIR / '**' / '*.vhdr'}"
derivatives_path: "{OUT_DIR}"
"""
datasets = yaml.safe_load(DATASETS_YAML)
print("Datasets:", list(datasets))
Datasets: ['quickstart']
Step 3 — Pipeline config as YAML¶
The pipeline below is defined entirely in YAML — the format used in
pipeline.yml files checked into version control.
This pipeline has three derivatives:
BasicPrep: band-pass filter → 2-second epochs.
Spectrum: Welch PSD on each epoch.
BandPower: relative power in δ, θ, α, β bands, averaged across epochs (
save: false— computed but not written to disk;for_dataframe: true— included in the aggregated dataframe).
PIPELINE_YAML = """\
mount_point: null
DerivativeDefinitions:
BasicPrep:
overwrite: false
nodes:
- id: 0
derivative: SourceFile
- id: 1
node: basic_preprocessing
args:
mne_object: id.0
filter_args: {l_freq: 1.0, h_freq: 80.0}
epoch_config: {duration: 2.0, overlap: 0.0}
Spectrum:
overwrite: false
nodes:
- id: 0
derivative: BasicPrep.fif
- id: 1
node: mne_spectrum_array
args:
meeg: id.0
method: welch
method_kwargs: {n_per_seg: 200}
BandPower:
save: false
for_dataframe: true
nodes:
- id: 0
derivative: Spectrum.nc
- id: 1
node: extract_data_var
args: {dataset_like: id.0, data_var: spectrum}
- id: 2
node: bandpower
args:
psd_like: id.1
relative: true
bands:
delta: [1.0, 4.0]
theta: [4.0, 8.0]
alpha: [8.0, 13.0]
beta: [13.0, 30.0]
- id: 3
node: aggregate_across_dimension
args: {xarray_data: id.2, dim: epochs, operation: mean}
DerivativeList:
- BasicPrep
- Spectrum
- BandPower
"""
pipeline_config = yaml.safe_load(PIPELINE_YAML)
pipeline_config["datasets"] = datasets # inject dynamic dataset paths
print("Pipeline defined with derivatives:", pipeline_config["DerivativeList"])
Pipeline defined with derivatives: ['BasicPrep', 'Spectrum', 'BandPower']
Step 4 — Dry Run¶
Inspect the execution plan for BasicPrep without running any computation.
The returned dataframe shows which outputs are cached and which would be computed.
plan = iterate_derivative_pipeline(pipeline_config, "BasicPrep", dry_run=True)
# 'plan' column contains per-step dicts — expand for display
steps = []
for _, row in plan.iterrows():
for step in row["plan"]:
steps.append({"file": row["file_path"].split("/")[-1], **step})
print(pd.DataFrame(steps)[["file", "id", "kind", "cached"]].to_string(index=False))
2026-05-15 18:28:11 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:11 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:11 [debug ] Registered derivative name=BandPower
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Starting derivative processing total_files=3
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [info ] Dry run: dataset=quickstart derivative=BasicPrep file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0 overwrite=False plan=[{'id': 'final', 'kind': 'derivative_output', 'name': 'BasicPrep', 'prefix': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep', 'cached': False, 'paths': [], 'has_error_marker': False, 'error_path': None}, {'id': 0, 'kind': 'source', 'name': 'SourceFile', 'path': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr'}] reference_base=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [info ] Dry run: dataset=quickstart derivative=BasicPrep file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1 overwrite=False plan=[{'id': 'final', 'kind': 'derivative_output', 'name': 'BasicPrep', 'prefix': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep', 'cached': False, 'paths': [], 'has_error_marker': False, 'error_path': None}, {'id': 0, 'kind': 'source', 'name': 'SourceFile', 'path': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr'}] reference_base=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Dry run: dataset=quickstart derivative=BasicPrep file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2 overwrite=False plan=[{'id': 'final', 'kind': 'derivative_output', 'name': 'BasicPrep', 'prefix': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep', 'cached': False, 'paths': [], 'has_error_marker': False, 'error_path': None}, {'id': 0, 'kind': 'source', 'name': 'SourceFile', 'path': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr'}] reference_base=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [info ] Completed derivative processing total_files=3
file id kind cached
sub-S1_ses-SE0_task-rest.vhdr final derivative_output False
sub-S1_ses-SE0_task-rest.vhdr 0 source NaN
sub-S0_ses-SE0_task-rest.vhdr final derivative_output False
sub-S0_ses-SE0_task-rest.vhdr 0 source NaN
sub-S2_ses-SE0_task-rest.vhdr final derivative_output False
sub-S2_ses-SE0_task-rest.vhdr 0 source NaN
Step 5 — Execute the Pipeline¶
run_pipeline runs all derivatives in DerivativeList, sorted by dependency order. Already-cached outputs are skipped automatically.
run_pipeline(pipeline_config, raise_on_error=True)
# List produced files
produced = sorted(OUT_DIR.rglob("*@*.fif")) + sorted(OUT_DIR.rglob("*@*.nc"))
print(f"\nProduced {len(produced)} derivative file(s):")
for f in produced:
print(f" {f.relative_to(WORKDIR)}")
2026-05-15 18:28:11 [info ] Derivative execution order order=['BasicPrep', 'Spectrum', 'BandPower']
2026-05-15 18:28:11 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:11 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:11 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BandPower'
2026-05-15 18:28:11 [debug ] Registered derivative name=BandPower
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Starting derivative processing total_files=3
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Execute node derivative=BasicPrep id=1 kwargs={'mne_object': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}} node=basic_preprocessing
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Raw file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<RawBrainVision | sub-S1_ses-SE0_task-rest.eeg, 8 x 6000 (30.0 s), ~388 KiB, data loaded>
2026-05-15 18:28:11 [debug ] Filter Applied filter_args={'l_freq': 1.0, 'h_freq': 80.0}
Not setting metadata
15 matching events found
No baseline correction applied
0 projection items activated
Using data from preloaded Raw for 15 events and 400 original time points ...
0 bad epochs dropped
2026-05-15 18:28:11 [debug ] EPOCH SEGMENTATION with make_fixed_length_epochs epoch_config={'duration': 2.0, 'overlap': 0.0}
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<Epochs | 15 events (all good), 0 – 1.995 s (baseline off), ~388 KiB, data loaded,
'1': 15>, writer=<function basic_preprocessing.<locals>.<lambda> at 0x7f517951b250>) name=.fif
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/preprocessing.py:143: RuntimeWarning: This filename (/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif) does not conform to MNE naming conventions. All epochs files should end with -epo.fif, -epo.fif.gz, _epo.fif or _epo.fif.gz
".fif": Artifact(item=mne_object, writer=lambda path: mne_object.save(path, overwrite=True))
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Execute node derivative=BasicPrep id=1 kwargs={'mne_object': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}} node=basic_preprocessing
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Raw file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<RawBrainVision | sub-S0_ses-SE0_task-rest.eeg, 8 x 6000 (30.0 s), ~388 KiB, data loaded>
2026-05-15 18:28:11 [debug ] Filter Applied filter_args={'l_freq': 1.0, 'h_freq': 80.0}
Not setting metadata
15 matching events found
No baseline correction applied
0 projection items activated
Using data from preloaded Raw for 15 events and 400 original time points ...
0 bad epochs dropped
2026-05-15 18:28:11 [debug ] EPOCH SEGMENTATION with make_fixed_length_epochs epoch_config={'duration': 2.0, 'overlap': 0.0}
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<Epochs | 15 events (all good), 0 – 1.995 s (baseline off), ~388 KiB, data loaded,
'1': 15>, writer=<function basic_preprocessing.<locals>.<lambda> at 0x7f517956d240>) name=.fif
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/preprocessing.py:143: RuntimeWarning: This filename (/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif) does not conform to MNE naming conventions. All epochs files should end with -epo.fif, -epo.fif.gz, _epo.fif or _epo.fif.gz
".fif": Artifact(item=mne_object, writer=lambda path: mne_object.save(path, overwrite=True))
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Execute node derivative=BasicPrep id=1 kwargs={'mne_object': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}} node=basic_preprocessing
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Raw file=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<RawBrainVision | sub-S2_ses-SE0_task-rest.eeg, 8 x 6000 (30.0 s), ~388 KiB, data loaded>
2026-05-15 18:28:11 [debug ] Filter Applied filter_args={'l_freq': 1.0, 'h_freq': 80.0}
Not setting metadata
15 matching events found
No baseline correction applied
0 projection items activated
Using data from preloaded Raw for 15 events and 400 original time points ...
0 bad epochs dropped
2026-05-15 18:28:11 [debug ] EPOCH SEGMENTATION with make_fixed_length_epochs epoch_config={'duration': 2.0, 'overlap': 0.0}
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<Epochs | 15 events (all good), 0 – 1.995 s (baseline off), ~388 KiB, data loaded,
'1': 15>, writer=<function basic_preprocessing.<locals>.<lambda> at 0x7f517956d240>) name=.fif
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/preprocessing.py:143: RuntimeWarning: This filename (/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif) does not conform to MNE naming conventions. All epochs files should end with -epo.fif, -epo.fif.gz, _epo.fif or _epo.fif.gz
".fif": Artifact(item=mne_object, writer=lambda path: mne_object.save(path, overwrite=True))
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=BasicPrep file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Completed derivative processing total_files=3
2026-05-15 18:28:11 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:11 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:11 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BandPower'
2026-05-15 18:28:11 [debug ] Registered derivative name=BandPower
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [debug ] Processing file dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Starting derivative processing total_files=3
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Using cached derivative child_derivative=BasicPrep.fif derivative=Spectrum id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif']
2026-05-15 18:28:11 [debug ] Execute node derivative=Spectrum id=1 kwargs={'meeg': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}} node=mne_spectrum_array
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Epochs file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<EpochsFIF | 15 events (all good), 0 – 1.995 s (baseline off), ~389 KiB, data loaded,
'1': 15>
Effective window size : 1.280 (s)
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<xarray.Dataset> Size: 125kB
Dimensions: (epochs: 15, spaces: 8, frequencies: 129)
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Data variables:
spectrum (epochs, spaces, frequencies) float64 124kB 0.004506 ... 2.2...
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., writer=<function mne_spectrum_array.<locals>.<lambda> at 0x7f517924d1b0>) name=.nc
<frozen importlib._bootstrap>:241: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@Spectrum.nc
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Using cached derivative child_derivative=BasicPrep.fif derivative=Spectrum id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif']
2026-05-15 18:28:11 [debug ] Execute node derivative=Spectrum id=1 kwargs={'meeg': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}} node=mne_spectrum_array
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Epochs file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<EpochsFIF | 15 events (all good), 0 – 1.995 s (baseline off), ~389 KiB, data loaded,
'1': 15>
Effective window size : 1.280 (s)
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<xarray.Dataset> Size: 125kB
Dimensions: (epochs: 15, spaces: 8, frequencies: 129)
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Data variables:
spectrum (epochs, spaces, frequencies) float64 124kB 0.004506 ... 2.2...
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., writer=<function mne_spectrum_array.<locals>.<lambda> at 0x7f5178704820>) name=.nc
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@Spectrum.nc
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Using cached derivative child_derivative=BasicPrep.fif derivative=Spectrum id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif']
2026-05-15 18:28:11 [debug ] Execute node derivative=Spectrum id=1 kwargs={'meeg': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}} node=mne_spectrum_array
2026-05-15 18:28:11 [debug ] Attempting to load MEEG file file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:11 [debug ] Loaded MEEG file as Epochs file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:11 [debug ] MNEReport: loaded MNE object from file input=<EpochsFIF | 15 events (all good), 0 – 1.995 s (baseline off), ~389 KiB, data loaded,
'1': 15>
Effective window size : 1.280 (s)
2026-05-15 18:28:11 [debug ] Processed artifact file=Artifact(item=<xarray.Dataset> Size: 125kB
Dimensions: (epochs: 15, spaces: 8, frequencies: 129)
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Data variables:
spectrum (epochs, spaces, frequencies) float64 124kB 0.004506 ... 2.2...
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., writer=<function mne_spectrum_array.<locals>.<lambda> at 0x7f5178704b80>) name=.nc
2026-05-15 18:28:11 [debug ] Saved artifact file=/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@Spectrum.nc
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=Spectrum file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=Spectrum file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:11 [info ] Processed file successfully dataset=quickstart derivative=Spectrum file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
2026-05-15 18:28:11 [info ] Completed derivative processing total_files=3
2026-05-15 18:28:11 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:11 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:11 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BandPower'
2026-05-15 18:28:11 [debug ] Registered derivative name=BandPower
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [info ] Derivative is marked with save=False; skipping execution. derivative=BandPower
Produced 6 derivative file(s):
derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@BasicPrep.fif
derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@BasicPrep.fif
derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@BasicPrep.fif
derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@Spectrum.nc
derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@Spectrum.nc
derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@Spectrum.nc
Step 6 — Assemble Dataframe¶
build_derivative_dataframe() collects every
for_dataframe=True derivative into a single dataframe.
output_format="wide" gives one row per file with derivative columns.
df = build_derivative_dataframe(pipeline_config, output_format="wide")
# Extract readable subject labels from the file path
df["subject"] = df["file_path"].apply(
lambda p: next(
(part for part in Path(p).parts if part.startswith("sub-")),
Path(p).stem.split("_")[0],
)
)
print(f"DataFrame shape: {df.shape}")
print(df.head())
2026-05-15 18:28:11 [debug ] build_derivative_dataframe: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:11 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:11 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:11 [info ] Overriding existing derivative registration for 'BandPower'
2026-05-15 18:28:11 [debug ] Registered derivative name=BandPower
2026-05-15 18:28:11 [warning ] Some requested derivatives are either undefined or flagged out of dataframe collection. missing_derivatives=['BasicPrep', 'Spectrum']
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPower': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'aggregate_across_dimension', 'args': {'xarray_data': 'id.2', 'dim': 'epochs', 'operation': 'mean'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPower'], 'datasets': {'quickstart': {'name': 'Quickstart', 'file_pattern': '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives'}}}
2026-05-15 18:28:11 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:11 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: processing dataset dataset=quickstart
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: resolved pattern dataset=quickstart pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr
2026-05-15 18:28:11 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:11 [debug ] get_files_from_pattern: found files count=3
2026-05-15 18:28:11 [debug ] find_unique_root: called mode=maximal n_paths=3 strict=True style=auto
2026-05-15 18:28:11 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:11 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']
2026-05-15 18:28:11 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_quickstart_mk8j_x5n/rawdata prefix_len=4
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:11 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_quickstart_mk8j_x5n/rawdata dataset=quickstart
2026-05-15 18:28:11 [info ] Found files in dataset dataset=quickstart file_count=3
2026-05-15 18:28:11 [debug ] get_all_files_across_datasets: dataset summary dataset=quickstart file_count=3
2026-05-15 18:28:11 [info ] File discovery complete total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=3
2026-05-15 18:28:11 [debug ] build_derivative_dataframe: enumerated files per_dataset={'quickstart': ['/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr', '/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr']} total_files=3
2026-05-15 18:28:11 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:11 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:11 [debug ] Using cached derivative child_derivative=Spectrum.nc derivative=BandPower id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@Spectrum.nc']
2026-05-15 18:28:11 [debug ] Execute node derivative=BandPower id=1 kwargs={'dataset_like': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr@Spectrum.nc', 'data_var': 'spectrum'} node=extract_data_var
2026-05-15 18:28:11 [debug ] Execute node derivative=BandPower id=2 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, frequencies: 129)> Size: 124kB
array([[[4.50562839e-03, 3.00017237e-02, 3.73740202e-02, ...,
6.33255476e-07, 4.75953066e-07, 7.47351194e-08],
[4.19434507e-03, 8.16999531e-02, 1.28448183e-01, ...,
9.74247847e-07, 8.40125408e-07, 3.71560301e-07],
[4.75255554e-03, 9.79946924e-03, 7.20194067e-03, ...,
3.84532435e-06, 8.47464634e-07, 4.25811252e-07],
...,
[1.19417748e-02, 4.73911519e-02, 1.30273674e-01, ...,
5.06215297e-07, 1.42572388e-06, 5.06666679e-07],
[3.99821386e-02, 1.38817426e-01, 1.21510613e-01, ...,
1.01979676e-06, 1.00495312e-06, 3.58637182e-07],
[1.21488698e-02, 4.46953426e-02, 5.55970143e-02, ...,
5.96812592e-07, 5.23342732e-08, 4.28863010e-07]],
[[1.11855833e-03, 7.93416476e-03, 5.09333534e-02, ...,
6.55122921e-08, 9.96323295e-07, 1.38601655e-08],
[2.27320662e-02, 9.83927650e-02, 1.69744077e-01, ...,
1.50235314e-06, 1.77985831e-06, 9.33300971e-07],
[9.59004824e-03, 8.74455423e-02, 1.87491544e-01, ...,
7.13972387e-07, 3.90718693e-07, 1.26944072e-07],
...
2.47211784e-06, 3.83235870e-07, 5.01137656e-07],
[3.38865821e-04, 4.45636170e-02, 1.21216731e-01, ...,
1.83501601e-06, 7.93533393e-07, 2.12560024e-07],
[2.32119803e-03, 5.33628837e-02, 2.87100071e-02, ...,
5.80230114e-07, 6.34263377e-07, 3.12647414e-10]],
[[3.29091696e-02, 1.10200855e-01, 7.60528899e-02, ...,
1.30662728e-06, 1.47181363e-07, 8.39246276e-08],
[5.68429892e-03, 3.63735275e-02, 7.38519828e-02, ...,
2.26326160e-07, 3.22813831e-07, 1.02982775e-07],
[8.86326984e-03, 5.48196652e-02, 7.91756554e-02, ...,
6.94771282e-07, 3.42595005e-07, 4.54115841e-08],
...,
[4.94330070e-03, 1.32150970e-02, 1.29230562e-02, ...,
2.59402675e-07, 6.47486044e-07, 1.96833344e-07],
[1.01590322e-02, 6.90213905e-02, 6.25004954e-02, ...,
1.03636771e-06, 6.46046061e-07, 3.26391393e-07],
[1.98166968e-03, 5.56990634e-03, 1.51266606e-02, ...,
2.84339628e-07, 5.19674156e-07, 2.28324558e-07]]],
shape=(15, 8, 129))
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=3 kwargs={'xarray_data': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, freqbands: 4)> Size: 4kB
array([[[0.12155715, 0.08389754, 0.10686738, 0.17452899],
[0.29593736, 0.10298961, 0.02808222, 0.16312418],
[0.13679686, 0.09818022, 0.13942352, 0.16535606],
[0.22889142, 0.22802793, 0.03311656, 0.11488168],
[0.20650679, 0.14521457, 0.08952555, 0.18535108],
[0.48400203, 0.04185406, 0.03014486, 0.09707228],
[0.15757469, 0.22367204, 0.02705151, 0.13638396],
[0.09969185, 0.13732839, 0.09184263, 0.19090389]],
[[0.19771592, 0.13560975, 0.05114324, 0.20581776],
[0.26331199, 0.13934103, 0.07157482, 0.1375238 ],
[0.30139536, 0.05715019, 0.05107924, 0.16405959],
[0.28792239, 0.07593009, 0.06208031, 0.139308 ],
[0.16669384, 0.13201092, 0.06446639, 0.26945034],
[0.17136113, 0.1068856 , 0.12666634, 0.16772708],
[0.23451017, 0.08753624, 0.04445412, 0.16827315],
[0.2326632 , 0.0929694 , 0.02813031, 0.18610014]],
[[0.36804995, 0.08295099, 0.07112746, 0.08690618],
[0.20491496, 0.09362325, 0.07990521, 0.11539219],
...
[0.13427379, 0.05870769, 0.09319055, 0.21455095],
[0.21351099, 0.08017494, 0.05950294, 0.21150186]],
[[0.19909505, 0.04528845, 0.08544133, 0.24642786],
[0.13955296, 0.19685678, 0.10155346, 0.21061889],
[0.19931472, 0.10281904, 0.11703 , 0.13793061],
[0.28150877, 0.0639088 , 0.0836968 , 0.1264223 ],
[0.19227104, 0.03025766, 0.08174708, 0.21542097],
[0.05794862, 0.20055168, 0.06545819, 0.17670689],
[0.27832403, 0.08383941, 0.09806965, 0.15764263],
[0.06773398, 0.20779299, 0.11906313, 0.15617599]],
[[0.2465181 , 0.13018419, 0.03543038, 0.11327196],
[0.16984439, 0.23234316, 0.06285869, 0.17628817],
[0.18673672, 0.09389142, 0.07947301, 0.2094921 ],
[0.17946178, 0.10173618, 0.0605435 , 0.19317134],
[0.21557484, 0.10374536, 0.11103299, 0.15900552],
[0.10370317, 0.14593028, 0.06370123, 0.22555358],
[0.16000837, 0.0405891 , 0.14753095, 0.15238742],
[0.22760439, 0.11084621, 0.11825236, 0.18225933]]])
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 80B 'delta' 'theta' 'alpha' 'beta'
freqband_low (freqbands) float64 32B 1.0 4.0 8.0 13.0
freqband_high (freqbands) float64 32B 4.0 8.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig..., 'dim': 'epochs', 'operation': 'mean'} node=aggregate_across_dimension
2026-05-15 18:28:12 [debug ] Collected dataframe row collected_derivatives=1 dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S1/ses-SE0/sub-S1_ses-SE0_task-rest.vhdr index=0
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:12 [debug ] Using cached derivative child_derivative=Spectrum.nc derivative=BandPower id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@Spectrum.nc']
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=1 kwargs={'dataset_like': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr@Spectrum.nc', 'data_var': 'spectrum'} node=extract_data_var
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=2 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, frequencies: 129)> Size: 124kB
array([[[4.50562839e-03, 3.00017237e-02, 3.73740202e-02, ...,
6.33255476e-07, 4.75953066e-07, 7.47351194e-08],
[4.19434507e-03, 8.16999531e-02, 1.28448183e-01, ...,
9.74247847e-07, 8.40125408e-07, 3.71560301e-07],
[4.75255554e-03, 9.79946924e-03, 7.20194067e-03, ...,
3.84532435e-06, 8.47464634e-07, 4.25811252e-07],
...,
[1.19417748e-02, 4.73911519e-02, 1.30273674e-01, ...,
5.06215297e-07, 1.42572388e-06, 5.06666679e-07],
[3.99821386e-02, 1.38817426e-01, 1.21510613e-01, ...,
1.01979676e-06, 1.00495312e-06, 3.58637182e-07],
[1.21488698e-02, 4.46953426e-02, 5.55970143e-02, ...,
5.96812592e-07, 5.23342732e-08, 4.28863010e-07]],
[[1.11855833e-03, 7.93416476e-03, 5.09333534e-02, ...,
6.55122921e-08, 9.96323295e-07, 1.38601655e-08],
[2.27320662e-02, 9.83927650e-02, 1.69744077e-01, ...,
1.50235314e-06, 1.77985831e-06, 9.33300971e-07],
[9.59004824e-03, 8.74455423e-02, 1.87491544e-01, ...,
7.13972387e-07, 3.90718693e-07, 1.26944072e-07],
...
2.47211784e-06, 3.83235870e-07, 5.01137656e-07],
[3.38865821e-04, 4.45636170e-02, 1.21216731e-01, ...,
1.83501601e-06, 7.93533393e-07, 2.12560024e-07],
[2.32119803e-03, 5.33628837e-02, 2.87100071e-02, ...,
5.80230114e-07, 6.34263377e-07, 3.12647414e-10]],
[[3.29091696e-02, 1.10200855e-01, 7.60528899e-02, ...,
1.30662728e-06, 1.47181363e-07, 8.39246276e-08],
[5.68429892e-03, 3.63735275e-02, 7.38519828e-02, ...,
2.26326160e-07, 3.22813831e-07, 1.02982775e-07],
[8.86326984e-03, 5.48196652e-02, 7.91756554e-02, ...,
6.94771282e-07, 3.42595005e-07, 4.54115841e-08],
...,
[4.94330070e-03, 1.32150970e-02, 1.29230562e-02, ...,
2.59402675e-07, 6.47486044e-07, 1.96833344e-07],
[1.01590322e-02, 6.90213905e-02, 6.25004954e-02, ...,
1.03636771e-06, 6.46046061e-07, 3.26391393e-07],
[1.98166968e-03, 5.56990634e-03, 1.51266606e-02, ...,
2.84339628e-07, 5.19674156e-07, 2.28324558e-07]]],
shape=(15, 8, 129))
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=3 kwargs={'xarray_data': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, freqbands: 4)> Size: 4kB
array([[[0.12155715, 0.08389754, 0.10686738, 0.17452899],
[0.29593736, 0.10298961, 0.02808222, 0.16312418],
[0.13679686, 0.09818022, 0.13942352, 0.16535606],
[0.22889142, 0.22802793, 0.03311656, 0.11488168],
[0.20650679, 0.14521457, 0.08952555, 0.18535108],
[0.48400203, 0.04185406, 0.03014486, 0.09707228],
[0.15757469, 0.22367204, 0.02705151, 0.13638396],
[0.09969185, 0.13732839, 0.09184263, 0.19090389]],
[[0.19771592, 0.13560975, 0.05114324, 0.20581776],
[0.26331199, 0.13934103, 0.07157482, 0.1375238 ],
[0.30139536, 0.05715019, 0.05107924, 0.16405959],
[0.28792239, 0.07593009, 0.06208031, 0.139308 ],
[0.16669384, 0.13201092, 0.06446639, 0.26945034],
[0.17136113, 0.1068856 , 0.12666634, 0.16772708],
[0.23451017, 0.08753624, 0.04445412, 0.16827315],
[0.2326632 , 0.0929694 , 0.02813031, 0.18610014]],
[[0.36804995, 0.08295099, 0.07112746, 0.08690618],
[0.20491496, 0.09362325, 0.07990521, 0.11539219],
...
[0.13427379, 0.05870769, 0.09319055, 0.21455095],
[0.21351099, 0.08017494, 0.05950294, 0.21150186]],
[[0.19909505, 0.04528845, 0.08544133, 0.24642786],
[0.13955296, 0.19685678, 0.10155346, 0.21061889],
[0.19931472, 0.10281904, 0.11703 , 0.13793061],
[0.28150877, 0.0639088 , 0.0836968 , 0.1264223 ],
[0.19227104, 0.03025766, 0.08174708, 0.21542097],
[0.05794862, 0.20055168, 0.06545819, 0.17670689],
[0.27832403, 0.08383941, 0.09806965, 0.15764263],
[0.06773398, 0.20779299, 0.11906313, 0.15617599]],
[[0.2465181 , 0.13018419, 0.03543038, 0.11327196],
[0.16984439, 0.23234316, 0.06285869, 0.17628817],
[0.18673672, 0.09389142, 0.07947301, 0.2094921 ],
[0.17946178, 0.10173618, 0.0605435 , 0.19317134],
[0.21557484, 0.10374536, 0.11103299, 0.15900552],
[0.10370317, 0.14593028, 0.06370123, 0.22555358],
[0.16000837, 0.0405891 , 0.14753095, 0.15238742],
[0.22760439, 0.11084621, 0.11825236, 0.18225933]]])
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 80B 'delta' 'theta' 'alpha' 'beta'
freqband_low (freqbands) float64 32B 1.0 4.0 8.0 13.0
freqband_high (freqbands) float64 32B 4.0 8.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig..., 'dim': 'epochs', 'operation': 'mean'} node=aggregate_across_dimension
2026-05-15 18:28:12 [debug ] Collected dataframe row collected_derivatives=1 dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S0/ses-SE0/sub-S0_ses-SE0_task-rest.vhdr index=1
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_quickstart_mk8j_x5n/derivatives
2026-05-15 18:28:12 [debug ] Using cached derivative child_derivative=Spectrum.nc derivative=BandPower id=0 paths=['/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@Spectrum.nc']
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=1 kwargs={'dataset_like': '/tmp/neurodags_quickstart_mk8j_x5n/derivatives/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr@Spectrum.nc', 'data_var': 'spectrum'} node=extract_data_var
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=2 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, frequencies: 129)> Size: 124kB
array([[[4.50562839e-03, 3.00017237e-02, 3.73740202e-02, ...,
6.33255476e-07, 4.75953066e-07, 7.47351194e-08],
[4.19434507e-03, 8.16999531e-02, 1.28448183e-01, ...,
9.74247847e-07, 8.40125408e-07, 3.71560301e-07],
[4.75255554e-03, 9.79946924e-03, 7.20194067e-03, ...,
3.84532435e-06, 8.47464634e-07, 4.25811252e-07],
...,
[1.19417748e-02, 4.73911519e-02, 1.30273674e-01, ...,
5.06215297e-07, 1.42572388e-06, 5.06666679e-07],
[3.99821386e-02, 1.38817426e-01, 1.21510613e-01, ...,
1.01979676e-06, 1.00495312e-06, 3.58637182e-07],
[1.21488698e-02, 4.46953426e-02, 5.55970143e-02, ...,
5.96812592e-07, 5.23342732e-08, 4.28863010e-07]],
[[1.11855833e-03, 7.93416476e-03, 5.09333534e-02, ...,
6.55122921e-08, 9.96323295e-07, 1.38601655e-08],
[2.27320662e-02, 9.83927650e-02, 1.69744077e-01, ...,
1.50235314e-06, 1.77985831e-06, 9.33300971e-07],
[9.59004824e-03, 8.74455423e-02, 1.87491544e-01, ...,
7.13972387e-07, 3.90718693e-07, 1.26944072e-07],
...
2.47211784e-06, 3.83235870e-07, 5.01137656e-07],
[3.38865821e-04, 4.45636170e-02, 1.21216731e-01, ...,
1.83501601e-06, 7.93533393e-07, 2.12560024e-07],
[2.32119803e-03, 5.33628837e-02, 2.87100071e-02, ...,
5.80230114e-07, 6.34263377e-07, 3.12647414e-10]],
[[3.29091696e-02, 1.10200855e-01, 7.60528899e-02, ...,
1.30662728e-06, 1.47181363e-07, 8.39246276e-08],
[5.68429892e-03, 3.63735275e-02, 7.38519828e-02, ...,
2.26326160e-07, 3.22813831e-07, 1.02982775e-07],
[8.86326984e-03, 5.48196652e-02, 7.91756554e-02, ...,
6.94771282e-07, 3.42595005e-07, 4.54115841e-08],
...,
[4.94330070e-03, 1.32150970e-02, 1.29230562e-02, ...,
2.59402675e-07, 6.47486044e-07, 1.96833344e-07],
[1.01590322e-02, 6.90213905e-02, 6.25004954e-02, ...,
1.03636771e-06, 6.46046061e-07, 3.26391393e-07],
[1.98166968e-03, 5.56990634e-03, 1.51266606e-02, ...,
2.84339628e-07, 5.19674156e-07, 2.28324558e-07]]],
shape=(15, 8, 129))
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': True, 'bands': {'delta': [1.0, 4.0], 'theta': [4.0, 8.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPower id=3 kwargs={'xarray_data': <xarray.DataArray 'spectrum' (epochs: 15, spaces: 8, freqbands: 4)> Size: 4kB
array([[[0.12155715, 0.08389754, 0.10686738, 0.17452899],
[0.29593736, 0.10298961, 0.02808222, 0.16312418],
[0.13679686, 0.09818022, 0.13942352, 0.16535606],
[0.22889142, 0.22802793, 0.03311656, 0.11488168],
[0.20650679, 0.14521457, 0.08952555, 0.18535108],
[0.48400203, 0.04185406, 0.03014486, 0.09707228],
[0.15757469, 0.22367204, 0.02705151, 0.13638396],
[0.09969185, 0.13732839, 0.09184263, 0.19090389]],
[[0.19771592, 0.13560975, 0.05114324, 0.20581776],
[0.26331199, 0.13934103, 0.07157482, 0.1375238 ],
[0.30139536, 0.05715019, 0.05107924, 0.16405959],
[0.28792239, 0.07593009, 0.06208031, 0.139308 ],
[0.16669384, 0.13201092, 0.06446639, 0.26945034],
[0.17136113, 0.1068856 , 0.12666634, 0.16772708],
[0.23451017, 0.08753624, 0.04445412, 0.16827315],
[0.2326632 , 0.0929694 , 0.02813031, 0.18610014]],
[[0.36804995, 0.08295099, 0.07112746, 0.08690618],
[0.20491496, 0.09362325, 0.07990521, 0.11539219],
...
[0.13427379, 0.05870769, 0.09319055, 0.21455095],
[0.21351099, 0.08017494, 0.05950294, 0.21150186]],
[[0.19909505, 0.04528845, 0.08544133, 0.24642786],
[0.13955296, 0.19685678, 0.10155346, 0.21061889],
[0.19931472, 0.10281904, 0.11703 , 0.13793061],
[0.28150877, 0.0639088 , 0.0836968 , 0.1264223 ],
[0.19227104, 0.03025766, 0.08174708, 0.21542097],
[0.05794862, 0.20055168, 0.06545819, 0.17670689],
[0.27832403, 0.08383941, 0.09806965, 0.15764263],
[0.06773398, 0.20779299, 0.11906313, 0.15617599]],
[[0.2465181 , 0.13018419, 0.03543038, 0.11327196],
[0.16984439, 0.23234316, 0.06285869, 0.17628817],
[0.18673672, 0.09389142, 0.07947301, 0.2094921 ],
[0.17946178, 0.10173618, 0.0605435 , 0.19317134],
[0.21557484, 0.10374536, 0.11103299, 0.15900552],
[0.10370317, 0.14593028, 0.06370123, 0.22555358],
[0.16000837, 0.0405891 , 0.14753095, 0.15238742],
[0.22760439, 0.11084621, 0.11825236, 0.18225933]]])
Coordinates:
* epochs (epochs) int64 120B 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 80B 'delta' 'theta' 'alpha' 'beta'
freqband_low (freqbands) float64 32B 1.0 4.0 8.0 13.0
freqband_high (freqbands) float64 32B 4.0 8.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig..., 'dim': 'epochs', 'operation': 'mean'} node=aggregate_across_dimension
2026-05-15 18:28:12 [debug ] Collected dataframe row collected_derivatives=1 dataset=quickstart file_path=/tmp/neurodags_quickstart_mk8j_x5n/rawdata/sub-S2/ses-SE0/sub-S2_ses-SE0_task-rest.vhdr index=2
DataFrame shape: (3, 36)
index dataset ... BandPower.nc@freqbands-beta_spaces-EEG007 subject
0 0 quickstart ... 0.165119 sub-S1
1 1 quickstart ... 0.165119 sub-S0
2 2 quickstart ... 0.165119 sub-S2
[3 rows x 36 columns]
Step 7 — Visualise Band Power¶
Group by subject and plot mean relative band power per frequency band.
band_cols = [c for c in df.columns if any(b in c for b in ["delta", "theta", "alpha", "beta"])]
if band_cols:
# Melt to long form for plotting
df_long = df[["subject", *band_cols]].melt(
id_vars="subject", var_name="band_channel", value_name="relative_power"
)
# Extract band name from column label
df_long["band"] = df_long["band_channel"].apply(
lambda x: next((b for b in ["delta", "theta", "alpha", "beta"] if b in x), None)
)
band_means = df_long.groupby(["subject", "band"])["relative_power"].mean().reset_index()
bands = ["delta", "theta", "alpha", "beta"]
band_means = band_means[band_means["band"].isin(bands)]
subjects = sorted(band_means["subject"].unique())
x = np.arange(len(bands))
width = 0.8 / len(subjects)
fig, ax = plt.subplots(figsize=(8, 4))
for i, sub in enumerate(subjects):
vals = [
band_means.loc[
(band_means["subject"] == sub) & (band_means["band"] == b), "relative_power"
].mean()
for b in bands
]
ax.bar(x + i * width, vals, width=width, label=sub)
ax.set_xticks(x + width * (len(subjects) - 1) / 2)
ax.set_xticklabels(bands)
ax.set_ylabel("Relative Power")
ax.set_title("Mean Relative Band Power per Subject")
ax.legend(title="Subject")
plt.tight_layout()
plt.savefig(WORKDIR / "band_power.png", dpi=100)
plt.show()
print(f"Plot saved to {WORKDIR / 'band_power.png'}")
else:
print("No band power columns found in dataframe.")
Plot saved to /tmp/neurodags_quickstart_mk8j_x5n/band_power.png
What’s Next¶
Swap
generate_dummy_datasetfor real BIDS data by pointingfile_patternat your raw EEG files.Save
PIPELINE_YAML/DATASETS_YAMLtopipeline.ymlanddatasets.ymlfor version-controlled, reproducible workflows.Run the same workflow from the CLI with commands such as
neurodags validate pipeline.yml,neurodags dry-run pipeline.yml --derivative BasicPrep, andneurodags run pipeline.yml.Add custom nodes via
new_definitions: my_nodes.py.Scale up: set
n_jobs=-1for file-level parallelism via joblib.Inspect any
.ncfile interactively with the built-in Dash explorer:neurodags view path/to/file.nc
Total running time of the script: (0 minutes 8.649 seconds)