Note
Go to the end to download the full example code.
DAG Capabilities: Branching and Multi-Input Nodes¶
This example demonstrates the core DAG structural patterns in NeuroDAGs:
Linear chain — nodes executing sequentially (BasicPrep → Spectrum)
Fan-out — one node’s output feeds two independent branches
Fan-in — a single node that depends on two previous nodes
YAML configuration — pipeline defined as a readable YAML string
Custom inline nodes — register your own node without a separate file
Pipeline graph:
SourceFile
│
BasicPrep (linear chain)
│
Spectrum
│
id.1: extract_data_var
╱ ╲
id.2 id.3 ← fan-out: same upstream, two parallel branches
(abs) (rel)
╲ ╱
id.4: concat_bandpower ← fan-in: depends on BOTH id.2 and id.3
Setup¶
import tempfile
from pathlib import Path
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
import yaml
from neurodags.datasets import generate_dummy_dataset
from neurodags.definitions import Artifact, NodeResult
from neurodags.nodes import register_node
from neurodags.orchestrators import build_derivative_dataframe, run_pipeline
WORKDIR = Path(tempfile.mkdtemp(prefix="neurodags_dag_"))
DATA_DIR = WORKDIR / "rawdata"
OUT_DIR = WORKDIR / "derivatives"
OUT_DIR.mkdir(parents=True, exist_ok=True)
print(f"Working directory: {WORKDIR}")
Working directory: /tmp/neurodags_dag_pdd4qt1z
Custom node — the fan-in (two-input) example¶
Register a node that accepts two upstream DataArrays and
concatenates them along a new normalization coordinate.
This is the key pattern: one node, two predecessor branches.
@register_node(name="concat_bandpower", override=True)
def concat_bandpower(absolute, relative) -> NodeResult:
"""Stack absolute and relative band power along a new 'normalization' axis."""
if isinstance(absolute, NodeResult):
absolute = absolute.artifacts[".nc"].item
if isinstance(relative, NodeResult):
relative = relative.artifacts[".nc"].item
combined = xr.concat(
[absolute, relative],
dim=xr.DataArray(["absolute", "relative"], dims="normalization"),
)
combined.name = "bandpower"
return NodeResult(
artifacts={".nc": Artifact(item=combined, writer=lambda p: combined.to_netcdf(p))}
)
Step 1 — Generate Synthetic Dataset¶
Two subjects, 10 seconds each — enough for a branching-pipeline demo.
generate_dummy_dataset(
data_params={
"DATASET": "dag_demo",
"PATTERN": "sub-%subject%/sub-%subject%_task-rest",
"NSUBS": 2,
"NSESSIONS": 1,
"NTASKS": 1,
"NACQS": 1,
"NRUNS": 1,
"PREFIXES": {
"subject": "S",
"session": "SE",
"task": "T",
"acquisition": "A",
"run": "R",
},
"ROOT": str(DATA_DIR),
},
generation_args={
"NCHANNELS": 8,
"SFREQ": 200.0,
"STOP": 10.0,
"NUMEVENTS": 5,
"random_state": 0,
},
)
source_files = sorted(DATA_DIR.rglob("*.vhdr"))
print(f"Generated {len(source_files)} source file(s)")
Creating RawArray with float64 data, n_channels=8, n_times=2000
Range : 0 ... 1999 = 0.000 ... 9.995 secs
Ready.
/home/runner/work/neurodags/neurodags/src/neurodags/datasets.py:703: RuntimeWarning: Encountered data in 'double' format. Converting to float32.
export_raw(fname=str(vhdr_path), raw=raw, fmt="brainvision", overwrite=True)
2026-05-15 18:28:12 [debug ] get_num_digits: called method=safe n=2
2026-05-15 18:28:12 [debug ] get_num_digits: computed (safe) digits=1 n=2
2026-05-15 18:28:12 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:12 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:12 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:12 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:12 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:12 [debug ] get_num_digits: computed (safe) digits=1 n=1
2026-05-15 18:28:12 [debug ] get_num_digits: called method=safe n=1
2026-05-15 18:28:12 [debug ] get_num_digits: computed (safe) digits=1 n=1
Generated 2 source file(s)
Step 2 — Datasets config as YAML¶
Defining datasets in YAML keeps the configuration version-controlled
and separate from code. For reproducible workflows, save this string
to a .yml file and point load_configuration at it.
DATASETS_YAML = f"""\
dag_demo:
name: DAG Demo
file_pattern: "{DATA_DIR / '**' / '*.vhdr'}"
derivatives_path: "{OUT_DIR}"
"""
datasets = yaml.safe_load(DATASETS_YAML)
print("Datasets:", list(datasets))
Datasets: ['dag_demo']
Step 3 — Pipeline config as YAML¶
The BandPowerBoth derivative shows all three DAG patterns:
id.0 loads the cached Spectrum artifact (cross-derivative dependency)
id.1 extracts the spectrum array (linear step)
id.2 and id.3 both read from id.1 → fan-out
id.4 reads from both id.2 and id.3 → fan-in
PIPELINE_YAML = """\
mount_point: null
DerivativeDefinitions:
# ── 1. Linear chain ─────────────────────────────────────────────────────
BasicPrep:
overwrite: false
nodes:
- id: 0
derivative: SourceFile
- id: 1
node: basic_preprocessing
args:
mne_object: id.0
filter_args: {l_freq: 1.0, h_freq: 80.0}
epoch_config: {duration: 2.0, overlap: 0.0}
Spectrum:
overwrite: false
nodes:
- id: 0
derivative: BasicPrep.fif
- id: 1
node: mne_spectrum_array
args:
meeg: id.0
method: welch
method_kwargs: {n_per_seg: 200}
# ── 2. Fan-out then fan-in ───────────────────────────────────────────────
BandPowerBoth:
save: false # computed on-the-fly; not written to disk
for_dataframe: true
nodes:
- id: 0
derivative: Spectrum.nc # load cached cross-derivative result
- id: 1 # shared upstream for both branches
node: extract_data_var
args: {dataset_like: id.0, data_var: spectrum}
- id: 2 # branch A — absolute power (fan-out from id.1)
node: bandpower
args:
psd_like: id.1
relative: false
bands:
delta: [1.0, 4.0]
alpha: [8.0, 13.0]
beta: [13.0, 30.0]
- id: 3 # branch B — relative power (fan-out from id.1)
node: bandpower
args:
psd_like: id.1
relative: true
bands:
delta: [1.0, 4.0]
alpha: [8.0, 13.0]
beta: [13.0, 30.0]
- id: 4 # fan-in: depends on BOTH id.2 (abs) and id.3 (rel)
node: concat_bandpower
args:
absolute: id.2
relative: id.3
DerivativeList:
- BasicPrep
- Spectrum
- BandPowerBoth
"""
pipeline_config = yaml.safe_load(PIPELINE_YAML)
pipeline_config["datasets"] = datasets
print("Pipeline derivatives:", pipeline_config["DerivativeList"])
Pipeline derivatives: ['BasicPrep', 'Spectrum', 'BandPowerBoth']
Step 4 — Visualise the DAG structure¶
Draw the node-level graph for BandPowerBoth before executing anything.
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# Left: cross-derivative pipeline DAG
ax = axes[0]
deriv_positions = {
"SourceFile": (2, 4),
"BasicPrep": (2, 3),
"Spectrum": (2, 2),
"BandPowerBoth": (2, 1),
}
deriv_colors = {
"SourceFile": "#bde0fe",
"BasicPrep": "#a8dadc",
"Spectrum": "#a8dadc",
"BandPowerBoth": "#ffb347",
}
for name, (x, y) in deriv_positions.items():
ax.scatter(x, y, s=3000, c=deriv_colors[name], zorder=3, edgecolors="black", linewidths=1.2)
ax.text(x, y, name, ha="center", va="center", fontsize=8, zorder=4)
for src, dst in [
("SourceFile", "BasicPrep"),
("BasicPrep", "Spectrum"),
("Spectrum", "BandPowerBoth"),
]:
x0, y0 = deriv_positions[src]
x1, y1 = deriv_positions[dst]
ax.annotate(
"",
xy=(x1, y1 + 0.18),
xytext=(x0, y0 - 0.18),
arrowprops={"arrowstyle": "->", "color": "gray", "lw": 1.5},
)
ax.set_xlim(0, 4)
ax.set_ylim(0, 5)
ax.axis("off")
ax.set_title("Pipeline-level DAG\n(derivative dependencies)", fontsize=10)
# Right: node-level DAG inside BandPowerBoth
ax = axes[1]
node_positions = {
0: (3, 4.5),
1: (3, 3.5),
2: (1.5, 2.3),
3: (4.5, 2.3),
4: (3, 1),
}
node_labels = {
0: "id.0\nSpectrum.nc",
1: "id.1\nextract_data_var",
2: "id.2\nbandpower\n(absolute)",
3: "id.3\nbandpower\n(relative)",
4: "id.4\nconcat_bandpower\n← fan-in",
}
node_colors = {0: "#bde0fe", 1: "#a8dadc", 2: "#cdb4db", 3: "#cdb4db", 4: "#ffb347"}
node_edges = [(0, 1), (1, 2), (1, 3), (2, 4), (3, 4)]
for nid, (x, y) in node_positions.items():
ax.scatter(x, y, s=2500, c=node_colors[nid], zorder=3, edgecolors="black", linewidths=1.2)
ax.text(x, y, node_labels[nid], ha="center", va="center", fontsize=7.5, zorder=4)
for src, dst in node_edges:
x0, y0 = node_positions[src]
x1, y1 = node_positions[dst]
ax.annotate(
"",
xy=(x1, y1 + 0.2),
xytext=(x0, y0 - 0.2),
arrowprops={"arrowstyle": "->", "color": "gray", "lw": 1.5},
)
# annotate the fan-out and fan-in
ax.text(0.5, 2.8, "fan-out\n(same id.1,\ntwo branches)", fontsize=7, color="purple", va="center")
ax.text(3.9, 1.55, "fan-in\n(id.2 + id.3\n→ id.4)", fontsize=7, color="darkorange", va="center")
ax.set_xlim(0, 6)
ax.set_ylim(0, 5.5)
ax.axis("off")
ax.set_title("Node-level DAG inside BandPowerBoth\n(fan-out → fan-in)", fontsize=10)
plt.tight_layout()
plt.savefig(WORKDIR / "dag_structure.png", dpi=100)
plt.show()
print(f"DAG diagram saved to {WORKDIR / 'dag_structure.png'}")
DAG diagram saved to /tmp/neurodags_dag_pdd4qt1z/dag_structure.png
Step 5 — Execute the Pipeline¶
run_pipeline runs all derivatives in DerivativeList, sorted by dependency order. Already-cached outputs are skipped.
run_pipeline(pipeline_config, raise_on_error=True)
produced = sorted(OUT_DIR.rglob("*@*.fif")) + sorted(OUT_DIR.rglob("*@*.nc"))
print(f"\nProduced {len(produced)} derivative file(s):")
for f in produced:
print(f" {f.relative_to(WORKDIR)}")
2026-05-15 18:28:12 [info ] Derivative execution order order=['BasicPrep', 'Spectrum', 'BandPowerBoth']
2026-05-15 18:28:12 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:12 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:12 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:12 [debug ] Registered derivative name=BandPowerBoth
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: processing dataset dataset=dag_demo
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: resolved pattern dataset=dag_demo pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:12 [debug ] get_files_from_pattern: found files count=2
2026-05-15 18:28:12 [debug ] find_unique_root: called mode=maximal n_paths=2 strict=True style=auto
2026-05-15 18:28:12 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:12 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']
2026-05-15 18:28:12 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_dag_pdd4qt1z/rawdata prefix_len=4
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_dag_pdd4qt1z/rawdata
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_dag_pdd4qt1z/rawdata dataset=dag_demo
2026-05-15 18:28:12 [info ] Found files in dataset dataset=dag_demo file_count=2
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: dataset summary dataset=dag_demo file_count=2
2026-05-15 18:28:12 [info ] File discovery complete total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] Processing file dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr index=0
2026-05-15 18:28:12 [debug ] Processing file dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr index=1
2026-05-15 18:28:12 [info ] Starting derivative processing total_files=2
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:12 [debug ] Execute node derivative=BasicPrep id=1 kwargs={'mne_object': '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}} node=basic_preprocessing
2026-05-15 18:28:12 [debug ] Attempting to load MEEG file file=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:12 [debug ] Loaded MEEG file as Raw file=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr
2026-05-15 18:28:12 [debug ] MNEReport: loaded MNE object from file input=<RawBrainVision | sub-S1_task-rest.eeg, 8 x 2000 (10.0 s), ~138 KiB, data loaded>
2026-05-15 18:28:12 [debug ] Filter Applied filter_args={'l_freq': 1.0, 'h_freq': 80.0}
Not setting metadata
5 matching events found
No baseline correction applied
0 projection items activated
Using data from preloaded Raw for 5 events and 400 original time points ...
0 bad epochs dropped
2026-05-15 18:28:12 [debug ] EPOCH SEGMENTATION with make_fixed_length_epochs epoch_config={'duration': 2.0, 'overlap': 0.0}
2026-05-15 18:28:12 [debug ] Processed artifact file=Artifact(item=<Epochs | 5 events (all good), 0 – 1.995 s (baseline off), ~138 KiB, data loaded,
'1': 5>, writer=<function basic_preprocessing.<locals>.<lambda> at 0x7f517a81e320>) name=.fif
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/preprocessing.py:143: RuntimeWarning: This filename (/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif) does not conform to MNE naming conventions. All epochs files should end with -epo.fif, -epo.fif.gz, _epo.fif or _epo.fif.gz
".fif": Artifact(item=mne_object, writer=lambda path: mne_object.save(path, overwrite=True))
2026-05-15 18:28:12 [debug ] Saved artifact file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:12 [debug ] Execute node derivative=BasicPrep id=1 kwargs={'mne_object': '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}} node=basic_preprocessing
2026-05-15 18:28:12 [debug ] Attempting to load MEEG file file=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:12 [debug ] Loaded MEEG file as Raw file=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr
2026-05-15 18:28:12 [debug ] MNEReport: loaded MNE object from file input=<RawBrainVision | sub-S0_task-rest.eeg, 8 x 2000 (10.0 s), ~138 KiB, data loaded>
2026-05-15 18:28:12 [debug ] Filter Applied filter_args={'l_freq': 1.0, 'h_freq': 80.0}
Not setting metadata
5 matching events found
No baseline correction applied
0 projection items activated
Using data from preloaded Raw for 5 events and 400 original time points ...
0 bad epochs dropped
2026-05-15 18:28:12 [debug ] EPOCH SEGMENTATION with make_fixed_length_epochs epoch_config={'duration': 2.0, 'overlap': 0.0}
2026-05-15 18:28:12 [debug ] Processed artifact file=Artifact(item=<Epochs | 5 events (all good), 0 – 1.995 s (baseline off), ~138 KiB, data loaded,
'1': 5>, writer=<function basic_preprocessing.<locals>.<lambda> at 0x7f51781b3520>) name=.fif
/home/runner/work/neurodags/neurodags/src/neurodags/nodes/preprocessing.py:143: RuntimeWarning: This filename (/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif) does not conform to MNE naming conventions. All epochs files should end with -epo.fif, -epo.fif.gz, _epo.fif or _epo.fif.gz
".fif": Artifact(item=mne_object, writer=lambda path: mne_object.save(path, overwrite=True))
2026-05-15 18:28:12 [debug ] Saved artifact file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:12 [info ] Processed file successfully dataset=dag_demo derivative=BasicPrep file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr index=0
2026-05-15 18:28:12 [info ] Processed file successfully dataset=dag_demo derivative=BasicPrep file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr index=1
2026-05-15 18:28:12 [info ] Completed derivative processing total_files=2
2026-05-15 18:28:12 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:12 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:12 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BandPowerBoth'
2026-05-15 18:28:12 [debug ] Registered derivative name=BandPowerBoth
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: processing dataset dataset=dag_demo
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: resolved pattern dataset=dag_demo pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:12 [debug ] get_files_from_pattern: found files count=2
2026-05-15 18:28:12 [debug ] find_unique_root: called mode=maximal n_paths=2 strict=True style=auto
2026-05-15 18:28:12 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:12 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']
2026-05-15 18:28:12 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_dag_pdd4qt1z/rawdata prefix_len=4
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_dag_pdd4qt1z/rawdata
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_dag_pdd4qt1z/rawdata dataset=dag_demo
2026-05-15 18:28:12 [info ] Found files in dataset dataset=dag_demo file_count=2
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: dataset summary dataset=dag_demo file_count=2
2026-05-15 18:28:12 [info ] File discovery complete total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] Processing file dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr index=0
2026-05-15 18:28:12 [debug ] Processing file dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr index=1
2026-05-15 18:28:12 [info ] Starting derivative processing total_files=2
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:12 [debug ] Using cached derivative child_derivative=BasicPrep.fif derivative=Spectrum id=0 paths=['/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif']
2026-05-15 18:28:12 [debug ] Execute node derivative=Spectrum id=1 kwargs={'meeg': '/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}} node=mne_spectrum_array
2026-05-15 18:28:12 [debug ] Attempting to load MEEG file file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:12 [debug ] Loaded MEEG file as Epochs file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:12 [debug ] MNEReport: loaded MNE object from file input=<EpochsFIF | 5 events (all good), 0 – 1.995 s (baseline off), ~139 KiB, data loaded,
'1': 5>
Effective window size : 1.280 (s)
2026-05-15 18:28:12 [debug ] Processed artifact file=Artifact(item=<xarray.Dataset> Size: 43kB
Dimensions: (epochs: 5, spaces: 8, frequencies: 129)
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Data variables:
spectrum (epochs, spaces, frequencies) float64 41kB 0.005941 ... 2.56...
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., writer=<function mne_spectrum_array.<locals>.<lambda> at 0x7f517a81e5f0>) name=.nc
2026-05-15 18:28:12 [debug ] Saved artifact file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@Spectrum.nc
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:12 [debug ] Using cached derivative child_derivative=BasicPrep.fif derivative=Spectrum id=0 paths=['/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif']
2026-05-15 18:28:12 [debug ] Execute node derivative=Spectrum id=1 kwargs={'meeg': '/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}} node=mne_spectrum_array
2026-05-15 18:28:12 [debug ] Attempting to load MEEG file file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif kwargs={'preload': True, 'verbose': 'error'}
2026-05-15 18:28:12 [debug ] Loaded MEEG file as Epochs file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif
2026-05-15 18:28:12 [debug ] MNEReport: loaded MNE object from file input=<EpochsFIF | 5 events (all good), 0 – 1.995 s (baseline off), ~139 KiB, data loaded,
'1': 5>
Effective window size : 1.280 (s)
2026-05-15 18:28:12 [debug ] Processed artifact file=Artifact(item=<xarray.Dataset> Size: 43kB
Dimensions: (epochs: 5, spaces: 8, frequencies: 129)
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Data variables:
spectrum (epochs, spaces, frequencies) float64 41kB 0.005941 ... 2.56...
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., writer=<function mne_spectrum_array.<locals>.<lambda> at 0x7f517a7780d0>) name=.nc
2026-05-15 18:28:12 [debug ] Saved artifact file=/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@Spectrum.nc
2026-05-15 18:28:12 [info ] Processed file successfully dataset=dag_demo derivative=Spectrum file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr index=0
2026-05-15 18:28:12 [info ] Processed file successfully dataset=dag_demo derivative=Spectrum file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr index=1
2026-05-15 18:28:12 [info ] Completed derivative processing total_files=2
2026-05-15 18:28:12 [debug ] iterate_call_pipeline: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:12 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:12 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BandPowerBoth'
2026-05-15 18:28:12 [debug ] Registered derivative name=BandPowerBoth
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: processing dataset dataset=dag_demo
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: resolved pattern dataset=dag_demo pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:12 [debug ] get_files_from_pattern: found files count=2
2026-05-15 18:28:12 [debug ] find_unique_root: called mode=maximal n_paths=2 strict=True style=auto
2026-05-15 18:28:12 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:12 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']
2026-05-15 18:28:12 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_dag_pdd4qt1z/rawdata prefix_len=4
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_dag_pdd4qt1z/rawdata
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_dag_pdd4qt1z/rawdata dataset=dag_demo
2026-05-15 18:28:12 [info ] Found files in dataset dataset=dag_demo file_count=2
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: dataset summary dataset=dag_demo file_count=2
2026-05-15 18:28:12 [info ] File discovery complete total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=2
2026-05-15 18:28:12 [info ] Derivative is marked with save=False; skipping execution. derivative=BandPowerBoth
Produced 4 derivative file(s):
derivatives/sub-S0/sub-S0_task-rest.vhdr@BasicPrep.fif
derivatives/sub-S1/sub-S1_task-rest.vhdr@BasicPrep.fif
derivatives/sub-S0/sub-S0_task-rest.vhdr@Spectrum.nc
derivatives/sub-S1/sub-S1_task-rest.vhdr@Spectrum.nc
Step 6 — Inspect the Fan-in Result¶
build_derivative_dataframe collects for_dataframe=True derivatives.
BandPowerBoth re-runs the full node chain (id.0–id.4) and flattens the
4-D result (epochs × channels × freqbands × normalization) into columns.
df = build_derivative_dataframe(pipeline_config, output_format="wide")
df["subject"] = df["file_path"].apply(
lambda p: next(
(part for part in Path(p).parts if part.startswith("sub-")),
Path(p).stem,
)
)
print(f"DataFrame shape: {df.shape}")
band_cols = [c for c in df.columns if "BandPower" in c]
print(f"Band-power columns ({len(band_cols)}):", band_cols[:6], "...")
2026-05-15 18:28:12 [debug ] build_derivative_dataframe: called datasets_configuration=None pipeline_configuration={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BasicPrep'
2026-05-15 18:28:12 [debug ] Registered derivative name=BasicPrep
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'Spectrum'
2026-05-15 18:28:12 [debug ] Registered derivative name=Spectrum
2026-05-15 18:28:12 [info ] Overriding existing derivative registration for 'BandPowerBoth'
2026-05-15 18:28:12 [debug ] Registered derivative name=BandPowerBoth
2026-05-15 18:28:12 [warning ] Some requested derivatives are either undefined or flagged out of dataframe collection. missing_derivatives=['BasicPrep', 'Spectrum']
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: processing dataset dataset=dag_demo
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: resolved pattern dataset=dag_demo pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:12 [debug ] get_files_from_pattern: found files count=2
2026-05-15 18:28:12 [debug ] find_unique_root: called mode=maximal n_paths=2 strict=True style=auto
2026-05-15 18:28:12 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:12 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']
2026-05-15 18:28:12 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_dag_pdd4qt1z/rawdata prefix_len=4
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_dag_pdd4qt1z/rawdata
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_dag_pdd4qt1z/rawdata dataset=dag_demo
2026-05-15 18:28:12 [info ] Found files in dataset dataset=dag_demo file_count=2
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: dataset summary dataset=dag_demo file_count=2
2026-05-15 18:28:12 [info ] File discovery complete total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: called pipeline_input={'mount_point': None, 'DerivativeDefinitions': {'BasicPrep': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'SourceFile'}, {'id': 1, 'node': 'basic_preprocessing', 'args': {'mne_object': 'id.0', 'filter_args': {'l_freq': 1.0, 'h_freq': 80.0}, 'epoch_config': {'duration': 2.0, 'overlap': 0.0}}}]}, 'Spectrum': {'overwrite': False, 'nodes': [{'id': 0, 'derivative': 'BasicPrep.fif'}, {'id': 1, 'node': 'mne_spectrum_array', 'args': {'meeg': 'id.0', 'method': 'welch', 'method_kwargs': {'n_per_seg': 200}}}]}, 'BandPowerBoth': {'save': False, 'for_dataframe': True, 'nodes': [{'id': 0, 'derivative': 'Spectrum.nc'}, {'id': 1, 'node': 'extract_data_var', 'args': {'dataset_like': 'id.0', 'data_var': 'spectrum'}}, {'id': 2, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 3, 'node': 'bandpower', 'args': {'psd_like': 'id.1', 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}}}, {'id': 4, 'node': 'concat_bandpower', 'args': {'absolute': 'id.2', 'relative': 'id.3'}}]}}, 'DerivativeList': ['BasicPrep', 'Spectrum', 'BandPowerBoth'], 'datasets': {'dag_demo': {'name': 'DAG Demo', 'file_pattern': '/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr', 'derivatives_path': '/tmp/neurodags_dag_pdd4qt1z/derivatives'}}}
2026-05-15 18:28:12 [debug ] Loading YAML rules arg_type=dict
2026-05-15 18:28:12 [debug ] Loading YAML from in-memory mapping action=deepcopy keys=4
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: processing dataset dataset=dag_demo
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: resolved pattern dataset=dag_demo pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr
2026-05-15 18:28:12 [debug ] get_files_from_pattern: called pattern=/tmp/neurodags_dag_pdd4qt1z/rawdata/**/*.vhdr recursive=True
2026-05-15 18:28:12 [debug ] get_files_from_pattern: found files count=2
2026-05-15 18:28:12 [debug ] find_unique_root: called mode=maximal n_paths=2 strict=True style=auto
2026-05-15 18:28:12 [debug ] find_unique_root: inferred style inferred=posix
2026-05-15 18:28:12 [debug ] find_unique_root: normalized paths sample sample=['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']
2026-05-15 18:28:12 [debug ] find_unique_root: prefix info common_prefix=/tmp/neurodags_dag_pdd4qt1z/rawdata prefix_len=4
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=1 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=2 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=3 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: candidate check depth=4 unique=True
2026-05-15 18:28:12 [debug ] find_unique_root: selected maximal root root=/tmp/neurodags_dag_pdd4qt1z/rawdata
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: common root for dataset common_root=/tmp/neurodags_dag_pdd4qt1z/rawdata dataset=dag_demo
2026-05-15 18:28:12 [info ] Found files in dataset dataset=dag_demo file_count=2
2026-05-15 18:28:12 [debug ] get_all_files_across_datasets: dataset summary dataset=dag_demo file_count=2
2026-05-15 18:28:12 [info ] File discovery complete total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] get_all_files_from_pipeline_configuration: completed total_datasets=1 total_files=2
2026-05-15 18:28:12 [debug ] build_derivative_dataframe: enumerated files per_dataset={'dag_demo': ['/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr', '/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr']} total_files=2
2026-05-15 18:28:12 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:12 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:12 [debug ] Using cached derivative child_derivative=Spectrum.nc derivative=BandPowerBoth id=0 paths=['/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@Spectrum.nc']
2026-05-15 18:28:12 [debug ] Execute node derivative=BandPowerBoth id=1 kwargs={'dataset_like': '/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S1/sub-S1_task-rest.vhdr@Spectrum.nc', 'data_var': 'spectrum'} node=extract_data_var
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=2 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, frequencies: 129)> Size: 41kB
array([[[5.94127897e-03, 5.17956298e-02, 5.31863419e-02, ...,
8.81075874e-07, 5.42786018e-07, 9.08968899e-08],
[2.95322987e-03, 3.15620894e-02, 3.87596966e-02, ...,
3.89249201e-07, 2.73782035e-07, 3.16368831e-07],
[2.39424013e-03, 2.25115671e-01, 1.64331624e-01, ...,
1.07484003e-06, 9.82229934e-08, 4.81665302e-09],
...,
[1.51071220e-02, 7.98487118e-02, 5.45707058e-02, ...,
2.94929106e-06, 3.09175884e-06, 5.20456826e-07],
[4.71703191e-03, 7.92136242e-03, 8.98544733e-03, ...,
3.57706512e-06, 8.40748598e-07, 4.14936098e-07],
[2.13308570e-04, 2.87910820e-02, 3.18051259e-02, ...,
1.97158147e-06, 1.07220718e-06, 2.44227382e-08]],
[[1.35229668e-03, 9.59713533e-03, 6.15881056e-02, ...,
7.92233826e-08, 1.20503206e-06, 1.66903249e-08],
[3.23719979e-02, 1.02560787e-01, 1.04193457e-01, ...,
2.80660950e-06, 2.04714367e-07, 5.27505226e-07],
[3.49345530e-02, 1.21793341e-01, 7.30010791e-02, ...,
5.06558060e-07, 2.88132118e-07, 1.32626060e-07],
...
7.97844801e-07, 1.33028035e-06, 1.47410596e-07],
[1.18133400e-02, 1.48826531e-01, 2.83600942e-01, ...,
7.52343747e-07, 7.74704865e-07, 3.70322677e-07],
[3.84787396e-02, 1.38396335e-01, 8.11427313e-02, ...,
1.96951092e-06, 3.47977203e-07, 2.04464257e-07]],
[[3.46169102e-02, 1.37140047e-01, 9.53475244e-02, ...,
1.60167922e-06, 3.86712023e-07, 1.00777924e-07],
[2.53822980e-02, 1.26110468e-01, 1.52970363e-01, ...,
1.42051196e-07, 1.81612764e-07, 1.80186322e-07],
[4.03601399e-02, 1.27266649e-01, 8.60509058e-02, ...,
1.68553829e-06, 1.91254965e-07, 1.09922210e-07],
...,
[8.27259389e-03, 4.45932702e-02, 9.37072888e-02, ...,
2.81008256e-07, 3.59394786e-07, 1.15024499e-07],
[5.05019450e-03, 1.69449084e-01, 2.68555660e-01, ...,
2.67783398e-07, 1.05084564e-06, 1.59865686e-07],
[2.06423763e-03, 2.75468731e-02, 3.63829309e-02, ...,
4.15108788e-07, 2.57588793e-07, 2.56000625e-07]]],
shape=(5, 8, 129))
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=3 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, frequencies: 129)> Size: 41kB
array([[[5.94127897e-03, 5.17956298e-02, 5.31863419e-02, ...,
8.81075874e-07, 5.42786018e-07, 9.08968899e-08],
[2.95322987e-03, 3.15620894e-02, 3.87596966e-02, ...,
3.89249201e-07, 2.73782035e-07, 3.16368831e-07],
[2.39424013e-03, 2.25115671e-01, 1.64331624e-01, ...,
1.07484003e-06, 9.82229934e-08, 4.81665302e-09],
...,
[1.51071220e-02, 7.98487118e-02, 5.45707058e-02, ...,
2.94929106e-06, 3.09175884e-06, 5.20456826e-07],
[4.71703191e-03, 7.92136242e-03, 8.98544733e-03, ...,
3.57706512e-06, 8.40748598e-07, 4.14936098e-07],
[2.13308570e-04, 2.87910820e-02, 3.18051259e-02, ...,
1.97158147e-06, 1.07220718e-06, 2.44227382e-08]],
[[1.35229668e-03, 9.59713533e-03, 6.15881056e-02, ...,
7.92233826e-08, 1.20503206e-06, 1.66903249e-08],
[3.23719979e-02, 1.02560787e-01, 1.04193457e-01, ...,
2.80660950e-06, 2.04714367e-07, 5.27505226e-07],
[3.49345530e-02, 1.21793341e-01, 7.30010791e-02, ...,
5.06558060e-07, 2.88132118e-07, 1.32626060e-07],
...
7.97844801e-07, 1.33028035e-06, 1.47410596e-07],
[1.18133400e-02, 1.48826531e-01, 2.83600942e-01, ...,
7.52343747e-07, 7.74704865e-07, 3.70322677e-07],
[3.84787396e-02, 1.38396335e-01, 8.11427313e-02, ...,
1.96951092e-06, 3.47977203e-07, 2.04464257e-07]],
[[3.46169102e-02, 1.37140047e-01, 9.53475244e-02, ...,
1.60167922e-06, 3.86712023e-07, 1.00777924e-07],
[2.53822980e-02, 1.26110468e-01, 1.52970363e-01, ...,
1.42051196e-07, 1.81612764e-07, 1.80186322e-07],
[4.03601399e-02, 1.27266649e-01, 8.60509058e-02, ...,
1.68553829e-06, 1.91254965e-07, 1.09922210e-07],
...,
[8.27259389e-03, 4.45932702e-02, 9.37072888e-02, ...,
2.81008256e-07, 3.59394786e-07, 1.15024499e-07],
[5.05019450e-03, 1.69449084e-01, 2.68555660e-01, ...,
2.67783398e-07, 1.05084564e-06, 1.59865686e-07],
[2.06423763e-03, 2.75468731e-02, 3.63829309e-02, ...,
4.15108788e-07, 2.57588793e-07, 2.56000625e-07]]],
shape=(5, 8, 129))
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=4 kwargs={'absolute': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, freqbands: 3)> Size: 960B
array([[[0.06348204, 0.05258287, 0.0858947 ],
[0.12533109, 0.03576644, 0.10333269],
[0.10289021, 0.03946224, 0.1269642 ],
[0.17847491, 0.01645002, 0.09542307],
[0.14010689, 0.05821092, 0.14061912],
[0.18352955, 0.07777756, 0.17453095],
[0.04625958, 0.04657737, 0.05523234],
[0.03485288, 0.05274275, 0.10013759]],
[[0.10048218, 0.0259934 , 0.10460606],
[0.07029853, 0.0676584 , 0.10622799],
[0.04403522, 0.0367996 , 0.13005126],
[0.19408211, 0.05275699, 0.10136721],
[0.25197519, 0.04183793, 0.08950499],
[0.11050146, 0.05628906, 0.17921248],
[0.19906601, 0.03373686, 0.1083582 ],
[0.07300881, 0.05094751, 0.13419569]],
[[0.346415 , 0.06694619, 0.08179736],
[0.15245372, 0.04650872, 0.10923927],
...
[0.09590624, 0.05013763, 0.09732567],
[0.04104436, 0.04136209, 0.13110766]],
[[0.08362224, 0.07170483, 0.11434832],
[0.07575369, 0.05553495, 0.11623923],
[0.1106851 , 0.04750144, 0.13700245],
[0.27763257, 0.02698599, 0.089056 ],
[0.06646501, 0.08942865, 0.13350428],
[0.11969672, 0.0871033 , 0.18064968],
[0.28235696, 0.0368758 , 0.12223599],
[0.05608705, 0.03230529, 0.1278332 ]],
[[0.07943184, 0.06282553, 0.0835689 ],
[0.15268156, 0.06142129, 0.06892359],
[0.18853604, 0.02763996, 0.08840897],
[0.14456723, 0.04708214, 0.06856778],
[0.04719227, 0.04783876, 0.09278719],
[0.13987419, 0.0520159 , 0.14588002],
[0.2219873 , 0.0334662 , 0.10242513],
[0.09232292, 0.04818469, 0.09215236]]])
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 60B 'delta' 'alpha' 'beta'
freqband_low (freqbands) float64 24B 1.0 8.0 13.0
freqband_high (freqbands) float64 24B 4.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig..., 'relative': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, freqbands: 3)> Size: 960B
array([[[0.12419 , 0.10286795, 0.16803592],
[0.222585 , 0.06352035, 0.18351638],
[0.14409513, 0.05526586, 0.17781014],
[0.29650329, 0.02732868, 0.15852792],
[0.20665961, 0.08586192, 0.20741515],
[0.21930695, 0.09293958, 0.20855416],
[0.13851511, 0.13946668, 0.16538226],
[0.07436653, 0.11253864, 0.21366631]],
[[0.19770864, 0.05114459, 0.20582279],
[0.12540791, 0.12069809, 0.18950368],
[0.06800801, 0.05683333, 0.20085123],
[0.26331451, 0.0715763 , 0.13752662],
[0.38698798, 0.06425544, 0.13746335],
[0.13120851, 0.06683716, 0.21279539],
[0.30139553, 0.05107923, 0.16405953],
[0.11822569, 0.08250107, 0.21730774]],
[[0.36804551, 0.07112638, 0.08690487],
[0.21063985, 0.06425944, 0.15093199],
...
[0.15926689, 0.08326115, 0.16162408],
[0.08910548, 0.08979527, 0.28462892]],
[[0.13371906, 0.11466211, 0.18285266],
[0.13531636, 0.09920028, 0.20763436],
[0.19909436, 0.08544301, 0.24643259],
[0.30767425, 0.02990605, 0.09869245],
[0.11301069, 0.15205584, 0.22699778],
[0.13955306, 0.10155276, 0.21061744],
[0.31805434, 0.04153788, 0.13768985],
[0.09724992, 0.05601447, 0.22165132]],
[[0.12772995, 0.10102626, 0.13438253],
[0.22091262, 0.08886953, 0.09972449],
[0.24710806, 0.0362268 , 0.11587476],
[0.21665661, 0.07055995, 0.10275955],
[0.09853972, 0.09988962, 0.19374411],
[0.16970925, 0.06311085, 0.17699612],
[0.29257431, 0.04410771, 0.13499403],
[0.19696193, 0.10279734, 0.19659805]]])
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 60B 'delta' 'alpha' 'beta'
freqband_low (freqbands) float64 24B 1.0 8.0 13.0
freqband_high (freqbands) float64 24B 4.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig...} node=concat_bandpower
2026-05-15 18:28:13 [debug ] Collected dataframe row collected_derivatives=1 dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S1/sub-S1_task-rest.vhdr index=0
2026-05-15 18:28:13 [debug ] get_path: called mount_point=None path_type=str
2026-05-15 18:28:13 [debug ] get_path: returning direct path path=/tmp/neurodags_dag_pdd4qt1z/derivatives
2026-05-15 18:28:13 [debug ] Using cached derivative child_derivative=Spectrum.nc derivative=BandPowerBoth id=0 paths=['/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@Spectrum.nc']
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=1 kwargs={'dataset_like': '/tmp/neurodags_dag_pdd4qt1z/derivatives/sub-S0/sub-S0_task-rest.vhdr@Spectrum.nc', 'data_var': 'spectrum'} node=extract_data_var
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=2 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, frequencies: 129)> Size: 41kB
array([[[5.94127897e-03, 5.17956298e-02, 5.31863419e-02, ...,
8.81075874e-07, 5.42786018e-07, 9.08968899e-08],
[2.95322987e-03, 3.15620894e-02, 3.87596966e-02, ...,
3.89249201e-07, 2.73782035e-07, 3.16368831e-07],
[2.39424013e-03, 2.25115671e-01, 1.64331624e-01, ...,
1.07484003e-06, 9.82229934e-08, 4.81665302e-09],
...,
[1.51071220e-02, 7.98487118e-02, 5.45707058e-02, ...,
2.94929106e-06, 3.09175884e-06, 5.20456826e-07],
[4.71703191e-03, 7.92136242e-03, 8.98544733e-03, ...,
3.57706512e-06, 8.40748598e-07, 4.14936098e-07],
[2.13308570e-04, 2.87910820e-02, 3.18051259e-02, ...,
1.97158147e-06, 1.07220718e-06, 2.44227382e-08]],
[[1.35229668e-03, 9.59713533e-03, 6.15881056e-02, ...,
7.92233826e-08, 1.20503206e-06, 1.66903249e-08],
[3.23719979e-02, 1.02560787e-01, 1.04193457e-01, ...,
2.80660950e-06, 2.04714367e-07, 5.27505226e-07],
[3.49345530e-02, 1.21793341e-01, 7.30010791e-02, ...,
5.06558060e-07, 2.88132118e-07, 1.32626060e-07],
...
7.97844801e-07, 1.33028035e-06, 1.47410596e-07],
[1.18133400e-02, 1.48826531e-01, 2.83600942e-01, ...,
7.52343747e-07, 7.74704865e-07, 3.70322677e-07],
[3.84787396e-02, 1.38396335e-01, 8.11427313e-02, ...,
1.96951092e-06, 3.47977203e-07, 2.04464257e-07]],
[[3.46169102e-02, 1.37140047e-01, 9.53475244e-02, ...,
1.60167922e-06, 3.86712023e-07, 1.00777924e-07],
[2.53822980e-02, 1.26110468e-01, 1.52970363e-01, ...,
1.42051196e-07, 1.81612764e-07, 1.80186322e-07],
[4.03601399e-02, 1.27266649e-01, 8.60509058e-02, ...,
1.68553829e-06, 1.91254965e-07, 1.09922210e-07],
...,
[8.27259389e-03, 4.45932702e-02, 9.37072888e-02, ...,
2.81008256e-07, 3.59394786e-07, 1.15024499e-07],
[5.05019450e-03, 1.69449084e-01, 2.68555660e-01, ...,
2.67783398e-07, 1.05084564e-06, 1.59865686e-07],
[2.06423763e-03, 2.75468731e-02, 3.63829309e-02, ...,
4.15108788e-07, 2.57588793e-07, 2.56000625e-07]]],
shape=(5, 8, 129))
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': False, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=3 kwargs={'psd_like': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, frequencies: 129)> Size: 41kB
array([[[5.94127897e-03, 5.17956298e-02, 5.31863419e-02, ...,
8.81075874e-07, 5.42786018e-07, 9.08968899e-08],
[2.95322987e-03, 3.15620894e-02, 3.87596966e-02, ...,
3.89249201e-07, 2.73782035e-07, 3.16368831e-07],
[2.39424013e-03, 2.25115671e-01, 1.64331624e-01, ...,
1.07484003e-06, 9.82229934e-08, 4.81665302e-09],
...,
[1.51071220e-02, 7.98487118e-02, 5.45707058e-02, ...,
2.94929106e-06, 3.09175884e-06, 5.20456826e-07],
[4.71703191e-03, 7.92136242e-03, 8.98544733e-03, ...,
3.57706512e-06, 8.40748598e-07, 4.14936098e-07],
[2.13308570e-04, 2.87910820e-02, 3.18051259e-02, ...,
1.97158147e-06, 1.07220718e-06, 2.44227382e-08]],
[[1.35229668e-03, 9.59713533e-03, 6.15881056e-02, ...,
7.92233826e-08, 1.20503206e-06, 1.66903249e-08],
[3.23719979e-02, 1.02560787e-01, 1.04193457e-01, ...,
2.80660950e-06, 2.04714367e-07, 5.27505226e-07],
[3.49345530e-02, 1.21793341e-01, 7.30010791e-02, ...,
5.06558060e-07, 2.88132118e-07, 1.32626060e-07],
...
7.97844801e-07, 1.33028035e-06, 1.47410596e-07],
[1.18133400e-02, 1.48826531e-01, 2.83600942e-01, ...,
7.52343747e-07, 7.74704865e-07, 3.70322677e-07],
[3.84787396e-02, 1.38396335e-01, 8.11427313e-02, ...,
1.96951092e-06, 3.47977203e-07, 2.04464257e-07]],
[[3.46169102e-02, 1.37140047e-01, 9.53475244e-02, ...,
1.60167922e-06, 3.86712023e-07, 1.00777924e-07],
[2.53822980e-02, 1.26110468e-01, 1.52970363e-01, ...,
1.42051196e-07, 1.81612764e-07, 1.80186322e-07],
[4.03601399e-02, 1.27266649e-01, 8.60509058e-02, ...,
1.68553829e-06, 1.91254965e-07, 1.09922210e-07],
...,
[8.27259389e-03, 4.45932702e-02, 9.37072888e-02, ...,
2.81008256e-07, 3.59394786e-07, 1.15024499e-07],
[5.05019450e-03, 1.69449084e-01, 2.68555660e-01, ...,
2.67783398e-07, 1.05084564e-06, 1.59865686e-07],
[2.06423763e-03, 2.75468731e-02, 3.63829309e-02, ...,
4.15108788e-07, 2.57588793e-07, 2.56000625e-07]]],
shape=(5, 8, 129))
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* frequencies (frequencies) float64 1kB 0.0 0.7812 1.562 ... 99.22 100.0
Attributes:
metadata: {\n "method": "welch",\n "method_kwargs": {\n "n_per_seg"..., 'relative': True, 'bands': {'delta': [1.0, 4.0], 'alpha': [8.0, 13.0], 'beta': [13.0, 30.0]}} node=bandpower
2026-05-15 18:28:13 [debug ] Execute node derivative=BandPowerBoth id=4 kwargs={'absolute': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, freqbands: 3)> Size: 960B
array([[[0.06348204, 0.05258287, 0.0858947 ],
[0.12533109, 0.03576644, 0.10333269],
[0.10289021, 0.03946224, 0.1269642 ],
[0.17847491, 0.01645002, 0.09542307],
[0.14010689, 0.05821092, 0.14061912],
[0.18352955, 0.07777756, 0.17453095],
[0.04625958, 0.04657737, 0.05523234],
[0.03485288, 0.05274275, 0.10013759]],
[[0.10048218, 0.0259934 , 0.10460606],
[0.07029853, 0.0676584 , 0.10622799],
[0.04403522, 0.0367996 , 0.13005126],
[0.19408211, 0.05275699, 0.10136721],
[0.25197519, 0.04183793, 0.08950499],
[0.11050146, 0.05628906, 0.17921248],
[0.19906601, 0.03373686, 0.1083582 ],
[0.07300881, 0.05094751, 0.13419569]],
[[0.346415 , 0.06694619, 0.08179736],
[0.15245372, 0.04650872, 0.10923927],
...
[0.09590624, 0.05013763, 0.09732567],
[0.04104436, 0.04136209, 0.13110766]],
[[0.08362224, 0.07170483, 0.11434832],
[0.07575369, 0.05553495, 0.11623923],
[0.1106851 , 0.04750144, 0.13700245],
[0.27763257, 0.02698599, 0.089056 ],
[0.06646501, 0.08942865, 0.13350428],
[0.11969672, 0.0871033 , 0.18064968],
[0.28235696, 0.0368758 , 0.12223599],
[0.05608705, 0.03230529, 0.1278332 ]],
[[0.07943184, 0.06282553, 0.0835689 ],
[0.15268156, 0.06142129, 0.06892359],
[0.18853604, 0.02763996, 0.08840897],
[0.14456723, 0.04708214, 0.06856778],
[0.04719227, 0.04783876, 0.09278719],
[0.13987419, 0.0520159 , 0.14588002],
[0.2219873 , 0.0334662 , 0.10242513],
[0.09232292, 0.04818469, 0.09215236]]])
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 60B 'delta' 'alpha' 'beta'
freqband_low (freqbands) float64 24B 1.0 8.0 13.0
freqband_high (freqbands) float64 24B 4.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig..., 'relative': <xarray.DataArray 'spectrum' (epochs: 5, spaces: 8, freqbands: 3)> Size: 960B
array([[[0.12419 , 0.10286795, 0.16803592],
[0.222585 , 0.06352035, 0.18351638],
[0.14409513, 0.05526586, 0.17781014],
[0.29650329, 0.02732868, 0.15852792],
[0.20665961, 0.08586192, 0.20741515],
[0.21930695, 0.09293958, 0.20855416],
[0.13851511, 0.13946668, 0.16538226],
[0.07436653, 0.11253864, 0.21366631]],
[[0.19770864, 0.05114459, 0.20582279],
[0.12540791, 0.12069809, 0.18950368],
[0.06800801, 0.05683333, 0.20085123],
[0.26331451, 0.0715763 , 0.13752662],
[0.38698798, 0.06425544, 0.13746335],
[0.13120851, 0.06683716, 0.21279539],
[0.30139553, 0.05107923, 0.16405953],
[0.11822569, 0.08250107, 0.21730774]],
[[0.36804551, 0.07112638, 0.08690487],
[0.21063985, 0.06425944, 0.15093199],
...
[0.15926689, 0.08326115, 0.16162408],
[0.08910548, 0.08979527, 0.28462892]],
[[0.13371906, 0.11466211, 0.18285266],
[0.13531636, 0.09920028, 0.20763436],
[0.19909436, 0.08544301, 0.24643259],
[0.30767425, 0.02990605, 0.09869245],
[0.11301069, 0.15205584, 0.22699778],
[0.13955306, 0.10155276, 0.21061744],
[0.31805434, 0.04153788, 0.13768985],
[0.09724992, 0.05601447, 0.22165132]],
[[0.12772995, 0.10102626, 0.13438253],
[0.22091262, 0.08886953, 0.09972449],
[0.24710806, 0.0362268 , 0.11587476],
[0.21665661, 0.07055995, 0.10275955],
[0.09853972, 0.09988962, 0.19374411],
[0.16970925, 0.06311085, 0.17699612],
[0.29257431, 0.04410771, 0.13499403],
[0.19696193, 0.10279734, 0.19659805]]])
Coordinates:
* epochs (epochs) int64 40B 0 1 2 3 4
* spaces (spaces) <U6 192B 'EEG000' 'EEG001' ... 'EEG006' 'EEG007'
* freqbands (freqbands) <U5 60B 'delta' 'alpha' 'beta'
freqband_low (freqbands) float64 24B 1.0 8.0 13.0
freqband_high (freqbands) float64 24B 4.0 13.0 30.0
Attributes:
metadata: {\n "bands": {\n "delta": {\n "low": 1.0,\n "hig...} node=concat_bandpower
2026-05-15 18:28:13 [debug ] Collected dataframe row collected_derivatives=1 dataset=dag_demo file_path=/tmp/neurodags_dag_pdd4qt1z/rawdata/sub-S0/sub-S0_task-rest.vhdr index=1
DataFrame shape: (2, 244)
Band-power columns (240): ['BandPowerBoth.nc@epochs-0_freqbands-delta_normalization-absolute_spaces-EEG000', 'BandPowerBoth.nc@epochs-0_freqbands-alpha_normalization-absolute_spaces-EEG000', 'BandPowerBoth.nc@epochs-0_freqbands-beta_normalization-absolute_spaces-EEG000', 'BandPowerBoth.nc@epochs-0_freqbands-delta_normalization-absolute_spaces-EEG001', 'BandPowerBoth.nc@epochs-0_freqbands-alpha_normalization-absolute_spaces-EEG001', 'BandPowerBoth.nc@epochs-0_freqbands-beta_normalization-absolute_spaces-EEG001'] ...
Step 7 — Plot Band Power by Normalization Type¶
Separate the absolute and relative columns and compare them side-by-side.
abs_cols = [c for c in band_cols if "absolute" in c]
rel_cols = [c for c in band_cols if "relative" in c]
if abs_cols and rel_cols:
subjects = sorted(df["subject"].unique())
n_subs = len(subjects)
x = np.arange(len(abs_cols))
width = 0.8 / n_subs
fig, axes = plt.subplots(1, 2, figsize=(12, 4), sharey=False)
for ax, cols, title in zip(
axes,
[abs_cols, rel_cols],
["Absolute Band Power", "Relative Band Power"],
strict=False,
):
for i, sub in enumerate(subjects):
row = df[df["subject"] == sub]
vals = row[cols].mean(axis=0).values if not row.empty else np.zeros(len(cols))
ax.bar(x + i * width, vals, width=width, label=sub)
ax.set_xticks(x + width * (n_subs - 1) / 2)
short_labels = [c.split("@")[-1].split("_")[0] for c in cols]
ax.set_xticklabels(short_labels, rotation=30, ha="right", fontsize=8)
ax.set_title(title)
ax.legend(title="Subject")
plt.tight_layout()
plt.savefig(WORKDIR / "band_power_both.png", dpi=100)
plt.show()
print(f"Plot saved to {WORKDIR / 'band_power_both.png'}")
else:
print("No band-power columns found — check pipeline execution.")
Plot saved to /tmp/neurodags_dag_pdd4qt1z/band_power_both.png
Summary¶
Key takeaways:
YAML config makes pipelines readable and version-controllable.
The same pipeline can be driven from the CLI with
neurodags validate,neurodags run,neurodags dry-run,neurodags dataframe, andneurodags dag.Custom nodes (
@register_node) integrate seamlessly — no plugins needed.Fan-out: point multiple node
argsat the sameid.N.Fan-in: list multiple
id.Nreferences in one node’sargs; the topological sorter resolves execution order automatically.The same patterns compose: chains, branches, and merges can be nested arbitrarily deep within a single derivative or across derivatives.
Total running time of the script: (0 minutes 2.753 seconds)