Windowed Analysis Tutorial#

This tutorial provides a deep dive into Neurodent’s windowed analysis capabilities for extracting features from continuous EEG data.

Overview#

Windowed Analysis Results (WAR) is the core feature extraction system in Neurodent. It:

  1. Divides continuous EEG data into time windows

  2. Computes features for each window

  3. Aggregates results across time and channels

  4. Provides filtering and quality control methods

This approach is efficient for long recordings and enables parallel processing.

import sys
from pathlib import Path
import logging
from datetime import datetime

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

from neurodent import core, visualization, constants

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 10
      7 import matplotlib.pyplot as plt
      8 import pandas as pd
---> 10 from neurodent import core, visualization, constants
     12 logging.basicConfig(level=logging.INFO)
     13 logger = logging.getLogger()

ImportError: cannot import name 'visualization' from 'neurodent' (/home/runner/work/neurodent/neurodent/src/neurodent/__init__.py)

1. Feature Categories#

Neurodent extracts four main categories of features:

Linear Features (per channel)#

Single-value metrics for each channel in each time window:

# Available linear features
print("Linear features:")
for feature in constants.LINEAR_FEATURES:
    print(f"  - {feature}")

# Examples:
# - rms: Root mean square amplitude
# - logrms: Log of RMS amplitude
# - ampvar: Amplitude variance
# - psdtot: Total power spectral density
# - psdslope: Slope of PSD on log-log scale

Band Features (per frequency band)#

Features computed for each frequency band (delta, theta, alpha, beta, gamma):

# Available band features
print("\nBand features:")
for feature in constants.BAND_FEATURES:
    print(f"  - {feature}")

# Frequency bands
print("\nFrequency bands:")
for band, (lo, hi) in constants.FREQ_BANDS.items():
    print(f"  {band.capitalize()}: {lo}-{hi} Hz")

Matrix Features (connectivity)#

Features measuring relationships between channels:

# Available matrix features
print("\nMatrix features:")
for feature in constants.MATRIX_FEATURES:
    print(f"  - {feature}")

# Examples:
# - cohere: Spectral coherence between channel pairs
# - pcorr: Pearson correlation between channels

2. Computing Windowed Analysis#

Basic Usage#

# Using included test data
data_path = Path("../../.tests/integration/data/A10/A10_recording.edf")
animal_id = "A10"

# Create LongRecordingOrganizer with SpikeInterface mode
# mode options: 'si' (SpikeInterface), 'mne', or None
lro = core.LongRecordingOrganizer(
    item=data_path,
    mode="si",
    extract_func="read_edf",
    manual_datetimes=datetime(2023, 12, 13),
)

# Create AnimalOrganizer using pattern-based discovery
ao = visualization.AnimalOrganizer(
    pattern="../../.tests/integration/data/{animal}/*.edf",
    animal_id=animal_id,
    lro_kwargs={
        "mode": "si",
        "extract_func": "read_edf",
        "manual_datetimes": datetime(2023, 12, 13),
    },
)

# Compute all features
war_all = ao.compute_windowed_analysis(
    features=['all'],
    exclude=['nspike', 'lognspike'],  # Exclude spike features if no spikes
    multiprocess_mode='serial'
)

Access information in WindowedAnalysis object#

You can access the summary of WindowedAnalysis object using the get_info method, and access the computed features in the WindowedAnalysis object using the get_result method.

war_info = war_all.get_info()
print(war_info)

result = war_all.get_result(
    features=["all"],
    exclude=['nspike', 'lognspike']
)
display(result)

Selective Feature Computation#

For faster processing, compute only needed features:

# Compute specific features
war_selective = ao.compute_windowed_analysis(
    features=['rms', 'logrms', 'psdband', 'cohere'],
    multiprocess_mode='serial'
)

result = war_selective.get_result(
    features=['rms', 'logrms', 'psdband', 'cohere']
)

print(f"Computed features: {list(result.keys())}")

Parallel Processing#

For large datasets, use parallel processing:

# Option 1: Multiprocessing (uses all CPU cores)
war_mp = ao.compute_windowed_analysis(
    features=['rms', 'psdband'],
    multiprocess_mode='multiprocess'
)

# Option 2: Dask (for distributed computing)
# Requires Dask cluster setup
war_dask = ao.compute_windowed_analysis(
    features=['rms', 'psdband'],
    multiprocess_mode='dask'
)

3. Data Quality and Filtering#

Configuration-Driven Filtering#

Alternative approach using configuration dictionary:

filter_config = {
    'logrms_range': {'z_range': 3},
    'high_rms': {'max_rms': 500},
    'low_rms': {'min_rms': 10},
    # 'high_beta': {'max_beta_prop': 0.4},
    'reject_channels_by_session': {},
    'morphological_smoothing': {'smoothing_seconds': 4.0}
}

war_filtered_config = war_all.apply_filters(
    filter_config,
    min_valid_channels=3
)

Available Filters#

  • filter_logrms_range(z_range): Remove outliers based on log RMS

  • filter_high_rms(max_rms): Remove high amplitude artifacts

  • filter_low_rms(min_rms): Remove low amplitude periods

  • filter_high_beta(max_beta_prop): Remove high beta activity (muscle artifacts)

  • filter_reject_channels_by_session(): Identify and reject bad channels

  • morphological_smoothing(smoothing_seconds): Smooth data morphologically

4. Data Aggregation#

Average across time windows, producing a single row per group (e.g., per recording session and light/dark phase). Channel information is preserved.

# Aggregate time windows
war_filtered.aggregate_time_windows()

5. Channel Management#

Reorder and Pad Channels#

Ensure consistent channel ordering across animals:

# Define standard channel order
standard_channels = [
    "LMot", "RMot",  # Motor cortex
    "LBar", "RBar",  # Barrel cortex
    "LAud", "RAud",  # Auditory cortex
    "LVis", "RVis",  # Visual cortex
    "LHip", "RHip"   # Hippocampus
]

war_filtered.reorder_and_pad_channels(
    standard_channels,
    use_abbrevs=True  # Use abbreviated channel names
)

print(f"Channels: {war_filtered.channel_names}")

6. Accessing Computed Features#

WAR objects store features in a pandas DataFrame. Use get_result() to retrieve features with full channel information, or get_channel_averaged_result() to average across channels (or channel pairs for connectivity features), producing scalar values per time window.

# Get the full result DataFrame
result_df = war_filtered.get_result(
    features=['rms', 'psdband', 'cohere']
)
print(f"Result columns: {list(result_df.columns)}")
print(f"Result shape: {result_df.shape}")
print(f"\nRMS values (per-channel arrays):\n{result_df['rms'].iloc[0]}")

# Get channel-averaged result (scalars per time window)
df_avg = war_filtered.get_channel_averaged_result(
    features=['logrms', 'logpsdband', 'zcohere']
)
print(f"\nChannel-averaged columns: {list(df_avg.columns)}")
print(f"\nSample logrms value: {df_avg['logrms'].iloc[0]}")  # single float

7. Metadata and Grouping Variables#

WAR objects contain metadata for grouping and analysis:

# Access metadata
print(f"Animal ID: {war_filtered.animal_id}")
print(f"Genotype: {war_filtered.genotype}")
print(f"Animal days: {war_filtered.animaldays}")
print(f"Channel names: {war_filtered.channel_names}")

8. Circadian Analysis (ZeitgeberAnalysisResult)#

Once your data is filtered and metadata (like genotype and timestamps) is verified, you can analyze circadian rhythms.

The ZeitgeberAnalysisResult wrapper uses this metadata to:

  1. Shift Timestamps: Converts absolute timestamps to Zeitgeber Time (ZT), where ZT0 is “Lights On”.

  2. Define Baseline: Subtracts a baseline period (e.g., the first 12 hours of the light phase) to normalization data.

  3. Prepare for Visualization: Duplicates data for 48-hour “double-plotted” actograms.

For plotting these results, see the Visualization Tutorial.

from neurodent import core

# Wrap the result for circadian analysis
zar = core.ZeitgeberAnalysisResult(
    war_filtered,
    baseline_hours=12,
    zeitgeber_shift_hours=6,
    shift_for_48h=True
)

# Access the processed data
df_zar = zar.get_result(features=['rms'])

timestamps = war_filtered.result['timestamp']
print(f"Original Time Range: {timestamps.min()} to {timestamps.max()}")
print(f"ZT Coordinate Range: {df_zar.total_minutes.min()} to {df_zar.total_minutes.max()} min")
print(f"Baseline-Corrected Columns: {[c for c in df_zar.columns if '_nobase' in c][:3]}")

9. Saving and Loading#

Save WAR objects for later analysis:

import tempfile

# Use a temporary directory for demonstration purposes
with tempfile.TemporaryDirectory() as tmpdir:
    output_path = Path(tmpdir) / animal_id
    output_path.mkdir(parents=True, exist_ok=True)

    # Save WAR
    war_filtered.save_pickle_and_json(output_path)
    print(f"Saved to {output_path}")

    # Load WAR
    war_loaded = visualization.WindowAnalysisResult.load_pickle_and_json(output_path)
    print(f"Loaded from {output_path}")

10. Best Practices#

Feature Selection#

  • Start with basic features (rms, psdband) before computing expensive ones (cohere, psd)

  • Exclude spike features if you don’t have spike data

  • Use selective feature computation for faster iteration

Filtering#

  • Always inspect data before and after filtering

  • Use conservative thresholds initially, then adjust

  • Consider biological significance (e.g., high beta may indicate muscle artifacts)

Processing#

  • Use serial mode for debugging

  • Use multiprocess for local analysis of large datasets

  • Use Dask for cluster computing

Quality Control#

  • Check channel consistency across animals

  • Verify metadata (genotype, day, etc.)

  • Save intermediate results frequently

Summary#

This tutorial covered:

  1. Feature categories and types

  2. Computing windowed analysis with different options

  3. Data quality control and filtering

  4. Channel management and standardization

  5. Accessing computed features

  6. Metadata and grouping variables

  7. Saving and loading results

  8. Best practices

Next Steps#