Skip to content

Conversation

@colehanan1
Copy link
Owner

@colehanan1 colehanan1 commented Jan 8, 2026

  • Create dedicated ablations/ subfolder within condition output directory
  • Move ablation_summary.csv to ablations/ablation_summary.csv
  • Move ablation_comparison.png to ablations/ablation_comparison.png
  • Individual ablation results remain in separate ablate_* folders

Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com

Summary by CodeRabbit

Release Notes

  • New Features

    • Added ablation analysis tool for testing circuit robustness by zeroing receptors and measuring performance impact.
    • Added focus mode analysis tool to evaluate receptor subsets and generate performance curves (MSE vs. number of receptors).
  • Documentation

    • Added Robustness Analysis section to README with CLI usage examples and commands.
    • Added comprehensive documentation on stability assessment for LASSO-identified receptor circuits.

✏️ Tip: You can customize this high-level summary in your review settings.

colehanan1 and others added 4 commits January 8, 2026 12:14
Implements ablation analysis to measure the robustness of LASSO-identified
receptor circuits by zeroing out selected receptor channels and refitting.

New features:
- scripts/lasso_with_ablations.py: CLI script for ablation analysis
  - Supports --ablate (comma-separated) and --ablate_file (one per line)
  - ablation_set_mode: single (each receptor individually) or all_in_one
  - missing_receptor_policy: error or skip
  - Outputs: baseline_model.json, ablation_summary.csv, per-ablation artifacts

- Helper functions in behavioral_prediction.py:
  - resolve_receptor_names(): Case-insensitive receptor name matching
  - apply_receptor_ablation(): Zero out specified receptor columns
  - fit_lasso_with_fixed_scaler(): Refit LASSO with preserved baseline scaler

Key design decisions:
- Fit scaler on baseline X, reuse for ablation variants (apples-to-apples)
- Case-insensitive exact matching only (no fuzzy matching)
- Deterministic ordering and filenames

Tests: 25 new tests in test_lasso_ablation.py covering:
- Receptor name resolution
- Ablation matrix operations
- LASSO fitting with fixed scaler
- Output file format validation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements focus mode analysis to measure how many receptors are needed
for accurate behavioral prediction by restricting to top-N receptors
from baseline and refitting LASSO.

New features:
- scripts/lasso_with_focus_mode.py: CLI script for focus mode analysis
  - --topn_list: comma-separated N values (e.g., "1,2,3,5,10,15,20")
  - --focus_receptors: explicit receptor list (overrides topn_list)
  - --baseline_select_by: ranking method (abs_weight default)
  - Outputs: baseline_model.json, focus_curve.csv, focus_curve.png

- Helper functions in behavioral_prediction.py:
  - restrict_to_receptors(): Subset feature matrix to specified receptors
  - get_top_receptors_by_weight(): Rank receptors by |LASSO weight|

Key design decisions:
- For each N, fit scaler on restricted X (fair comparison within N)
- Deterministic receptor ranking by absolute weight
- Case-insensitive receptor name matching
- Generates MSE vs N and R² vs N curves

Tests: 24 new tests in test_lasso_focus_mode.py covering:
- Feature matrix restriction
- Top-N receptor selection (determinism verified)
- Focus mode integration workflow
- Output file format validation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…folder

- Create dedicated ablations/ subfolder within condition output directory
- Move ablation_summary.csv to ablations/ablation_summary.csv
- Move ablation_comparison.png to ablations/ablation_comparison.png
- Individual ablation results remain in separate ablate_* folders

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@coderabbitai
Copy link

coderabbitai bot commented Jan 8, 2026

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Introduces two new CLI scripts for robustness analysis of LASSO-identified receptor circuits: lasso_with_ablations.py (ablation-based testing via receptor zeroing) and lasso_with_focus_mode.py (focus-mode testing on receptor subsets). Includes supporting utilities in the behavioral prediction module, documentation, and comprehensive test coverage.

Changes

Cohort / File(s) Summary
Documentation
README.md, docs/BEHAVIORAL_PREDICTION_ANALYSIS.md
Added "Robustness Analysis" sections describing new CLI scripts, example invocations, and key arguments for ablation and focus mode workflows.
Robustness Analysis Scripts
scripts/lasso_with_ablations.py, scripts/lasso_with_focus_mode.py
New CLI entry points implementing end-to-end ablation and focus mode pipelines. Define AblationResult and FocusResult dataclasses, argument parsing, baseline/ablation/focus fitting workflows, CSV/JSON output, and comparison visualizations.
Core Utilities
src/door_toolkit/pathways/behavioral_prediction.py
Added five helper functions: resolve_receptor_names (strict name matching), apply_receptor_ablation (zero specified columns), fit_lasso_with_fixed_scaler (LASSO with pre-fitted scaler), restrict_to_receptors (subset feature matrix), and get_top_receptors_by_weight (rank receptors).
Module Exports
src/door_toolkit/pathways/__init__.py
Re-exported five new functions from behavioral_prediction module to expand public API surface.
Test Suite
tests/test_lasso_ablation.py, tests/test_lasso_focus_mode.py
Comprehensive unit and integration tests covering helper functions, script workflows, file I/O, and edge cases for both ablation and focus mode pipelines.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant CLI as lasso_with_ablations.py
    participant Predictor as LassoBehavioralPredictor
    participant FileIO as File I/O
    participant Viz as Visualization

    User->>CLI: Run with config + receptors
    CLI->>CLI: Parse args & resolve receptor names
    CLI->>Predictor: Build predictor
    CLI->>Predictor: run_baseline (fit LASSO on full set)
    Predictor-->>CLI: baseline_result (metrics, weights, scaler)
    CLI->>CLI: Ablate receptors (zero columns)
    CLI->>Predictor: fit_lasso_with_fixed_scaler (refit on ablated)
    Predictor-->>CLI: ablation_result (delta metrics)
    CLI->>FileIO: Save per-ablation artifacts
    FileIO-->>CLI: Saved
    CLI->>Viz: plot_ablation_comparison (baseline vs. ablations)
    Viz-->>CLI: PNG generated
    CLI->>FileIO: Save summary CSV
    CLI->>User: Print summary + exit
Loading
sequenceDiagram
    actor User
    participant CLI as lasso_with_focus_mode.py
    participant Predictor as LassoBehavioralPredictor
    participant Ranker as Receptor Ranking
    participant FileIO as File I/O
    participant Viz as Visualization

    User->>CLI: Run with config + focus targets
    CLI->>CLI: Parse args
    CLI->>Predictor: Build predictor
    CLI->>Predictor: run_baseline (fit LASSO on full set)
    Predictor-->>CLI: baseline (weights, R², MSE, scaler)
    CLI->>Ranker: get_top_receptors_by_weight (rank by |weight|)
    Ranker-->>CLI: Ranked receptor list
    CLI->>CLI: Generate focus sets (explicit or top-N)
    loop For each focus set
        CLI->>CLI: restrict_to_receptors (subset features)
        CLI->>Predictor: fit_lasso_with_fixed_scaler (refit on subset)
        Predictor-->>CLI: focus_result (subset metrics)
    end
    CLI->>FileIO: Save baseline.json, focus JSONs, focus_curve.csv
    FileIO-->>CLI: Saved
    CLI->>Viz: plot_focus_curve (MSE & R² vs. N receptors)
    Viz-->>CLI: PNG generated
    CLI->>User: Print summary + exit
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 Two scripts hop into the fray,
Ablations and focus to test the way,
Receptors ablated, weights ranked high,
Robustness measured with a keen eye!
Behold the curves and the summary stats,
Code review, pass — no ifs or brats! 🌿

✨ Finishing touches
  • 📝 Generate docstrings

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bc9234f and 1ed3db4.

📒 Files selected for processing (8)
  • README.md
  • docs/BEHAVIORAL_PREDICTION_ANALYSIS.md
  • scripts/lasso_with_ablations.py
  • scripts/lasso_with_focus_mode.py
  • src/door_toolkit/pathways/__init__.py
  • src/door_toolkit/pathways/behavioral_prediction.py
  • tests/test_lasso_ablation.py
  • tests/test_lasso_focus_mode.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@colehanan1 colehanan1 merged commit 3225982 into main Jan 8, 2026
0 of 16 checks passed
@colehanan1 colehanan1 deleted the feature/lasso-ablation-analysis branch January 8, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants