Resolve review comments: Fix unit tests and cleanup artifacts (new) #33

anurag-r20 · 2025-11-22T21:58:20Z

I have pushed updates to address all the review comments: I am creating a new PR because the previous PR was merged without the updates

Unit Tests: I reverted test_stochastic_benchmark.py, test_stats_pandas.py, and test_training_pandas.py to their original logic (restoring the names library and mock setups) to ensure the tests run correctly against the codebase.
Interpolate Test: I reverted the fixture to use the original manual parameters and added a reset_index fix to handle the MultiIndex output correctly while keeping the strict assertions.
Formatting: Applied the requested whitespace/docstring formatting.
Artifacts: I updated .gitignore to handle the specific example folders and removed the accidental JSON file.

…leanup notebooks

review-notebook-app · 2025-11-22T21:58:25Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

anurag-r20 · 2025-11-23T01:43:40Z

This new commit for .gitignore addresses your comment from the previous PR on whether it ignores the data for other examples as well.

bernalde · 2025-11-23T20:27:22Z

examples/QEDC_to_WS_conversion/conversion.ipynb

Let's rerun this from scratch or not touch it

I think the data for this is not available within stochastic_benchmark itself. In the notebook it says to go to __results in the QED-C repository, but I do not find it there.

tests/test_interpolate_pandas.py

.gitignore

- Expand IBM_QAOA patterns to all examples subdirectories - Add general artifact patterns (plots, checkpoints, progress) - Add data file patterns (pkl, npz, npy) - Ignore accidentally created repo root directory

Add comprehensive test fixtures for IBM QAOA data processing: - 4 real experimental JSON files with varied instance/depth configurations - 4 synthetic edge case fixtures (multi-trial, missing fields, empty) - README documenting schema, boundaries, and usage patterns Fixtures support testing of: - Single vs multi-trial bootstrap behavior - IBM-specific filename parsing - Missing field handling - Empty/malformed data edge cases

Add 32 tests organized in 6 test classes covering: Unit Tests: - QAOAResult dataclass creation - parse_qaoa_trial with various data formats - load_qaoa_results with missing/malformed data - convert_to_dataframe transformations - group_name_fcn filename parsing - prepare_stochastic_benchmark_data pickle I/O Integration Tests: - process_qaoa_data end-to-end pipeline - GTMinEnergy injection for missing ground truth - Single-trial bootstrap fabrication - Interpolation fallback behavior - Train/test split generation Edge Cases: - Missing trainer information - Missing optimal parameters - Empty trials list - Multi-trial synthetic data All tests use fixtures and proper mocking for multiprocessing. Test coverage validates IBM-specific logic boundaries.

Implement 4 high-impact optimizations for ~1K file scale: 1. ProcessingConfig dataclass for centralized configuration: - persist_raw: gate pickle writes during ingestion - interpolate_diversity_threshold: diversity-based interpolation - fabricate_single_trial: control single-trial bootstrap - seed: reproducible train/test splits - log_progress_interval: configurable progress logging 2. Structured logging infrastructure: - Replace print statements with logging module - Add progress logging every N files - Proper INFO/WARNING levels for errors - Timestamps and levels for production observability 3. In-memory aggregation with conditional pickle persistence: - persist_raw=True: write pickles to exp_raw/ subdirectory - persist_raw=False: aggregate in memory, skip ingestion pickles - Generate temporary pickles only when needed for bootstrap - Expected 1-2s savings for 1K files when disabled 4. Diversity-based interpolation heuristic: - Replace row count (n_rows <= 5) with diversity metric - diversity = unique_instances × unique_depths - Skip interpolation when diversity < threshold - Prevents spurious skips on sparse but valid grids Additional improvements: - Add try/except for malformed JSON files with warnings - Use config.seed for reproducible train/test splits - Fix pickle paths to use exp_raw subdirectory convention - Add enumeration to ingestion loop for progress tracking All changes maintain backward compatibility with default config. Expected performance improvement: ~15s for 1K files (from ~20-25s).

Add comprehensive performance optimization guide: Phase 1 - Implemented (4 changes): - ProcessingConfig dataclass for centralized configuration - Structured logging infrastructure - In-memory aggregation with persist_raw flag - Diversity-based interpolation heuristic Phase 2 - Deferred Enhancements (6 optimizations): 1. Parallel I/O with ThreadPoolExecutor (3-5x potential speedup) 2. Parquet output format (faster writes, smaller files) 3. orjson for JSON parsing (~2x speedup) 4. Lazy bootstrap fabrication (skip unnecessary computation) 5. Categorical dtypes for memory efficiency 6. Rich diversity metrics (entropy-based quality assessment) Each enhancement documented with: - Problem/solution description - Expected impact and thresholds - Implementation complexity - Testing requirements Target metrics: - Phase 1: <15s for 1K files (from ~20-25s baseline) - Phase 2: <10s with parallelization - Scale guidance: When to apply each optimization

Clear execution outputs and intermediate results to reduce repo size. Notebook structure and analysis code preserved.

Add ibm_qaoa_analysis_hardware.ipynb for analyzing real quantum hardware results from IBM systems. Complements simulation analysis with hardware- specific metrics and comparisons.

bernalde · 2025-11-25T00:46:33Z

tests/fixtures/ibm_qaoa/README.md

+- `###` - Instance ID (e.g., 000, 001, 002)
+- `N##R3R` - Problem size indicator
+- `_#.json` - Depth parameter (p)
+


Can you verify if this is true @anurag-r20 ?

Except N is the problem size and R3R is the type of graph. p is not the depth

In that case, let's modify this file

Let's split the N## line from the R#R lkine and explain all the alternatives (heavy-hex, Erdos-renyi, ...)

tests/fixtures/ibm_qaoa/README.md

bernalde · 2025-11-25T00:48:25Z

examples/IBM_QAOA/ibm_qaoa_analysis_hardware.ipynb

Is this ready for review? There is a lot of commented code and unavailable code. Some of it seemed copied from the ibm_qaoa_analysis notebook

No that is work in progress,

In that case, let's remove it from this PR to have this merged

Move all pandas-specific test files from tests/Pandas_Group_Tests/ to tests/: - test_interpolate_pandas.py - test_stats_pandas.py - test_stochastic_benchmark_pandas.py - test_training_pandas.py Remove empty Pandas_Group_Tests subdirectory for better test organization. All 13 tests still passing after move.

- Update processing script to detect three optimization states: 'opt', 'noOpt', and None - Add marker differentiation in plotting: circles for opt, x for noOpt, squares for no flag - Use depth-specific colors for all marker types with appropriate legend labels - Extract optimization flag from filename patterns (_opt_, _noOpt_, or neither) - Fallback to Energy metric when Approximation Ratio not available in JSON

bernalde · 2025-11-25T23:26:05Z

examples/IBM_QAOA/ibm_qaoa_analysis.ipynb

This needs to be rerun without the Opt vs noOpt distinction as they are two separate solvers. In fact, have the solver_namemerge them

This needs to be run from scratch

examples/IBM_QAOA/ibm_qaoa_processing.py

bernalde · 2025-11-25T23:28:17Z

tests/fixtures/ibm_qaoa/README.md

+- `###` - Instance ID (e.g., 000, 001, 002)
+- `N##R3R` - Problem size indicator
+- `_#.json` - Depth parameter (p)
+


Let's split the N## line from the R#R lkine and explain all the alternatives (heavy-hex, Erdos-renyi, ...)

- Load minmax cuts from JSON files in R3R/minmax_cuts directory - Add maxcut_approximation_ratio() function using formula: cut_val = energy + 0.5 * sum_weights approx_ratio = (cut_val - min_cut) / (max_cut - min_cut) - Update convert_to_dataframe() to use calculated approximation ratios - Update process_qaoa_data() to load and pass minmax data - Add proper error handling and validation for edge cases

- Test minmax cuts loading from directory - Test approximation ratio calculation - Test end-to-end processing with minmax integration - Verify non-NaN approximation ratios in output

…d comparison - Update data loading to use 'optimized' column from process_qaoa_data - Create separate method names (FA_opt, FA_noOpt, TQA_opt, TQA_noOpt) - Update methods_to_compare list to include all 7 method variants - Change legends to single-column layout for better readability - Invalidate cache to force reprocessing with new method names - All variants treated independently in statistical analysis and rankings

bernalde · 2026-01-07T00:41:49Z

examples/IBM_QAOA/ibm_qaoa_analysis.ipynb

This needs to be run from scratch

Fix unit tests, apply whitespace formatting, update .gitignore, and c…

7aa1177

…leanup notebooks

anurag-r20 requested a review from bernalde November 23, 2025 00:48

Changed .gitignore to only ignore data and files relevant to IBM_QAOA

f39b4f9

bernalde requested changes Nov 23, 2025

View reviewed changes

anurag-r20 added 7 commits November 24, 2025 19:00

chore: update .gitignore for examples and data artifacts

93dbe43

- Expand IBM_QAOA patterns to all examples subdirectories - Add general artifact patterns (plots, checkpoints, progress) - Add data file patterns (pkl, npz, npy) - Ignore accidentally created repo root directory

chore: clear notebook outputs in ibm_qaoa_analysis

0d9bff8

Clear execution outputs and intermediate results to reduce repo size. Notebook structure and analysis code preserved.

feat: add hardware analysis notebook for IBM QAOA

bef72e2

Add ibm_qaoa_analysis_hardware.ipynb for analyzing real quantum hardware results from IBM systems. Complements simulation analysis with hardware- specific metrics and comparisons.

bernalde requested changes Nov 25, 2025

View reviewed changes

anurag-r20 added 3 commits November 24, 2025 21:35

README.md edit minor corrections in .json file nomencalture

cba3085

bernalde requested changes Nov 25, 2025

View reviewed changes

anurag-r20 added 4 commits December 22, 2025 02:51

Add integration test for minmax cuts functionality

009316a

- Test minmax cuts loading from directory - Test approximation ratio calculation - Test end-to-end processing with minmax integration - Verify non-NaN approximation ratios in output

Update analysis notebook with latest cell outputs

b533a7b

bernalde requested changes Jan 7, 2026

View reviewed changes

examples/IBM_QAOA/ibm_qaoa_analysis.ipynb

Copy link

Owner

bernalde Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be run from scratch

Resolve review comments: Fix unit tests and cleanup artifacts (new) #33

Are you sure you want to change the base?

Resolve review comments: Fix unit tests and cleanup artifacts (new) #33

Uh oh!

Conversation

anurag-r20 commented Nov 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app bot commented Nov 22, 2025

Uh oh!

anurag-r20 commented Nov 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

anurag-r20 commented Nov 22, 2025 •

edited

Loading