Column generation MVP #2
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ementation Add comprehensive column generation implementation based on Coluna.jl patterns: - Core ColGen module with generic API functions - Default implementation with Dantzig-Wolfe reformulation support - Test module with GAP example using ReformulationKit - Integration with existing test infrastructure
Add comprehensive helper functions for creating variables and constraints in MOI models: - add_variable! with keyword arguments for bounds, types, and coefficients - add_constraint! using direct MOI constraint set instances - Type-safe API using MOI.Integer(), MOI.ZeroOne(), MOI.LessThan(), etc. - Comprehensive test suite with 68 test assertions - Support for constraint coefficient updates and objective coefficient setting This provides a clean, Julia-idiomatic interface for column generation algorithms while maintaining direct MOI integration without abstraction layers.
- Add configurable artificial_var_cost field to MixedPhase1and2 struct - Implement complete setup_reformulation\! method handling all constraint types: * Equality constraints: ax = b → ax + s⁺ - s⁻ = b (2 artificial variables) * ≤ constraints: ax ≤ b → ax + s = b (1 artificial variable) * ≥ constraints: ax ≥ b → ax - s = b (1 artificial variable) - Add comprehensive test suite with 43 test assertions - Fix JuMP Model vs MOI backend compatibility issue - Support objective sense detection for proper cost assignment
- Rename src/ColGen/impl.jl to src/ColGen/dw_colgen.jl - Rename ColGenDefaultImplementation to DantzigWolfeColGenImpl - Update all function signatures and references throughout the codebase - Update include statement in ColGen.jl module - All tests pass successfully
… file - Create new test/ColGenTests/dw_colgen.jl for Dantzig-Wolfe column generation tests - Move artificial variable tests from helpers.jl to dw_colgen.jl - Update ColGenTests.jl to include and run the new dw_colgen tests - All 43 artificial variable test assertions still pass - Clean separation between helper function tests and algorithm-specific tests
…sting - Add type-safe dictionaries to DantzigWolfeColGenImpl for tracking artificial variables: * eq_art_vars: equality constraints → tuple of (s⁺, s⁻) variables * leq_art_vars: ≤ constraints → single artificial variable * geq_art_vars: ≥ constraints → single artificial variable - Update setup_reformulation\! to populate tracking dictionaries and detect convexity constraints - Convexity constraint artificial variables get 10x higher cost (10000 vs 1000) - Fix method signature compatibility in coluna.jl - Add comprehensive test with proper RK.DantzigWolfeReformulation: * Tests artificial variable creation for all constraint types * Verifies correct coefficients: +1.0 for ≤, -1.0 for ≥, ±1.0 for equality * Tests convexity constraints get higher artificial variable costs * Validates proper integration with ReformulationKit framework All tests pass with complete MOI type safety and no performance overhead.
- Add convexity_artificial_var_cost field to MixedPhase1and2 struct - Update constructor to accept both regular and convexity costs with backward compatibility - Replace hardcoded 10x multiplier in setup_reformulation\! with configurable cost - Update test to explicitly configure convexity artificial variable cost - Maintain default behavior: convexity cost = 10x regular cost when not specified
Refactor Dantzig-Wolfe column generation implementation by extracting iteration logic and stabilization into dedicated files for better code organization. - Extract master/pricing optimization logic into dw_colgen_iteration.jl - Create dw_stabilization.jl with stabilization method stubs - Update Master struct to wrap MOI backend with artificial variable tracking - Reorganize function definitions across modules for cleaner separation
- Add validation in run_column_generation to check optimizer attachment - Add assert in DantzigWolfeColGenImpl constructor for safety - Throw clear error with fix instructions when no optimizer found - Add comprehensive test suite covering error and success cases - Update example to properly attach optimizer after decomposition Users must now attach optimizer to master after decomposition: JuMP.set_optimizer(ReformulationKit.master(reformulation), HiGHS.Optimizer)
- Expand MasterPrimalSolution struct to include variable_values field - Update optimize_master_lp_problem\! to retrieve primal values for all variables - Add comprehensive tests for variable value storage and retrieval - Fix get_primal_sol function signature to properly reference parameter - Update test to use get_primal_sol accessor function instead of direct field access - Validate expected optimal solution values (x1=5, x2=0) with objective value 5.0 🤖 Generated with [Claude Code](https://claude.ai/code)
- Add PricingSubproblem struct with MOI model and ReformulationKit mappings - Implement compute_reduced_costs! using efficient mapping structures - Add update_reduced_costs! to modify subproblem objectives - Include comprehensive unit test with mathematical verification (c - y^T × A)
- Add test_update_reduced_costs_basic() to verify objective coefficient updates - Test creates JuMP subproblem with 3 variables and known reduced costs - Verifies that update_reduced_costs! correctly modifies MOI objective coefficients - All 19 tests in solution integration test suite now pass
…ng solutions - Add comprehensive pricing solution structure with PricingPrimalMoiSolution - Implement optimize_pricing_problem! to solve subproblems and extract solutions - Add DefaultPricingStrategy for systematic subproblem iteration - Fix artificial variable coefficients for proper constraint violation handling - Enhance debugging output and code organization - Update pricing initialization bounds based on optimization sense
…em optimizer - Fix getter function parameters in PricingSolution (missing parameter names) - Set primal and dual bounds equal for exact phase pricing - Add optimizers to subproblems in test example - Add TODO comment for dual bound clarification
…ests - Update PricingPrimalMoiSolution to include subproblem_id field - Add _compute_original_column_cost private method using OriginalCostMapping - Add _compute_master_constraint_membership private method for A*x computation - Implement complete insert_columns! function using add_variable! helper - Fix get_obj_val to access primal_sol.obj_value correctly - Update optimize_pricing_problem! signature to pass subproblem ID - Add comprehensive unit tests for both private methods - Clean up debug output and function signatures - Update stabilization function signatures for new column types
…mposition Implement complete dual bound calculation with convexity and subproblem contributions. Add helper functions for convexity constraint handling and multiplicity bounds.
- Add is_improving attribute to PricingPrimalMoiSolution to track improving reduced costs - Filter columns based on reduced cost sign (negative for minimization, positive for maximization) - Enhance ColGenIterationOutput to store iteration data for logging - Add comprehensive iteration logging showing iteration number, columns added, dual bound, LP objective, and IP bound - Update tests to include new is_improving parameter in constructors - Remove debug output from coluna.jl Note: Column filtering logic may need refinement as it currently prevents all columns after first iteration
…plified testing
- Transform DantzigWolfeColGenImpl from monolithic struct to parametric type DantzigWolfeColGenImpl{M, P}
- Add provider pattern with ReformulationMasterProvider and ReformulationPricingSubprobsProvider
- Move convexity constraints from reformulation to Master struct with type parameter
- Create test utilities in test/ColGenTests/test_utils.jl with MockMaster and MockPricingSubprobs
- Add create_for_testing() factory function for one-line test setup
- Update all interface methods to use provider pattern
- Refactor _compute_master_constraint_membership() to accept Master instead of reformulation
- Update setup_reformulation! to delegate to provider implementation
- Clean separation between production and test code
Replace verbose MOI API calls with clean JuMP macros in wolsey_integration.jl: - Use @variable, @constraint, @objective instead of manual MOI construction - Extract MOI backend via JuMP.backend() for ColGen API compatibility - Maintain full test functionality while improving code readability - Follow existing patterns used in other test files All tests pass with improved maintainability.
- Fix MixedPhase1and2 artificial variable cost defaults to 10000.0 - Correct ColGenIterationOutput constructor to use master_lp_dual_sol instead of master_lp_primal_sol - Implement proper reduced cost calculation including convexity constraint contribution - Add _subproblem_convexity_contrib function to compute convexity dual values - Update improving solution detection with tolerance (1e-6) instead of zero - Add debug output for reduced cost values and dual bound components - Add assertion for valid constraint references in add_variable\! helper
…el support - Add Base.show methods for MasterPrimalSolution with multiple model type support: - JuMP.Model: converts MOI indices to VariableRef for accessing JuMP variable names - MOI models: direct MOI.VariableName() access - Fallback method: uses _[index] format when no model provided - Implement formatted output with tree-like structure using | and └ connectors - Variables sorted by MOI index for consistent output - Add name parameter to add_variable\! and add_constraint\! helper functions - Create comprehensive test suite (34 test cases) covering: - Named and unnamed variables in both JuMP and MOI contexts - Mixed variable naming scenarios - Edge cases (empty solutions, invalid indices) - Output formatting verification - Integration ready: supports existing coluna.jl show() calls with JuMP models
…hancement - Add three Base.show methods for MasterDualSolution (MOI model, JuMP model, fallback) - Implement variable bounds detection and special formatting (e.g., "x >= 0.0: 2.5") - Add comprehensive constraint name resolution for both MOI and JuMP models - Include 52 passing tests covering all printing scenarios and edge cases - Support tree-like formatting with proper connectors (| and └)
- Add recompute_cost method for MasterDualSolution to manually compute dual objective value - Method multiplies dual values by their corresponding RHS values from constraint sets - Handles different constraint types: LessThan, GreaterThan, EqualTo - Graceful error handling for invalid/non-existent constraints - Add 5 comprehensive unit test functions covering: - Basic functionality with mixed constraint types - Empty dual solutions - Invalid constraint indices handling - Variable bounds constraints - Zero dual values - All tests verify correct dual cost computation independently of solver
Add comprehensive end-to-end tests for column generation using the Generalized Assignment Problem: - Create centralized GAP instance data function with 3 machines, 5 jobs - Test 1: Classic formulation with >= master constraints (working) - Test 2: Constant term in objective function - Test 3: Maximization objective with negative costs - Test 4: Equality constraints in master problem - Test 5: <= master constraints with negative coefficients Features: - Comprehensive test documentation explaining each test's purpose - Centralized data function for consistency across tests - Integration with existing ColGenTests module - Proper error handling and feasibility verification - Increased column generation iteration limit for better convergence Tests 2-5 marked as broken pending implementation fixes.
Refactoring changes: - Add HiGHS optimizer dependency to Project.toml - Remove old generalized assignment test function in favor of new GAP E2E tests - Remove Wolsey integration test file (functionality moved to other test files) - Clean up ColGenTests.jl by removing commented code and redundant functions - Streamline test runner to focus on new comprehensive GAP E2E test suite The new test structure is more organized and comprehensive while maintaining all essential functionality.
…afe wrappers - Add unified PrimalMoiSolution and DualMoiSolution base types in moi_solutions.jl - Implement MasterPrimalSolution and MasterDualSolution as wrapper types for type safety - Delegate Base.show methods from wrapper types to unified solution types - Update PricingPrimalMoiSolution to wrap unified PrimalMoiSolution - Fix all field access patterns to use wrapper.sol.field syntax - Add recompute_cost wrapper function for MasterDualSolution - Update all test files to use new constructor patterns and field access - Maintain backward compatibility while eliminating code duplication
Split monolithic dw_colgen_iteration.jl into focused modules: - master_optimization.jl: Master problem optimization and solutions - reduced_costs.jl: Reduced cost computation and updates - pricing_optimization.jl: Pricing strategies and solutions - dual_bounds.jl: Dual bound computation utilities - column_insertion.jl: Column cost computation and insertion - ip_management.jl: Integer programming management utilities Split corresponding test file into 6 test modules mirroring source structure. All 390+ tests pass with new modular organization.
│ _constr_sign function and update tests
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.