Performance Issue with Multiband Images in COREG_LOCAL

[SARcycle]

Performance Issue with Multiband Images in COREG_LOCAL

Environment

• AROSICS version: 1.11.0
• Python version: 3.10.18
• OS: Ubuntu 24.04.2 LTS x86_64
• Kernel: 6.11.0-29-generic
• CPU: AMD Ryzen Threadripper PRO 5975WX
• GPU: NVIDIA GeForce RTX 4080
• Memory: 128 GB

Issue Description

I'm experiencing significant performance degradation when using COREG_LOCAL with multiband images compared to single-band images, despite the documentation indicating that coregistration should use only a single band regardless of input format.

Test Setup

Data: Sentinel-2 10m tiles covering Switzerland (date: 20250619, orbit: 65)

• Downloaded from browser.dataspace.copernicus.eu
• Mosaicked using gdalbuildvrt to create multiband VRT (bands B02, B03, B04, B08)
• Reference image extent equalized to match multiband VRT
• Data validated and visualizable in QGIS

Code Used

kwargs = {
    'path_out': 'S2_mosaic_coreg.tif',
    'projectDir': '/path/to/folder',
    'q': False,
    'nodata': (0, 0),
    'out_crea_options': ['COMPRESS=DEFLATE', 'PREDICTOR=2'],
    'CPUs': 6,
    'progress': True,
    'fmt_out': 'GTIFF',
    'r_b4match': 1,
    's_b4match': 4,  # B04
    'window_size': [128, 128],
    'max_iter': 10,
    'max_shift': 5,
    'grid_res': 580,
}

# Perform coregistration
CRL = COREG_LOCAL(im_reference_masked, multiband_mosaic_10m, **kwargs)

Performance Issues Observed

1. Processing time:

   • Single band: ~10 minutes
   • Multiband (4 bands): ~3.5 hours (21x slower)

2. Memory consumption: Significantly higher with multiband input, requiring reduction to 6 CPUs

3. Expected behavior: Since s_b4match=4 specifies band 3 for coregistration, I expected similar performance to single-band processing

Workaround Attempted

I tried processing only B04 directly:

multiband_mosaic_10m = '/path/to/folder/S2-L2A-mosaic_20250619T101559_B04_10m.vrt'
del kwargs['s_b4match']
CRL = COREG_LOCAL(im_reference_masked, multiband_mosaic_10m, **kwargs)

# Then apply to multiband image
im_target = '/path/to/S2-L2A-multiband_20250619T101559_10m.vrt'
DeShifter.DESHIFTER(im2shift=str(im_target), coreg_results=coreg_info, **kwargs).correct_shifts()

This approach is faster but requires an additional step, which seems counterintuitive given the documentation.

Questions

1. Performance expectation: Should multiband and single-band coregistration have similar performance when using s_b4match parameter? Is my assumption correct?

2. Documentation interpretation: Does the current behavior align with the intended design, or am I misunderstanding the documentation?

3. Best practice: Is the two-step workaround (single-band coregistration + DeShifter) the recommended approach for multiband images?

4. Different GSD support: Is there a straightforward way to apply calculated CRL results to images with the same extent but different Ground Sample Distance (GSD)?

Additional Notes

The resulting coregistration from the multiband approach appears valid, but the performance impact makes it impractical for routine processing of large datasets.

Thank you for your time and for maintaining this valuable tool. I'm happy to provide additional information or test potential solutions if needed.

[danschef]

Thanks for reporting this, sounds reasonable to me. I am in vacation until end of July, so here are just some quick thoughts:

The runtime until warping and writing the final result should be similar to the single band case because there is indeed only one band used for tie point generation.

Could you provide the output (STDOUT logs) to get a idea what exactly takes so long? What are the image dimensions of the input images?

The issue might also be related to the VRT format. It might be worth to stack the bands to a native multiband image and try again.

In case the reference image has a much larger extent than the target image, I would recommend to first crop the reference image to a similar extent and then run AROSICS.

[SARcycle]

Response to Performance Issue Questions

Thank you for the quick response! Here are the details you requested:

STDOUT Logs and Performance Summary

I've run comprehensive tests with different formats and captured the logs. Here's the performance breakdown:

Format	Processing Time	Tie Point Calculation Time
Single-band VRT	~14 minutes	13:47
Single-band TIF	~14 minutes	13:47
Multiband VRT	~3.75 hours	3:45:26
Multiband TIF	~51 minutes	50:51

Key finding: The bottleneck is specifically in the tie point grid calculation phase, not in warping/writing.

Image Dimensions

Reference image (SPOT5, masked):

• Size: 81,960 × 81,960 pixels
• Pixel size: 5m × 5m
• Data type: Byte
• File size: ~4.9 GB

Target image (Sentinel-2 multiband):

• Size: 40,980 × 40,980 pixels
• Pixel size: 10m × 10m
• Data type: UInt16
• Multiband TIF size: ~13.4 GB
• Multiband VRT size: 2,925 bytes (text file)

Both images cover the same geographic extent with identical corner coordinates. The same is true for the single band images.

STDOUT Logs Comparison

Single-band processing (VRT and TIF - identical results):

Calculating footprint polygon and actual data corner coordinates for reference image...
Polygonize progress     |==================================================| 100.0% Complete  => 0:01:04
Calculating footprint polygon and actual data corner coordinates for image to be shifted...
Polygonize progress     |==================================================| 100.0% Complete  => 0:00:15
Matching window position (X,Y): 550681.846039048/5188658.486443013
Initializing tie points grid...
Equalizing pixel grids and projections of reference and target image...
Warping progress     |==================================================| 100.0% Complete  => 0:01:15
Warping progress     |==================================================| 100.0% Complete  => 0:00:09
Calculating tie point grid (3541 points) using 6 CPU cores...
        progress: |==================================================| 100.0% Complete  => 0:13:47
Found 3227 matches.
[... filtering steps identical ...]
1585 valid tie points remain after filtering.
Correcting geometric shifts...
Translating progress |==================================================| 100.0% Complete  => 0:00:01
Warping progress     |==================================================| 100.0% Complete  => 0:00:15
Writing GeoArray of size (40980, 40980) to [output_path]

Multiband VRT processing:

[... identical initial steps ...]
Calculating tie point grid (3541 points) using 6 CPU cores...
        progress: |==================================================| 100.0% Complete  => 3:45:26
Found 3227 matches.
[... identical filtering and final steps ...]
Writing GeoArray of size (40980, 40980, 4) to [output_path]

Multiband TIF processing:

[... identical initial steps ...]
Calculating tie point grid (3541 points) using 6 CPU cores...
        progress: |==================================================| 100.0% Complete  => 0:50:51
Found 3227 matches.
[... identical filtering and final steps ...]
Writing GeoArray of size (40980, 40980, 4) to [output_path]

VRT Structure

The multiband VRT is structured as nested VRTs - each band references another VRT file that was created by mosaicking individual Sentinel-2 granules:

<VRTDataset rasterXSize="40980" rasterYSize="40980">
  <VRTRasterBand dataType="UInt16" band="1">
    <ComplexSource>
      <SourceFilename relativeToVRT="1">S2-L2A-mosaic_20250619T101559_B02_10m.vrt</SourceFilename>
      <SourceBand>1</SourceBand>
      [... properties ...]
    </ComplexSource>
  </VRTRasterBand>
  [... similar structure for bands 2-4 ...]
</VRTDataset>

Memory Usage Patterns

• Initial footprint calculation: RAM usage spikes to >90 GB
• Tie point calculation: Stabilizes around 45-70 GB with periodic spikes every ~5 seconds
• CPU usage: 6 CPUs for all tests (limited by memory constraints)

Key Observations

1. Consistent tie point grids: All tests calculated the same 3541 tie points and found 3227 matches, confirming that the coregistration logic is using only the specified band (s_b4match=4). However, there are minute differences in results between single-band and multiband approaches, which is unexpected given that the same band should be used for coregistration in both cases.

2. VRT format impact: The dramatic difference between multiband VRT (3:45:26) and multiband TIF (45 minutes) suggests the issue is specifically with how nested VRT files are handled during tie point calculation.

3. Format-specific performance: Multiband TIF (50:51) performs significantly better than multiband VRT (3:45:26), while single-band formats show identical performance regardless of VRT vs TIF.

Questions for Further Investigation

1. Could the nested VRT structure be causing inefficient data access patterns during tie point calculation?
2. Is there a way to optimize VRT handling for multiband coregistration? The VRT format is preferred for operational use as it avoids data duplication and minimizes storage requirements (2.9 KB VRT vs 13.4 GB TIF in this case).
3. What could cause the minute differences in coregistration results between single-band and multiband approaches when the same band is being used?
4. Would it be beneficial to add a warning about performance implications when detecting nested VRT inputs?

The workaround using single-band coregistration followed by DeShifter.DESHIFTER would be an option, but optimizing the multiband VRT approach would be the ideal solution for operational workflows.

[danschef]

Thanks for the detailed investigation. The issue might be related to how the input bands are shared among multiprocessing workers when computing the tie points. Looks like I need to take a closer look at that after my vacation (end of July).