Feat/add server side compression support

[caue-paiva]

For motivation see the following issue: #3452

Solution Overview: Server-Side Compression

This implementation provides a server-side only compression solution for dcc.Store components. The compression and decompression occur entirely on the server during callback execution, with the following key characteristics:

• Automatic compression/decompression: Transparent to the developer once configured
• Callback-level scope: Each callback can have its own compression configuration
• Server-side processing: No client-side JavaScript modifications required
• Graceful fallbacks: Robust error handling with fallback to uncompressed data

Important Note: This is explicitly a server-side solution. The client-side JavaScript cannot access or manipulate the compressed data - it only receives the final decompressed JSON payload.

Implementation Details

New Components Added

1. BaseStoreCompressionManager: Abstract base class defining the compression interface
2. GzipCompressionManager: Implementation using gzip algorithm (default)
3. DeflateCompressionManager: Implementation using deflate algorithm
4. BrotliCompressionManager: Implementation using Brotli algorithm (optional dependency)
5. Helper function: get_compression_manager_from_kwargs() for parameter extraction

Core Functionality

• Configurable compression levels: Algorithm-specific compression levels (1-9 for gzip/deflate, 0-11 for Brotli)
• Size thresholds: Configurable minimum data size before compression is applied (default: 0 bytes)
• Algorithm selection: Support for gzip, deflate, and Brotli compression algorithms
• Automatic detection: Smart compression effectiveness checking with fallback to original data
• Store component targeting: Only compresses dcc.Store component data properties

Integration with Callback System

The compression manager integrates seamlessly into the existing callback architecture:

• Added CompressionManager parameter to callback decorators
• Integrated compression/decompression into both sync and async callback wrappers
• Maintains compatibility with existing callback features (background callbacks, clientside callbacks, etc.)
• Uses kwargs-only approach for consistency with other optional callback parameters

Usage Example

# Create compression manager with custom settings
compression_manager = GzipCompressionManager(
    level=6,        # Compression level (1-9)
    threshold=1024  # Only compress data larger than 1KB
)

app.layout = html.Div([
    html.Button("Load Large Dataset", id="load-btn"),
    dcc.Store(id="data-store"),
    html.Div(id="output")
])

@app.callback(
    Output("data-store", "data"),
    Input("load-btn", "n_clicks"),
    CompressionManager=compression_manager
)
def load_large_dataset(n_clicks):
    if n_clicks:
        # Generate large dataset (would typically be from database/API)
        large_df = pd.DataFrame({
            'col1': range(10000),
            'col2': ['data'] * 10000,
            'col3': [i * 2.5 for i in range(10000)]
        })
        return large_df.to_dict('records')
    return dash.no_update

@app.callback(
    Output("output", "children"),
    Input("data-store", "data"),
    CompressionManager=compression_manager  # Same or different manager
)
def process_data(data):
    if data:
        return f"Processed {len(data)} records"
    return "No data loaded"

Summary

This feature provides a server-side compression solution for dcc.Store components that:

• Significantly reduces network payload sizes for large datasets
• Maintains full backward compatibility
• Offers flexible configuration options
• Includes comprehensive test coverage
• Follows existing Dash architectural patterns

The implementation successfully addresses the performance bottlenecks identified in the original issue while maintaining the simplicity and reliability expected from the Dash framework.

Contributor Checklist

• I have broken down my PR scope into the following TODO tasks
  • Implement compression functionality in a new file _compression.py
  • Allow for callback functionality to use the compression feature though injecting the compression manager class with kwargs (as argument to the @callback decorator)
  • I have written new unit tests to test the new compression functionality and how it interacts with the callback decorator
• I have run the tests locally and they passed. (refer to testing section in contributing)
• I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR

optionals

• I have added entry in the CHANGELOG.md
• If this PR needs a follow-up in dash docs, community thread, I have mentioned the relevant URLS as follows
  • this GitHub #PR number updates the dash docs
  • here is the show and tell thread in Plotly Dash community

[gvwilson]

@caue-paiva thanks very much for the PR - I'll ask @T4rk1n to look it over when he gets a chance, but it may be a week or more before he can make time to evaluate it.

[caue-paiva]

Hello, could any of the reviewers take a look at this? I think the visual test is flaky

[T4rk1n]

I think the visual test is flaky

Yes the percy tests are flaky, no worry as long as the tests pass it's really just to look at them if it's expected some change from the frontend code.

[caue-paiva]

Hi guys, i am counting on this PR for an university project (that involves open-source contributions), could we get some reviews going?

[BSd3v]

Hello @caue-paiva,

I am just a fellow contributor here, but I think that this makes a dcc.Store unusable on the clientside, as you'd have to decompress the data. I have an open feature request / PR sorta (#3471) that allows for intercepting the data on the server side and replacing it with server side stores of variables. I feel that this would solve the issue that you are trying to handle.

As far as larger payloads, it could be better to compress the entire payload from the client with a header, and compressing the whole body to go to the server.

Just my thoughts on the matter. 🙂

[robertclaus]

@T4rk1n could you take a look at this PR? I think @BSd3v is right that this isn't a complete solution since it wouldn't play nicely with clientside callbacks. However, those cases would have more limited network traffic anyways - so compression wouldn't be as important to start with.

[T4rk1n]

I think the proposed api with the explicit callback argument might not the best approach for this.

The compression_manager could be an optional argument of dash.Dash instead and provide a default based on installed lib:

• if brotli is installed take that.
• next default to gzip
• can be set to None to disable compression.

If we go the default route with gzip, we'd have to also disable the compression if the dcc.Store is used in a clientside callback.

The compression should also be defined at the component level, I think a prop system like the persistence we have in the defaultProps

Example for the store:

Store.propTypes = {
   ...
   compression: PropTypes.boolean,
   compressed_props: PropTypes.arrayOf(PropTypes.oneOf(['data']),
   compression_type: PropTypes.oneOf(['gzip', 'zlib', 'brotli'])
}

Store.defaultProps = {
  ...,
  compressed_props: ['data'],
}

This way the compression could be used in other props like for dataframes and the component can handle the compression on the client too if required.