compressed initramfs

[Villemoes]

I'm playing around with the initramfs decompression code, so it's nice to be able to provide a compressed initramfs in virtme.

[amluto]

I'm okay with the concept and the interface, but I don't like piping through cat. Also, Python has modules for most of these. Maybe it could be something like:

def open_with_compression(filename, compressor):
    if compressor == 'gzip':
        return gzip.open(filename, ...)
    elif compressor == 'zstandard':
        import zstandard
        return zstandard.whatever(...)

I'm also not convinced that -9 is the right default.

[Villemoes]

I'm okay with the concept and the interface, but I don't like piping through cat.

Why not? It's not performance critical, and even so, I don't think it will ever be noticed.

Also, Python has modules for most of these. Maybe it could be something like:

Perhaps, but see below.

I'm also not convinced that -9 is the right default.

The command lines are taken from what the kernel itself would do with CONFIG_INITRAMFS_SOURCE (and other producers of initramfs images, e.g. yocto, also use those parameters), so it seemed most sensible to use the same when the idea was to test tweaks of the decompression/unpacking code. I'm not sure the python wrappers could easily be convinced to apply those same parameters, and e.g. for lz4, it's not obvious they'd even have support for the equivalent of the required -l flag.

I can change to do open_with_compression and avoid cat for the none case, but I do think this needs to invoke the actual compressors with the same parameters that would be used IRL.

[amluto]

I'm okay with the concept and the interface, but I don't like piping through cat.

Why not? It's not performance critical, and even so, I don't think it will ever be noticed.

Also, Python has modules for most of these. Maybe it could be something like:

Perhaps, but see below.

The idea would be to have fewer dependencies. Python includes most of the compressors you're using, and avoiding digging them out of the path will avoid admittedly rare failure cases.

I'm also not convinced that -9 is the right default.

The command lines are taken from what the kernel itself would do with CONFIG_INITRAMFS_SOURCE (and other producers of initramfs images, e.g. yocto, also use those parameters), so it seemed most sensible to use the same when the idea was to test tweaks of the decompression/unpacking code. I'm not sure the python wrappers could easily be convinced to apply those same parameters, and e.g. for lz4, it's not obvious they'd even have support for the equivalent of the required -l flag.

I can change to do open_with_compression and avoid cat for the none case, but I do think this needs to invoke the actual compressors with the same parameters that would be used IRL.

I'm okay with that, although, if the goal is performance, then there ought to be a default that's fast, and -9 is probably not optimal for a blob that will be decompressed exactly once.

But the goal right now seems to be testing, so I'm okay with your defaults.