Skip to content

Cloud-AI provider image size is too large #7

@gibbotronic

Description

@gibbotronic

When using a cloud provider for LLM processing of photos, some can fail with image size errors. Eg. with Anthropic Haiku 3.5, I got:
image exceeds 5 MB maximum: 5607404 bytes > 5242880

As the image is base64-encoded, the base64 is 33% larger than the source image. Eg. a 5MB JPG will produce a 6.65MB base64 content that is sent to Anthropic / the LLM provider.

So we actually need to resize the JPG preview we are sending to be 33% smaller than 5MB, therefore 3.76MB or smaller.

I fixed this by running ImageMagick to pass the -define jpeg:extent=3932160 (which is actually 3.75MB) in ai-pp3/dist/file-operations/file-handlers.js:
At the top:

import fsp from "node:fs/promises";

then

export async function createPreviewImage({ inputPath, previewPath, basePP3Path, quality, format = "jpeg", verbose, }) {
    try {
        await (basePP3Path
            ? convertDngToImageWithPP3({
                input: inputPath,
                output: previewPath,
                pp3Path: basePP3Path,
                format,
                quality,
            })
            : convertDngToImage({
                input: inputPath,
                output: previewPath,
                format,
                quality,
            }));
        const tempPath = path.join(
          path.dirname(previewPath),
          `temp_${path.basename(previewPath)}`
        );
        await spawn('magick', [
          previewPath,
          "-strip",
          "-interlace", "Plane",
          "-sampling-factor", "4:2:0",
          '-define', 'jpeg:extent=3932160',
          tempPath
        ]);
        const stats = await fsp.stat(tempPath);
        if (stats.size > 3932160) {
          await spawn("magick", [
            previewPath,
            "-resize", "2000x2000>",        // shrink if too big (keeps aspect)
            "-strip",
            "-interlace", "Plane",
            "-sampling-factor", "4:2:0",
            "-quality", "75",
            tempPath
          ]);
        }
        if (verbose) {
          console.log(`Preview file created at ${tempPath}`);
        }
        await fsp.rename(tempPath, previewPath);
        if (verbose) {
          console.log(`Preview file renamed from ${tempPath} to ${previewPath}`);
          console.log(`Preview file created at ${previewPath}`);
        }
        return true;
    }
    catch (error) {
        if (error instanceof Error)
            throw error;
        throw new Error("Unknown error creating preview image");
    }
}

You can probably think of a better way of resizing the JPG to be 3.75MB in node.js but for now that's what I ran (ImageMagick) because it seemed to be able to resize to a target MB size already easily enough.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions