The SQLite database gets super slow when it's really big.

[darcyabjones]

This is only really a problem for me right now, but may be an issue in future for others if they use a lot of precomputed data.

Essentially the sqlite defaults to some very safe options that we don't need.
So we can free up some locks, increase cache memory, use transactions better etc.

I think maybe having dedicated commands for creating, updating and deduplicating the database will be good.
Rather than handling ldjson in the precomputed script.

[darcyabjones]

I've updated scripts to make some better use of SQLite, but its still pretty slow.
I think creating separate tables with the same schema for each analysis should help with performance, since the b-tree indexes don't get so big.
Hopefully having to UNION them all in ranking doesn't slow down things too much.