MVP gnark GPU acceleration

[huitseeker]

We want to:

• accelerate the final gnark proof using the GPU + Ingonyama's Icicle,
• benchmark the speedups,
• integrate the choice of hardware, deployment instructions in the release package.

This would be guesstimated to reduce our e2e proof time by -5min.

[wwared]

This would be guesstimated to reduce our e2e proof time by -5min.

The real estimate of savings is actually much lower, because this only accelerates the final gnark groth16 proof, which is only about ~10% or ~40s of the total proof time:

prove_core: 79.436438622
compress: 222.975319382
shrink: 40.479668072
wrap: 50.424116137
-> gnark groth16: 46.502766146
Proving took 439.818354007s

These savings are also only present in Groth16 for gnark, so when we move to Plonk, we would lose out on these gains unless we implemented Icicle support for the Plonk backend in gnark (doable, but non-trivial)

[huitseeker]

Understood. How do those numbers compare to non-accelerated proving?

[wwared]

Understood. How do those numbers compare to non-accelerated proving?

Sorry for the confusion, the numbers I pasted above are for non-accelerated CPU-only proving, on an AWS R7 bare metal machine.

---

Regarding this issue, I unfortunately was unable to get the gnark gpu acceleration using icicle working. I opened an issue on gnark describing the main problems: Consensys/gnark#1166

The summary is:

1. The icicle backend in gnark is out of date, and updates to the internal API are causing compile errors
2. Gnark is depending on an old version of icicle (from Sep 28, 2023), which has different build instructions from current icicle (and there are about 10 months of commits and possible performance improvements it's not making use of)
3. Fixing the compile errors naively (https://github.com/wwared/gnark/commit/186e9de8321f7933e42f4d1b01c40a25f33fd4a5) enabled me to compile gnark-recursion-ffi with the icicle backend active and enabled, and it would even generate a proof using the GPU, but the generated proof would fail to verify

Furthermore:

1. In the CPU-only proving scenario, the gnark step only represents 10% of the total prover time, so any gnark GPU acceleration improvements are limited to that 10% only
2. We expect to move away from Groth16 and into Plonk as we port more PRs, but currently the only supported backend for gnark is Groth16. There is however an unmerged WIP PR for adding an icicle-accelerated Plonk/BN254 backend to gnark (FEAT: Add Icicle GPU-Accelleration to Plonk Prover behind build tag Consensys/gnark#1051) but we would not be able to use it as-is

My takeaway is the following:

1. Accelerating the gnark step will not give us massive performance gains when it comes to the total proof time (but it's still likely to be worth it to chase down upstream/ingonyama if we can to ensure the support is there and we can use it)
2. There are upstream issues with the current state of Icicle/Gnark integration that prevents us from using it right now
3. Moving away from Groth16 and its trusted setup issues and into Plonk is likely worth it, even if there is a performance hit to the prover, and it will further impact the GPU acceleration backend (we would need to pick up that WIP PR and update it, so it's still possible to gain back GPU acceleration after the switch)
4. Eventually, once the Icicle backend in Gnark is updated/fixed and supporting both Groth16 and Plonk, the changes needed to support GPU acceleration in Sphinx are minimal (See here, it's basically just adding -tags=icicle to the go build command line and passing the proper linker args)

For now I think we put this issue on hold until upstream responds to the issue I opened and after we have Plonk performance numbers as well.

[huitseeker]

For now I think we put this issue on hold until upstream responds to the issue I opened and after we have Plonk performance numbers as well.

Agreed!