feat: Add Inference Device and SEV GPU Attestation Support

[jax-cn]

This PR introduces a new Inference Device (dev_inference) that provides an OpenAI-compatible API for local LLM inference. It also adds a SEV GPU Device (dev_sev_gpu) to support NVIDIA GPU TEE attestation, ensuring secure and verifiable inference workloads.

To support these features, the core HTTP handling logic has been extended to support Server-Sent Events (SSE) for streaming responses.

Key Changes

🚀 New Features

• Inference Device (dev_inference.erl):
  • Implements an OpenAI-compatible API (e.g., /v1/chat/completions).
  • Manages the lifecycle of a local Python-based deterministic inference server.
  • Supports streaming responses via SSE.
• GPU Attestation (dev_sev_gpu.erl, dev_sev_gpu):
  • Added support for generating and verifying NVIDIA SEV-SNP attestations.
  • Includes native Python scripts for interacting with the TEE environment.

🛠 Core Modifications

• HTTP Streaming (hb_http.erl):
  • Updated reply/5 to handle stream_generator, enabling real-time token streaming for LLM responses.
  • Added proper CORS and header handling for event streams.
• Configuration (hb_opts.erl):
  • Registered inference@1.0 and sev_gpu@1.0 devices.
  • Added default routing for /v1/.* to the local inference server.
  • Added inference_opts for model configuration (hash, name, size).
• Storage (hb_store_lmdb.erl):
  • Exposed max_readers configuration to optimize LMDB for high-concurrency read scenarios.

🧹 Maintenance

• Lifecycle Management: Updated hb_app:stop/1 to ensure the inference server is gracefully shut down.
• Build System: Updated rebar.config with new profiles and hooks for setting up the inference and GPU environments.

Testing

1. Run: Start the node with the inference profile: HB_PRINT=inference rebar3 as inference shell.
2. Verify:
   2.1 Check /health endpoint.

curl --request GET \
  --url 'http://localhost:8734/~inference@1.0/health'

2.2 Test a completion request to /v1/chat/completions (both streaming and non-streaming).

curl --request POST \
  --url 'http://localhost:8734/~inference@1.0/chat/completions' \
  --header 'content-type: application/json' \
  --data '{
  "model": "qwen/qwen2.5-0.5b-instruct",
  "messages": [
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ]
}
'

2.3 Verify TEE attestation if running on supported hardware.