[Executorch] Introduce caching cpu memory allocator

[kimishpatel]

Stack from ghstack (oldest at bottom):

• [Executorch][LLM] Use caching allocator for runner #15730
• [Executorch] Make module constructors uniform across #15729
• [Executorch] Use temp allocator for allocating scratch memory #15728
• -> [Executorch] Introduce caching cpu memory allocator #15611
• Reduce allocation overhead in quantized sdpa #15610

Meant to use this for temp allocator for kernels. Specifically for sdpa, it seems that on iOS there is a significant overhead coming from allocations

Differential Revision: D85532079

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15611

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 3 Unrelated Failures

As of commit 6b37360 with merge base a09a4b7 ():

NEW FAILURE - The following job has failed:

• trunk / test-mcu-cortex-m-backend / linux-job (gh)
  RuntimeError: Command docker exec -t 4f83ea84bb631196a18f6c4a4e7f31784cd515a40317378944dbcb29df077a4a /exec failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-weight-only) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, non-quantized) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / android / run-emulator (gh) (trunk failure)
  Timeout waiting for emulator to boot.

This comment was automatically generated by Dr. CI and updates every 15 minutes.