Skip to content

Commit 5b00d91

Browse files
luffy-yuCopilot
andauthored
Add Run Android LlamaDemo with QNN backend (#16011)
### Motivation - The current Android LlamaDemo only supports XNNPACK. - The QNN-backend Android Demo is missing, the `qnn_llama_runner` approach is provided though. ### Summary - Add Guidance for building and running Android LlamaDemo with QNN-Backend - It's verified on Samsung S23 (SoC SM8550). ### Test plan - This PR has been tested on [LlamaDemo-Executorch-QNN](https://github.com/luffy-yu/LlamaDemo-Executorch-QNN). > The APK file and the pre-built model can be found in its README. - This PR primarily borrows doc from [QNN_ANDROID_FIX_SUMMARY](https://github.com/luffy-yu/LlamaDemo-Executorch-QNN/blob/master/QNN_ANDROID_FIX_SUMMARY.md). --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent c9bf066 commit 5b00d91

File tree

1 file changed

+139
-0
lines changed

1 file changed

+139
-0
lines changed

docs/source/backends-qualcomm.md

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,145 @@ After the above command, pre-processed inputs and outputs are put in `$EXECUTORC
294294
The command-line arguments are written in [utils.py](https://github.com/pytorch/executorch/blob/main/examples/qualcomm/utils.py#L139).
295295
The model, inputs, and output location are passed to `qnn_executorch_runner` by `--model_path`, `--input_list_path`, and `--output_folder_path`.
296296

297+
### Run [Android LlamaDemo](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo) with QNN backend
298+
299+
`$DEMO_APP` refers to the root of the executorch android demo, i.e., the directory containing `build.gradle.kts`.
300+
301+
***Step 1***: Rebuild ExecuTorch AAR
302+
303+
```bash
304+
# Build the AAR
305+
cd $EXECUTORCH_ROOT
306+
export BUILD_AAR_DIR=$EXECUTORCH_ROOT/aar-out
307+
./scripts/build_android_library.sh
308+
```
309+
310+
***Step 2***: Copy AAR to Android Project
311+
312+
```bash
313+
cp $EXECUTORCH_ROOT/aar-out/executorch.aar \
314+
$DEMO_APP/app/libs/executorch.aar
315+
```
316+
317+
***Step 3***: Build Android APK
318+
319+
```bash
320+
cd $DEMO_APP
321+
./gradlew clean assembleDebug -PuseLocalAar=true
322+
```
323+
324+
***Step 4***: Install on Device
325+
326+
```bash
327+
adb install -r app/build/outputs/apk/debug/app-debug.apk
328+
```
329+
330+
***Step 5***: Push model
331+
332+
```bash
333+
adb shell mkdir -p /data/local/tmp/llama
334+
adb push model.pte /data/local/tmp/llama
335+
adb push tokenizer.bin /data/local/tmp/llama
336+
```
337+
338+
***Step 6***: Run the Llama Demo
339+
340+
- Open the App on Android
341+
- Select `QUALCOMM` backend
342+
- Select `model.pte` Model
343+
- Select `tokenizer.bin` Tokenizer
344+
- Select Model Type
345+
- Click LOAD MODEL
346+
- It should show `Successfully loaded model.`
347+
348+
349+
#### Verification Steps
350+
351+
***Step 1***. Verify AAR Contains Your Changes
352+
353+
```bash
354+
# Check for debug strings in the AAR
355+
unzip -p $DEMO_APP/app/libs/executorch.aar jni/arm64-v8a/libexecutorch.so | \
356+
strings | grep "QNN" # Replace "QNN" with your actual debug string if needed
357+
```
358+
359+
If found, your changes are in the AAR.
360+
361+
***Step 2***. Verify APK Contains Correct Libraries
362+
363+
```bash
364+
# Check QNN library version in APK
365+
cd $DEMO_APP
366+
unzip -l app/build/outputs/apk/debug/app-debug.apk | grep "libQnnHtp.so"
367+
```
368+
369+
Expected size for QNN 2.37.0: ~2,465,440 bytes
370+
371+
***Step 3***. Monitor Logs During Model Loading
372+
373+
```bash
374+
adb logcat -c
375+
adb logcat | grep -E "ExecuTorch"
376+
```
377+
378+
#### Common Issues and Solutions
379+
380+
##### Issue 1: Error 18 (InvalidArgument)
381+
382+
- **Cause**: Wrong parameter order in Runner constructor or missing QNN config
383+
384+
- **Solution**: Check `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/llama/runner/runner.h` for the correct constructor signature.
385+
386+
##### Issue 2: Error 1 (Internal) with QNN API Version Mismatch
387+
388+
- **Symptoms**:
389+
390+
```
391+
W [Qnn ExecuTorch]: Qnn API version 2.33.0 is mismatched
392+
E [Qnn ExecuTorch]: Using newer context binary on old SDK
393+
E [Qnn ExecuTorch]: Can't create context from binary. Error 5000
394+
```
395+
396+
- **Cause**: Model compiled with QNN SDK version X but APK uses QNN runtime version Y
397+
398+
- **Solution**:
399+
- Update `build.gradle.kts` with matching QNN runtime version
400+
401+
> **Note:** The version numbers below (`2.33.0` and `2.37.0`) are examples only. Please check for the latest compatible QNN runtime version or match your QNN SDK version to avoid API mismatches.
402+
403+
**Before**:
404+
```kotlin
405+
implementation("com.qualcomm.qti:qnn-runtime:2.33.0")
406+
```
407+
408+
**After**:
409+
```kotlin
410+
implementation("com.qualcomm.qti:qnn-runtime:2.37.0")
411+
```
412+
413+
- Or recompile model with matching QNN SDK version
414+
415+
##### Issue 3: Native Code Changes Not Applied
416+
417+
- **Symptoms**:
418+
- Debug logs don't appear
419+
- Behavior doesn't change
420+
421+
- **Cause**:
422+
- Gradle using Maven dependency instead of local AAR
423+
424+
- **Solution**:
425+
- Always build with `-PuseLocalAar=true` flag
426+
427+
##### Issue 4: Logs Not Appearing
428+
429+
- **Cause**: Wrong logging tag filter
430+
431+
- **Solution**: QNN uses "ExecuTorch" tag:
432+
433+
```bash
434+
adb logcat | grep "ExecuTorch"
435+
```
297436
298437
## Supported model list
299438

0 commit comments

Comments
 (0)