@@ -23,9 +23,20 @@ The easiest way to install Lightllm is using the official image. You can directl
2323 $ # Pull the official image
2424 $ docker pull ghcr.io/modeltc/lightllm:main
2525 $
26- $ # Run
26+ $ # Run,The current LightLLM service relies heavily on shared memory.
27+ $ # Before starting, please make sure that you have allocated enough shared memory
28+ $ # in your Docker settings; otherwise, the service may fail to start properly.
29+ $ #
30+ $ # 1. For text-only services, it is recommended to allocate more than 2GB of shared memory.
31+ $ # If your system has sufficient RAM, allocating 16GB or more is recommended.
32+ $ # 2.For multimodal services, it is recommended to allocate 16GB or more of shared memory.
33+ $ # You can adjust this value according to your specific requirements.
34+ $ #
35+ $ # If you do not have enough shared memory available, you can try lowering
36+ $ # the --running_max_req_size parameter when starting the service.
37+ $ # This will reduce the number of concurrent requests, but also decrease shared memory usage.
2738 $ docker run -it --gpus all -p 8080:8080 \
28- $ --shm-size 1g -v your_local_path:/data/ \
39+ $ --shm-size 2g -v your_local_path:/data/ \
2940 $ ghcr.io/modeltc/lightllm:main /bin/bash
3041
3142 You can also manually build the image from source and run it:
@@ -35,9 +46,9 @@ You can also manually build the image from source and run it:
3546 $ # Manually build the image
3647 $ docker build -t <image_name> .
3748 $
38- $ # Run
49+ $ # Run,
3950 $ docker run -it --gpus all -p 8080:8080 \
40- $ --shm-size 1g -v your_local_path:/data/ \
51+ $ --shm-size 2g -v your_local_path:/data/ \
4152 $ <image_name> /bin/bash
4253
4354 Or you can directly use the script to launch the image and run it with one click:
0 commit comments