Prerequisites
- Python 3.12
- pip >= 21.0
Steps
- Create ray offline packages folder:
mkdir ray_offline_pkgs && cd ray_offline_pkgs - Download all dependencies (specify Python 3.12, manylinux2014_x86_64 platform, binary-only):
pip download --python-version 312 --platform manylinux2014_x86_64 --only-binary :all: ray[default]==2.40.0# 临时加镜像执行下载 pip download \ --index-url https://pypi.tuna.tsinghua.edu.cn/simple \ --retries 10 \ --timeout 120 \ --python-version 312 \ --platform manylinux2014_x86_64 \ --only-binary :all: \ ray[default]==2.40.0 - Offline install:
pip install --no-index --find-links=./ray_offline_pkgs ray[default]==2.40.0
Next Steps
After installing Ray, you can proceed to:
- Multi-Node LLM Serving: vLLM + Ray — Deploy vLLM across multiple nodes using Ray
- Multi-Node LLM Serving: Architecture, Frameworks & Best Practices — Compare multi-node serving frameworks