« KVarN » : différence entre les versions
De Le Wiki de Lug
Autres actions
| Ligne 10 : | Ligne 10 : | ||
# source .venv/bin/activate | # source .venv/bin/activate | ||
# python -m pip install -U pip setuptools wheel | # python -m pip install -U pip setuptools wheel | ||
# export CUDA_HOME=/usr/local/cuda-<font color=blue>13.3</font> | |||
# export PATH=$CUDA_HOME/bin:$PATH | |||
# VLLM_USE_PRECOMPILED=1 pip install -e . | # VLLM_USE_PRECOMPILED=1 pip install -e . | ||
Version du 26 juillet 2026 à 10:24
Installation
# apt update && apt upgrade # apt install python3-pip python3-venv python3-full # cd /opt/ # git clone https://github.com/huawei-csl/KVarN.git # cd KVarN # python3 -m venv .venv # source .venv/bin/activate # python -m pip install -U pip setuptools wheel # export CUDA_HOME=/usr/local/cuda-13.3 # export PATH=$CUDA_HOME/bin:$PATH # VLLM_USE_PRECOMPILED=1 pip install -e .
Tests en vrac
# hf download shawnw3i/Qwen3.6-27B-AWQ-MTP --local-dir /opt/models/Qwen3.6-27B-AWQ-MTP