Turkey (Turkish)
Germany (German)
Worldwide (English)
Current bitsandbytes supports NVIDIA CUDA 11.8-13.0, plus Intel XPU, Gaudi and CPU backends.
Halkali merkez neighborhood fatih st ozgur apt no 46 , Kucukcekmece , Istanbul , 34303 , TR
Fix CUDA detection failed, missing libcudart.so, missing libbitsandbytes_cudaXXX.so and CUDA/PyTorch mismatch errors on Windows, WSL and Linux.
Current bitsandbytes supports NVIDIA CUDA 11.8-13.0, plus Intel XPU, Gaudi and CPU backends.
The CUDA runtime reported by PyTorch does not have to match the separately installed CUDA toolkit; inspect `torch.version.cuda` together with bitsandbytes diagnostics.
Old GitHub errors may mention Linux-specific `libcudart.so` or source-build instructions. Modern Windows wheels exist, so do not blindly follow outdated fixes.
Record driver, GPU, PyTorch CUDA and bitsandbytes versions together. Many failures are not a single 'CUDA version' problem; package runtime, toolkit paths and driver support can differ.
nvidia-smipython - <<'PY'
import torch, bitsandbytes as bnb
print('torch', torch.__version__)
print('torch cuda', torch.version.cuda)
print('bnb', bnb.__version__)
print('gpu', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NO CUDA')
PYpython -m bitsandbytesThis message is an umbrella for missing drivers, missing runtimes, incompatible binaries, conflicting CUDA paths or unsupported GPU capability. Read the detected runtime path and binary name before reinstalling toolkits.
On Linux, the XXX in the `.so` name reflects the CUDA variant bitsandbytes is trying to load. If that binary is absent, inspect package version, CUDA overrides and wheel compatibility before compiling from source.
`libcudart.so` is a Linux runtime library. Searching for it or setting `LD_LIBRARY_PATH` on native Windows is the wrong fix. Modern Windows support should use a compatible wheel, NVIDIA driver and PyTorch CUDA build.
This override is for advanced cases where bitsandbytes should use a CUDA binary different from PyTorch's runtime. A wrong value can break a working environment. Test clean autodetection first.
WSL2 uses Linux user space while the NVIDIA driver lives on the Windows host. Do not install a second full Windows driver inside WSL. Install Linux PyTorch/bitsandbytes packages in WSL while the host driver provides GPU access.
bitsandbytes features have different minimum GPU architecture requirements. The official installation matrix lists separate compute-capability requirements for 8-bit optimizers and NF4/FP4. Import success does not guarantee every feature works on old GPUs.
Mixed conda/pip installs and stale CUDA paths make diagnosis difficult. Create a fresh venv with only PyTorch, Transformers, Accelerate and bitsandbytes. If that works, the old environment likely has dependency/path conflicts.
Source compilation is appropriate when official wheels do not cover your CUDA/architecture combination. On supported CUDA 11.8-13.0 and modern GPUs, jumping straight to source builds adds unnecessary complexity.
| Message | First check | Wrong reflex |
|---|---|---|
| CUDA detection failed | torch.version.cuda + diagnostics | Random CUDA reinstall |
| Missing libbitsandbytes_cudaXXX.so | Wheel/bnb version | Copy old build command |
| libcudart.so missing | Linux runtime path | Use LD_LIBRARY_PATH on Windows |
| No kernel image | GPU compute capability | Only change toolkit |
Do not blindly upgrade packages in a working training environment. Record GPU, driver, CUDA/PyTorch runtime, Transformers, Accelerate, PEFT, TRL, bitsandbytes/Diffusers, model revision and dataset fingerprint for every run. Reproduce minimally before changing production training.
Yes. Current bitsandbytes provides Windows support; older 'Linux only' advice is outdated.
PyTorch wheels can ship with their own CUDA runtime; system toolkit and PyTorch runtime are different layers.
Evaluate model size, precision, context, batch, LoRA/QLoRA or full fine-tuning and multi-GPU needs together instead of choosing by GPU name alone.