Proxmox GPU Passthrough for AI Workloads: Step-by-Step Guide

Key Takeaways

• There are two ways to give a Proxmox guest a GPU: PCIe passthrough to one VM (exclusive, full driver control) or device sharing with LXC containers (several containers, one host driver).
• Passthrough needs IOMMU (Intel VT-d or AMD-Vi) enabled in firmware and on the kernel command line, the GPU in a clean IOMMU group, and the card bound to vfio-pci before any host driver grabs it.
• The VM should use the q35 machine type with OVMF (UEFI) firmware; inside it you install the normal NVIDIA driver and check with nvidia-smi.
• For video transcoding, photo libraries or camera detection, sharing an integrated GPU with a container is simpler and usually enough. For local LLMs, VRAM is what counts, so you want a discrete card.

Passthrough or sharing: pick the right one first

Most guides jump straight to vfio-pci. Before touching the kernel, decide what the GPU is for, because Proxmox offers two different mechanisms.

	PCIe passthrough to a VM	Device sharing with LXC
Who uses the GPU	One VM, exclusively	Any number of containers
Driver lives in	The VM	The Proxmox host
Host can still use the GPU	No	Yes
Setup effort	IOMMU, vfio, VM firmware	One dev line per container
Typical use	Ollama, CUDA, training, Windows gaming VM	Jellyfin, Immich, Frigate, light inference

If you want to run an LLM server with its own CUDA stack, or a guest OS that is not Linux, use passthrough. If you want several Linux services to use hardware video decode or OpenVINO, share the device with containers. Both are covered below.

What I run myself: sharing, not passthrough

My own Proxmox host is a small machine with an 11th-gen Intel Core i7 and its integrated Iris Xe graphics, no discrete card. IOMMU is off on it, because nothing there needs it: the iGPU is not passed through to a VM, it is shared with the container that runs my photo library, so video transcoding and the machine-learning jobs can run on the GPU instead of the CPU.

The whole change is one line in the container config:

# /etc/pve/lxc/<id>.conf
dev0: /dev/dri/renderD128,mode=0666

That box is the wrong place for serious local LLMs, though. An integrated GPU shares system memory and has nowhere near the bandwidth a model needs. If local inference is the goal, plan for a discrete NVIDIA card and the VM route that follows.

Prerequisites for PCIe passthrough

• CPU and motherboard with IOMMU: Intel VT-d or AMD-Vi (on AMD boards it is often labelled “IOMMU” or “SVM + IOMMU”).
• Firmware settings: enable VT-d / IOMMU, and Above 4G Decoding plus Resizable BAR if your board has them (large-memory cards need them).
• A GPU that is not the host’s only display: if the card you pass through is also the console, you lose local video output. On a headless server that is fine; manage it over the web UI and SSH.
• A recent Proxmox VE (8.x or 9.x) and a backup of the host config before you start.

Step 1: enable IOMMU on the kernel command line

Recent Proxmox kernels turn IOMMU on by default for Intel, and AMD has had it on for years, but make it explicit so you know where it is set. iommu=pt keeps devices that are not passed through on the fast path.

Find out which bootloader you use:

proxmox-boot-tool status

If it reports GRUB (the default on ext4/LVM installs), edit /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"   # Intel
# GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"               # AMD
update-grub

If it reports systemd-boot (common with ZFS on UEFI), the command line lives in /etc/kernel/cmdline as a single line:

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt
proxmox-boot-tool refresh

Reboot, then confirm:

dmesg | grep -e DMAR -e IOMMU | head
cat /proc/cmdline

You want to see lines like DMAR: IOMMU enabled (Intel) or AMD-Vi: ... enabled and your parameters in /proc/cmdline.

Step 2: load the vfio modules

Add these to /etc/modules so they load at boot:

vfio
vfio_iommu_type1
vfio_pci

On kernels 6.2 and newer vfio_virqfd is part of vfio, so older guides that list it can be ignored.

Step 3: check the IOMMU groups

Passthrough works per IOMMU group: everything in the group goes to the VM together. Print the groups:

for d in /sys/kernel/iommu_groups/*/devices/*; do
  g=${d#*/iommu_groups/}; g=${g%%/*}
  printf 'group %s  ' "$g"; lspci -nns "${d##*/}"
done | sort -V

A good result is a group that holds only the GPU and its audio function, for example:

group 14  01:00.0 VGA compatible controller [0300]: NVIDIA ... [10de:2684]
group 14  01:00.1 Audio device [0403]: NVIDIA ... [10de:22ba]

If the GPU shares a group with a SATA controller or a network card, try another PCIe slot first; the slots wired to the CPU usually have cleaner groups. The pcie_acs_override kernel option can split groups, but it weakens the isolation IOMMU exists to provide, so treat it as a last resort on a machine you trust.

Step 4: bind the GPU to vfio-pci

Note the vendor and device IDs of every function of the card (the numbers in square brackets above), then create /etc/modprobe.d/vfio.conf:

options vfio-pci ids=10de:2684,10de:22ba disable_vga=1
softdep nouveau pre: vfio-pci
softdep nvidia pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci

The softdep lines make sure vfio-pci claims the card before the host’s GPU or audio driver can. They are safer than blacklisting snd_hda_intel outright, which would also take away the host’s own audio.

Rebuild the initramfs and reboot:

update-initramfs -u -k all
reboot

Then check that the card is held by vfio:

lspci -nnk -s 01:00
# Kernel driver in use: vfio-pci   <- for both functions

Step 5: create the VM

Create a VM with these settings (web UI or CLI):

• Machine: q35
• BIOS: OVMF (UEFI), with an EFI disk
• CPU type: host
• Memory: fixed size. A VM with a passed-through device pins all of its RAM, so ballooning gives you nothing; set the balloon to 0.
• PCI device: Hardware → Add → PCI Device → Raw Device → your GPU, with All Functions and PCI-Express ticked. Tick Primary GPU only if the card should drive the VM’s display.

The same from the shell, for VM 120:

qm set 120 --machine q35 --bios ovmf --cpu host --balloon 0
qm set 120 --hostpci0 0000:01:00,pcie=1

Using 0000:01:00 without the function number passes all functions of the card at once.

Step 6: drivers and a first test inside the VM

Install your Linux distribution as usual (Ubuntu Server 24.04 LTS is an easy choice for CUDA). Then:

sudo apt update
sudo ubuntu-drivers install      # picks the recommended NVIDIA driver
sudo reboot
nvidia-smi                       # the card, driver and CUDA version should appear

For an LLM server, Ollama is the quickest test:

curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8b "Say hello in one sentence."
ollama ps                        # PROCESSOR should read 100% GPU

If you run workloads in Docker, add the NVIDIA Container Toolkit and check that a container sees the card:

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

As a rough guide for local models at 4-bit quantization: an 8B model fits in about 6 GB of VRAM, a 14B model in about 10 GB, and 70B models need 40 GB or more. Context length adds to that.

The container route in detail

Intel and AMD integrated GPUs. Since Proxmox VE 8.2 you can pass device nodes to a container without hand-written lxc.cgroup2 rules. From the host:

pct set 114 --dev0 /dev/dri/renderD128,mode=0666

mode=0666 is the simple option. Stricter is gid=<id>, using the id of the render group inside the container, so only members of that group can open the device. Inside the container, install the userspace drivers your app needs (intel-media-va-driver or mesa-va-drivers, plus OpenVINO or VA-API support in the app) and check with vainfo.

NVIDIA cards in containers. Install the NVIDIA driver on the host, then the same driver version inside the container without its kernel module, and pass the device nodes:

dev0: /dev/nvidia0
dev1: /dev/nvidiactl
dev2: /dev/nvidia-uvm
dev3: /dev/nvidia-uvm-tools

The versions must match exactly, so every host driver update means updating the containers too. In exchange, several containers can use the same card at the same time, which a VM passthrough cannot do.

Troubleshooting

• “No IOMMU detected” when adding the device: VT-d / IOMMU is off in firmware, or the kernel parameter did not apply. Check /proc/cmdline and that you edited the config for the bootloader you actually use.
• The host freezes when the VM starts: a host driver still owns the card. lspci -nnk must show vfio-pci for every function; recheck the IDs and the softdep lines, then rebuild the initramfs.
• “Failed to set iommu for container”: another device in the same IOMMU group is still bound to a host driver. Pass it too, or move the card to another slot.
• The VM boots but there is no picture: expected if the GPU is not marked Primary GPU and nothing is connected. Use the Proxmox console and nvidia-smi over SSH.
• Older AMD cards work once, then fail after a VM reboot: the Polaris and Vega reset bug. The vendor-reset kernel module fixes most of these cards.
• Windows guest shows Error 43: an old restriction in NVIDIA’s consumer drivers, lifted from driver 465 onward. Update the guest driver.

FAQ

What is Proxmox GPU passthrough?

It hands a physical PCIe GPU to one virtual machine. The VM loads the vendor driver and talks to the card directly, so CUDA, NVENC and the rest behave as they would on bare metal, while the Proxmox host gives up the card entirely.

Can several VMs share one GPU?

Not with plain passthrough: one card, one running VM. Splitting a GPU between VMs needs vGPU on supported datacenter cards or SR-IOV. For Linux services, sharing the GPU with LXC containers is the practical way to let several workloads use one card.

Do I need a discrete GPU for local AI?

For LLMs, yes in practice: they need VRAM and memory bandwidth that integrated GPUs do not have. For camera object detection, transcoding and photo libraries, an Intel iGPU shared with a container works well.

Why q35 and OVMF instead of the defaults?

q35 gives the VM a native PCI Express bus, which modern GPUs and their drivers expect, and OVMF provides UEFI, which current cards and large BAR sizes need. The older i440fx machine with SeaBIOS works for some cards but causes more problems than it is worth.

Does passthrough slow the GPU down?

In practice the difference from bare metal is small for compute workloads, because the VM drives the card directly. What matters more is giving the VM enough CPU cores and fixed memory so the GPU is not left waiting.

Recommended Gear

If you want to build this setup, this is where I would start:

• Crucial RAM 32GB DDR4: more headroom for VMs
• Samsung 870 EVO SSD 1TB: fast storage for VMs and containers

Affiliate links: if you buy through them I earn a small commission, at no extra cost to you.

Related Articles

• Proxmox Home Lab: A Practical Guide to Self-Hosting
• Proxmox LXC vs VM: Choosing the Right Virtualization
• Proxmox Ollama Setup: Self-Hosted AI Server for Developers