Tenstorrent releases the vLLM TT plugin
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Released the vLLM TT plugin. Using vLLM's out-of-tree platform plugin mechanism, it automatically registers Tenstorrent accelerators when the ttnn library is present, keeping the OpenAI-compatible API unchanged. Supports Llama 3.1–3.3, Qwen 2.5–3.6, Mistral, Gemma 3–4, DeepSeek V3 and multimodal variants, but the model implementations live in TT-Metal rather than in the plugin. The main differences from GPU serving are stage-based scheduling split into prefill-only and decode-only, single-process lane data parallelism for Galaxy systems, on-device sampling and asynchronous readback; speculative decoding, LoRA, prompt logprobs and multi-host serving are not supported. No performance figures are given.
Why it matters
Shows a path by which a non-mainstream accelerator joins the mainstream inference stack through a plugin that leaves the vLLM core untouched — the barrier to hardware diversification is coming down.
Source — Primary sources first; secondary reports are labelled “Press”
Officialvllm.aihttps://vllm.ai/blog/2026-09-07-vllm-tt-pluginAdded in Week 38, 2026 · See that week
On the map — ⑧ Inference Infrastructure line
- Previous stopGoogle Cloud previews its 8th-generation TPUs at Cloud Next
- This recordvLLM TT plugin
- Next stopDeepSeek-V4.1-Flash