The InfoQ Podcast

← The InfoQ Podcast9. Nov. · 44 Min.

Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM

Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM9. Nov.44 Min.

In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI inference using llm-d and vLLM on Kubernetes. Edoardo explains how llm-d manages cache locality (KV and prefix caches), request routing, and batch inference across Kubernetes clusters to maximize expensive GPU utilization.

Read a transcript of this interview: https://bit.ly/4eLRHnE

Newsletter:

Subscribe to the Software Architects' Newsletter, a monthly roundup of the patterns and technologies senior practitioners are working through, with the news and lessons from people doing the work: https://www.infoq.com/software-architects-newsletter

InfoQ Online Certification Programs:

InfoQ’s online cohorts put you in a small group of senior engineers and architects from other companies who face the same decisions. You bring a real decision from your job and work through it using approaches from QCon talks. You write up what you learned as an article, which makes you put the trade-offs into words before your next proposal at work. You leave with new approaches, or confirmation that the calls you’re already making are the right ones. See dates:

https://certification.qconferences.com/

Upcoming Events:

QCon San Francisco 2026 (November 16-20, 2026)

https://qconsf.com/

QCon AI New York 2026 - AI Engineering Conference (December 15-16, 2026)

https://newyork.qcon.ai/