NVIDIA TensorRT Adds Multi-Device Inference Across Multiple GPUs in Dynamo-Triton
NVIDIA introduced TensorRT multi-device inference, a capability letting a single TensorRT network run across multiple GPUs using NCCL-backed distributed collectives while keeping TensorRT optimizations. The feature is fully supported starting with TensorRT 11.0 and is integrated into NVIDIA Dynamo-Triton for model serving, addressing generative AI workloads whose compute and memory demands exceed one GPU.
Sources and evidence
Summary last validated Sep 28, 2026
Reader actions
Report an issue
Use this for an incorrect summary, wrong source, duplicate story, or wrong category. Submissions are private and do not change the story automatically.
Related coverage
- agents
NVIDIA Highlights Developers Using Frontier AI Agents With Omniverse Libraries
NVIDIA published a blog post describing how developers combine frontier AI models with NVIDIA Omniverse libraries to build simulation applications. The post says developers direct AI agents to assemble assets, connect physics and rendering, and verify scene behavior, supporting work such as exploring scenarios, investigating failures and improving designs.
- open source
NVIDIA Details Five-Step Workflow for SimReady Robotics Assets
NVIDIA published a five-step workflow for converting CAD assets into SimReady robotics simulation assets using Omniverse libraries, SimReady Foundation specifications, and agentic NVIDIA skills. The process covers configuring and validating materials, collision geometry, joints, and physics properties beyond simple OpenUSD geometry conversion, preparing assets before robot behavior testing.
- agents
NVIDIA KGMON Team Places Second in KDD Cup 2026 Data Agents Competition
NVIDIA's KGMON team placed second in the KDD Cup 2026 Data Agents competition. The team built a system around making an agent's harness smaller, clearer, and easier to verify. The competition required agents to answer natural-language questions across heterogeneous sources including databases, CSV and JSON files, prose documents, PDFs, and briefing videos.
- infrastructure
PyTorch details session-aware agentic inference with NVIDIA Dynamo
PyTorch published a blog post describing session-aware agentic inference with NVIDIA Dynamo. It explains that agentic workloads differ from single-turn chat: an agent session can include a large initial prefill, repeated model calls, and parallel subagents, changing the traffic an inference server handles.
- infrastructure
AWS details multi-team GPU sharing on SageMaker HyperPod
AWS published a reference architecture for sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams. It uses AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fairness, and namespace-level cost allocation for chargeback.