MetaRoCE: New RDMA Transport for AI-Scale Ethernet

Ai2 replaces priority GPU scheduler with budget-based fair-share system

Ai2's AI Infrastructure team replaced its priority-based GPU scheduler with a system using GPU time budgets, hierarchical fair-share allocation, and time-slicing contracts. The institute manages thousands of NVIDIA H100, B200, and B300 GPUs across 88- to 1024-GPU clusters for about 150 researchers, with demand running 2-3x available capacity. The change moved GPU allocation debates into a transparent administrative budgeting process.

Ai2 details GPU scheduler using time budgets and fair-share allocation

Ai2 published a blog post explaining its new GPU scheduler for its clusters. The scheduler combines time budgets, fair-share allocation, and time-slicing to prioritize high-impact research, shorten queue waits, and keep GPUs busy.

PyTorch details session-aware agentic inference with NVIDIA Dynamo

PyTorch published a blog post describing session-aware agentic inference with NVIDIA Dynamo. It explains that agentic workloads differ from single-turn chat: an agent session can include a large initial prefill, repeated model calls, and parallel subagents, changing the traffic an inference server handles.

AWS details multi-team GPU sharing on SageMaker HyperPod

AWS published a reference architecture for sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams. It uses AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fairness, and namespace-level cost allocation for chargeback.

MIT's Christina Delimitrou targets data center energy efficiency

MIT Associate Professor Christina Delimitrou is rethinking how large cloud computing systems operate to reduce the environmental impact of data centers. Her work focuses on making these facilities more energy efficient as their environmental threat grows.