Cloudflare AI Gateway adds User Insights for model overuse

Cloudflare adds Clef-omni multimodal model, speeds Clef and cuts Clef-flash pricing

Cloudflare expanded its Clef decision model family with Clef-omni, which natively processes audio, video, images, and text in a single pipeline. The company also increased Clef inference speeds by up to 2.0x and lowered pricing for Clef-flash.

Ai2 replaces priority GPU scheduler with budget-based fair-share system

Ai2's AI Infrastructure team replaced its priority-based GPU scheduler with a system using GPU time budgets, hierarchical fair-share allocation, and time-slicing contracts. The institute manages thousands of NVIDIA H100, B200, and B300 GPUs across 88- to 1024-GPU clusters for about 150 researchers, with demand running 2-3x available capacity. The change moved GPU allocation debates into a transparent administrative budgeting process.

Ai2 details GPU scheduler using time budgets and fair-share allocation

Ai2 published a blog post explaining its new GPU scheduler for its clusters. The scheduler combines time budgets, fair-share allocation, and time-slicing to prioritize high-impact research, shorten queue waits, and keep GPUs busy.

PyTorch details session-aware agentic inference with NVIDIA Dynamo

PyTorch published a blog post describing session-aware agentic inference with NVIDIA Dynamo. It explains that agentic workloads differ from single-turn chat: an agent session can include a large initial prefill, repeated model calls, and parallel subagents, changing the traffic an inference server handles.

AWS details multi-team GPU sharing on SageMaker HyperPod

AWS published a reference architecture for sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams. It uses AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fairness, and namespace-level cost allocation for chargeback.