AWS details bi-directional KV transfer in vLLM to cut TTFT for multi-turn agents
AWS published a blog post on bi-directional KV cache transfer in vLLM for multi-turn agentic workloads. The approach lets prefill and decode nodes reuse previously computed context across turns instead of reprocessing growing conversation history. AWS reports time-to-first-token improvements of up to 1.6x for Qwen3-32B and 3.3x for GLM-5-FP8 on Amazon EC2 P5en instances with EFA.
Sources and evidence
Summary last validated Oct 19, 2026
Reader actions
Report an issue
Use this for an incorrect summary, wrong source, duplicate story, or wrong category. Submissions are private and do not change the story automatically.
Related coverage
- infrastructure
Barcelona Supercomputing Center Opens AI Factory Accelerator to Spanish Startups
The Barcelona Supercomputing Center's AI Factory Accelerator Programme is now in full operation after a pilot phase, giving startups access to sovereign compute, technical support and training across industries. Many participating companies are also members of the NVIDIA Inception program for startups, and as of today some gain expanded access.
- models
Amazon Bedrock adds structured outputs for Anthropic Claude models
Amazon Bedrock now supports structured outputs for all Anthropic Claude models on both the bedrock-runtime and bedrock-mantle endpoints. The feature returns schema-compliant JSON matching a defined structure using JSON Schema output format and strict tool use, eliminating the need for custom parsing, retry logic, or post-processing.
- infrastructure
AWS opens SageMaker Studio to new users via simplified sign-up
AWS now lets new Amazon SageMaker AI customers reach a fully configured SageMaker Studio environment in a few steps. Users can sign up with an existing Amazon, Apple, GitHub, or Google identity, with no advanced AWS account or credit card required for most customers, plus $100 in AWS Free Tier credits.
- infrastructure
SageMaker JumpStart adds over 100 models from 30 providers in 2026
Amazon SageMaker JumpStart onboarded more than 100 new models from 30 providers in the first three quarters of 2026, AWS said in a year-to-date review. The recap also covers JumpStart enhancements including a simplified Model Hub, pre-configured deployment options, and Hugging Face deep link integration.
- infrastructure
Cloudflare cuts AI code reviewer inference bill 68% with prompt caching and model routing
Cloudflare said its AI code reviewer's cost per run doubled, prompting an investigation into token usage. The company used prompt caching and intelligent model routing to cut its inference bill by 68% while speeding up reviews, according to a Cloudflare blog post.