Ray 2.58.0 发布：KV 缓存路由、Delta Lake 集成与 TPU 支持

AI 摘要

Ray 2.58.0 发布，重点包括 Ray Serve LLM 完成 KV 缓存和 token 感知路由，Ray Core 支持任务事件卸载，Ray Data 新增 Databricks Delta Lake 集成和 shuffle v2 后端，并推出实验性 Ray Sandbox 和 TPU 支持。这些改进提升了推理性能、数据处理效率和资源调度能力。

正文节选

# Highlights * **Ray Serve LLM:** In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the `LLMRouter` ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress replica (\#64642, \#64920, \#64949, \#65010, \#65095). KV cache and token aware routing is also aware of CPU KV caches, so offload

发布时间：2026-08-23 13:42

抓取时间：2026-08-23 15:43

来源机构：Anyscale