返回全部动态
Vortex：从 S3 到 GPU 的一次拷贝数据加载革新
原标题：Presentation: From S3 to GPU in One Copy: Rethinking Data Loading for ML Training
AI 摘要
SpiralDB 的 Onur Satici 在演讲中介绍了其开源的列式文件格式 Vortex，该格式旨在优化从 S3 到 GPU 的数据加载流程，通过消除中间磁盘存储和 CPU 解压瓶颈，实现高达 13 Gbps 的流式传输。Vortex 支持在 GPU 上直接解压和列裁剪，从而加速机器学习训练的数据迭代。
以上摘要由 AI 生成，可能存在误差。事实请以原文为准。
正文节选
Transcript Onur Satici: Before I get into anything, I want to show you this video. Not specifically like what's playing in the video, but how we produce this video. Because this is not backed by a regular MP4 file in some main disk, but this is actually a visualization of a file scan. What I mean by this is we have a Vortex file, Vortex being a columnar file format. It has three columns, one per color. Each column would then have a row per frame of this video. Each frame would include around 8 m
发布时间：2026-09-04 19:00
抓取时间：2026-09-04 19:06
来源机构：InfoQ