在 Amazon SageMaker AI 上用 llama.cpp 部署决策模型
原标题：Deploy decision models with llama.cpp on Amazon SageMaker AI
AI 摘要
AWS 在 Machine Learning Blog 发布教程，介绍如何用 llama.cpp 的 AWS Deep Learning Container 在 Amazon SageMaker AI 上部署决策模型。方案包含两条路径：标准路径通过 OpenAI 兼容的 chat completions 接口提供文本生成，结构化决策路径则构建子镜像并接入 SemIf（原 OpenJev）适配器，直接返回声明选项的概率而非生成答案 token。基准测试显示，Qwen3.5-4B Q4_K_M 在 GPU 端点上的 p50 延迟为 0.352 秒，比 CPU 配置快约 4.9 倍，准确率约 77.9%。
正文节选
Deploy decision models with llama.cpp on Amazon SageMaker AI Applications that route requests, select agent actions, or assign policy outcomes often need scores over a fixed set of choices instead of free-form text. Deploying these decision models with llama.cpp on Amazon SageMaker AI gives you a managed path for serving quantized models on CPU or GPU infrastructure. This post continues a series on specialized AWS Deep Learning Containers (DLCs) for generative AI inference. Previous posts cover