幂律图注意力：缩放点积注意力的精确推广与推理坍缩
原标题：Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
AI 摘要
Hugging Face 每日论文发布了一篇关于新型注意力机制的研究，提出 Power Law Graph Attention (PLGA)，用学习到的幂律双线性算子替代固定缩放点积注意力 (SDPA)，并构建了 PLDR-LLM 模型。论文通过定理和测量验证了 PLGA 在特定条件下包含 SDPA，并发现推理时存在输入不变性导致推理坍缩的现象。该研究还提供了 Lean 4 机器检查的证明核心，并在 TruthfulQA 指标上验证了块与顺序评分的一致性。
正文节选
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference Abstract A new attention mechanism replaces fixed scaled dot-product attention with a learned power-law bilinear operator, with verified architecture, measured stability, and machine-checked proofs. The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attent