语音伴随手势生成的运动分词器是否编码手势语义？
原标题：Do Motion Tokenizers for Co-Speech Gesture Generation Encode Gesture Semantics?
AI 摘要
该研究探究了用于语音伴随手势生成的离散运动分词器是否编码了手势语义。作者使用19个从原始运动到抽象交际功能的手势描述符，对重建训练得到的码本进行探测，发现几何和左右手属性可从token嵌入中轻易解码，而手势类别、手形和交际意图仅能弱解码。这表明仅靠重建目标不能保证捕获手势语义，评估码本在这些属性上的表现可指导更具语义的运动分词器设计。
正文节选
Do Motion Tokenizers for Co-Speech Gesture Generation Encode Gesture Semantics? Abstract Discrete motion tokenizers encode motion as atomic units and are widely used for co-speech gesture generation. It remains unclear which motion properties, especially those relevant to gesture semantics, are recoverable from these codebooks. We probe a reconstruction-trained codebook using 19 co-speech gesture descriptors spanning from raw motion to abstract communicative function. Results show that geometry