ACM Transactions on Graphics · SIGGRAPH Asia 2026 Journal Track ACM Transactions on Graphics · SIGGRAPH Asia 2026 期刊轨
doi:10.1145/3842525 ↗Gauge-Equivariant Attention for Rotation-Stable 360° Scene Understanding Gauge-Equivariant Attention for Rotation-Stable 360° Scene Understanding规范等变注意力：实现旋转稳定的 360° 场景理解
1 National University of Defense Technology (NUDT) · 2 Northwestern Polytechnical University (NPU) 1 国防科技大学（NUDT） · 2 西北工业大学（NPU）
* Corresponding author * 通讯作者
Spherical attention should not care where the camera points. 球面注意力本不该在意相机指向何处。
Panoramic 360° scene understanding increasingly relies on icosphere transformers, but a state-of-the-art spherical model loses more than half of its segmentation accuracy when the camera rotates by 90°, and controlled ablations identify gauge dependence in its relative-position bias as a major contributor.
全景 360° 场景理解正日益依赖二十面体球（icosphere）Transformer，但当前最先进的球面模型在相机旋转 90° 时会损失一半以上的分割精度；受控消融实验表明，其相对位置偏置中的规范依赖（gauge dependence）是造成这一现象的主要原因。
We propose gauge-equivariant relative position encoding (GE-RPE): a parameter-free Reynolds average of the bias over a finite cyclic subgroup Cn ⊂ SO(2) of gauge rotations. Plugged into a SphereUFormer backbone the change is invisible at deployment—zero added parameters and only 1.5–4.4% forward latency—and the matched three-seed GE-RPE model records a 1.3% drop (compared to 53% drop in the published checkpoint).
我们提出规范等变相对位置编码（gauge-equivariant relative position encoding，GE-RPE）：在规范旋转的有限循环子群 Cn ⊂ SO(2) 上对偏置做无参数的 Reynolds 平均。将其接入 SphereUFormer 骨干后，部署时几乎察觉不到变化——不增加任何参数，前向延迟仅增加 1.5–4.4%——而匹配三随机种子的 GE-RPE 模型仅出现 1.3% 的性能下降（已发表的检查点为 53%）。
Once the gauge defect is removed and a teacher-token permutation πR aligns the SSL views to the rotated student frame, iBOT+MAE pretraining stops being a liability and becomes a clean low-label lever: the full framework EquiSSL (GE-RPE + πR + iBOT+MAE) tightens the drop to 0.8% at 68.30% val mIoU and lifts 1%-label fine-tuning by +2.39 mIoU on the N=373 test split (and by +4.10 on the smaller N=40 val split); the same fix carries over to monocular depth and to zero-shot Structured3D segmentation. The construction is provably Cn-invariant and O(n-2)-close to the continuous SO(2) average, making the resulting model a usable 360° visual-computing primitive across panoramic relighting, immersive video, and cross-dataset transfer.
一旦消除规范缺陷，并由教师 token 置换 πR 将自监督学习的各视图对齐到旋转后的学生坐标系，iBOT+MAE 预训练便不再是负担，而成为提升低标签性能的有效手段：完整框架 EquiSSL（GE-RPE + πR + iBOT+MAE）将性能下降进一步收窄至 0.8%，验证集 mIoU 达 68.30%，并在 N=373 测试划分上使 1% 标签微调提升 +2.39 mIoU（在更小的 N=40 验证划分上提升 +4.10）；同样的修正还可迁移到单目深度估计与零样本 Structured3D 分割。该构造可证明具有 Cn 不变性，且与连续 SO(2) 平均相差 O(n-2)，使所得模型成为一种可用的 360° 视觉计算基元，可服务于全景重光照、沉浸式视频与跨数据集迁移。
One scalar carries the whole defect. 缺陷藏在一个标量里。
When projecting a 360° panoramic scene onto a discretized icosphere mesh, each vertex requires a local tangent coordinate system (gauge) to define relative spatial positions. Existing spherical transformers choose local frame orientations arbitrarily. When the camera rotates, the local relative angles shift inconsistently, corrupting the attention weights.
将 360° 全景场景投影到离散的二十面体球网格时，每个顶点都需要一个局部切坐标系（即规范，gauge）来定义相对空间位置。现有球面 Transformer 对局部坐标系的朝向作任意选择；当相机旋转时，各顶点的局部相对角度发生不一致的偏移，注意力权重因此被破坏。
The failure is architectural, not a data or capacity problem: the relative-position bias is read in a per-vertex frame, so the same scene observed under a different global rotation yields a different attention bias.
这一失效是架构层面的，而非数据或容量问题：相对位置偏置在逐顶点的局部坐标系中读取，因此同一场景在不同全局旋转下会得到不同的注意力偏置。
Key mathematical insight. True spherical equivariance requires attention weights to be invariant under local gauge transformations SO(2). By computing a finite cyclic group Reynolds average (Cn ⊂ SO(2)), GE-RPE eliminates gauge ambiguity without requiring extra model parameters or retraining backbones from scratch.
核心数学洞察。真正的球面等变性要求注意力权重在局部规范变换 SO(2) 下保持不变。GE-RPE 通过在有限循环群上计算 Reynolds 平均（Cn ⊂ SO(2)），在不引入额外模型参数、也无需从头重训练骨干的前提下消除规范歧义。
Remove it, then exploit it. 先消除，再利用。
EquiSSL approaches the defect from two sides. GE-RPE fixes the architecture: the relative-position bias becomes a Reynolds average over a finite cyclic subgroup of gauge rotations, so it no longer depends on how each vertex's frame was chosen.
EquiSSL 从两个方向处理这一缺陷。GE-RPE 修正架构：把相对位置偏置改为规范旋转有限循环子群上的 Reynolds 平均，使其不再依赖每个顶点坐标系的选取方式。
πR fixes the pretraining: a teacher-token permutation aligns self-supervised views under global rotations, turning iBOT + MAE into a rotation-consistent feature booster. Neither change adds parameters; the deployed backbone keeps its size, and the forward-pass overhead is 1.5–4.4%.
πR 修正预训练：教师 token 置换在全局旋转下对齐自监督视图，使 iBOT + MAE 成为旋转一致的特征增强手段。两项改动都不增加参数；部署时骨干规模不变，前向开销为 1.5–4.4%。
GE-RPE: Reynolds averagingGE-RPE：Reynolds 平均
Averages the relative-position bias across cyclic Cn rotations. Provably Cn-invariant, with O(n-2) convergence to the continuous SO(2) integral.
在循环子群 Cn 的旋转上对相对位置偏置取平均；可证明具有 Cn 不变性，并以 O(n-2) 的收敛速度逼近连续 SO(2) 积分。
Teacher permutation πR教师置换 πR
In multi-crop SSL (iBOT + MAE), rotating the student view used to misalign its tokens with the teacher. The exact icosphere permutation πR removes that mismatch and turns SSL into a rotation-robust feature booster.
在多裁剪自监督学习（iBOT + MAE）中，旋转学生视图曾使其 token 与教师网络错位；二十面体球上的精确置换 πR 消除了这种错配，使自监督学习成为提升旋转鲁棒性的特征增强手段。
Stable at 90°. Efficient at 1%. 90° 处稳定，1% 标签下高效。
Benchmarks follow the paper: the 0° and 90° columns report Stanford2D3D val mIoU, and the 1%-label column reports fine-tuning mIoU on the N=373 test split.
基准结果与论文一致：0° 与 90° 两列为 Stanford2D3D 验证集 mIoU，1% 标签列为 N=373 测试划分上的微调 mIoU。
| Method方法 | Backbone骨干 | 0° mIoU (%)0° mIoU (%) | 90° mIoU (%)90° mIoU (%) | Rotation drop (Δ)旋转性能下降（Δ） | 1% label mIoU1% 标签 mIoU |
|---|---|---|---|---|---|
| SphereUFormer (published)SphereUFormer（已发表） | TransformerTransformer | 67.50 | 31.72 | -35.78 (-53.0%) | 41.20 |
| SphereUFormer (matched 3-seed)SphereUFormer（匹配三随机种子） | TransformerTransformer | 66.82 | 65.95 | -0.87 (-1.3%) | 42.50 |
| EquiSSL (full framework)EquiSSL（完整框架） | Transformer + GE-RPETransformer + GE-RPE | 68.30 | 67.75 | -0.55 (-0.8%) | 44.89 (+2.39) |
EquiSSL's 1%-label gain of +2.39 mIoU is measured against the matched three-seed baseline. EquiSSL 在 1% 标签下 +2.39 mIoU 的提升相对于匹配三随机种子的基线测得。
Scroll the table horizontally to read all columns → 左右滑动表格以查看全部列 →
Beyond segmentation 分割之外
The same correction carries over to other tasks: zero-shot Structured3D accuracy improves by +3.1%, monocular depth relative RMSE drops by 18.4% under extreme rotation, and on the smaller N=40 val split the 1%-label gain rises to +4.10 mIoU.
同一修正还可迁移到其他任务：零样本 Structured3D 准确率提升 +3.1%，单目深度在极端旋转下的相对 RMSE 下降 18.4%，而在更小的 N=40 验证划分上，1% 标签的增益升至 +4.10 mIoU。
Cite this work. 引用本文。
If you find our work or code useful in your research, please cite: 如果我们的工作或代码对您的研究有所帮助，请引用：
@article{zhou2026gauge,
title = {Gauge-Equivariant Attention for Rotation-Stable {360$^\circ$} Scene Understanding},
author = {Zhou, Tianjian and Li, Yishan and Jiang, Jie and Zhang, Yifei},
journal = {ACM Transactions on Graphics},
volume = {45},
number = {6},
year = {2026},
note = {Article 242},
doi = {10.1145/3842525}
}
ACM Transactions on Graphics, Vol. 45, No. 6, Article 242 · DOI 10.1145/3842525 ACM Transactions on Graphics · 第 45 卷第 6 期 · 文章 242 · DOI 10.1145/3842525