时间序列预测中位置编码的引人注目特性
Transformer-based methods have made significant progress in time series forecasting, but research on positional encoding remains insufficient. This paper introduces two new positional encodings, Temporal Position Encoding (T-PE) and Variable Positional Encoding (V-PE), and a Transformer-based dual-branch framework named T2B-PE, demonstrating superior robustness and effectiveness in extensive experiments.
本文研究了基于解码器的Transformer模型在不同位置编码方式下对长度泛化的影响，发现NoPE表现更优且无需额外计算。同时，scratchpad对解决长度泛化问题并不总是有帮助，其格式对模型性能有很大影响。表明解码器-only的Transformer不一定需要显式的位置嵌入以在更长序列上泛化良好。
