"Multi-Stream Attention Residuals" An alternative to mHC and AttentionRes? Standard residual connections combine information from all previous Transformer layers into one hidden state, making earlier information harder to access as models get deeper. Multi-Stream Attention Residuals (MSAR) lets each Transformer layer retrieve information from earlier layers using multiple attention heads, with each head looking at a different range of previous tokens to decide which information to…