.@fleetingbits on how multi-agent RL could accidentally reward models for learning to jailbreak each other: "Pretend that an agent in a multi-agent training environment is malfunctioning. It can be in the interest of agents to develop the ability to jailbreak their fellow agents, because that helps them complete the task and therefore all be rewarded." "You would see the reward go up as you did your training run. And then at the end, when you released it into the world, your models might be…