"Training Language Models to Fail Gracefully" Due to how we use RL for LLMs, there is currently little incentive for models to admit uncertainty, preventing them to guess even when they're unlikely to succeed. This paper GracefulRL instead trains models to prefer success over safe fallback, and safe fallback over failure. They did it by using a modified RL objective that prevents the model from collapsing into always refusing. On AIME, it reduces incorrect answers from 79.1% to 8.9%, while…