When you send a prompt to a model, a couple of things happen: 1. The model processes the prompt and generates the first token (prefill stage). 2. Then, the model continuously processes the prompt + new tokens until it produces the complete answer (decode stage) These guys are doing something really cool: They are using different hardware for each stage. They run prefill on GPUs and decode on Positron's specialized inference systems. This results in faster generation.