DeepMind cofounder Shane Legg thinks that search is essential for a model to be genuinely creative. Pre-trained base models can do incredible things. But Shane thinks this is just a matter of them mixing together existing concepts from their training data. If he's right, coming up with genuinely novel ideas always involves searching a large space for "hidden gems".