the goal is to have an agent that can:
- Understand a complex problem description.
- Generate initial algorithmic solutions.
- Rigorously test its own code.
- Learn from failures and successes.
- Evolve increasingly sophisticated and efficient algorithms over time.
Exactly, pairing LLMs with a genetic algorithm that allows evolving a solution towards the goal is a really powerful approach. The other piece of the puzzle is models like MiMo from Xiaomi that’s relatively small at 7B params, but performs at the level of much larger models. I suspect you could likely create even smaller models that are highly specialized for specific types of tasks, and then pair them with algorithms like Alpha Evolve to iteratively work towards solutions in an efficient manner.