Simple Ideas

This is the whole secret of AlphaFold 2. That you use the internal frame, so it's invariant to the global frame you choose. That's high-school level math, but it's really the whole secret. Just do this and it will get you a nobel prize in chemistry.

There are probably more eloquent ways this has been formulated, but this is how I remember Shengchao Liu has put it during a talk about multi-modal foundation models at AAAI25, and it got me thinking. Obviously, it's a bit (or lot) of a hyperbole, but I think the intuition still stands:

Simple, elegant ideas often yield the most powerful solutions

We already know that incredible genius ideas often seem obvious in hindsight. They are (often) not some immensely convoluted process, but rather a key insight that unlocks something. In the case of AlphaFold 2 that was to use a variant of attention invariant to the global frame. Now I'm not saying it's a trivial feat, the formula can still look intimidating, but the idea is really that simple.

So, AlphaFold 2 is not a big deal? It's definitely a big deal. I don't mean to discredit their idea (and there's a ton more to AlphaFold), but rather to encourage taking a step back and think about the problem you're facing on a high level. What is really going on? More often than not, you can find a rather simple way to improve things.

What do we need then? What matters now more than anything is agency. Agency is a bit hard to define, but what I mean is close to what Wikipedia says under political economy:

Human agency refers to the ability to shape one’s life.

If you're driven and motivated (and a supportive environment helps here), then you will eventually find a way. To give another example from Henrik Karlsson.

When I think of agency I picture Werner Herzog's brother Lucki. When they ran out of money while filming Fitzcaralldo, Lucki, stuck in Lima, gets a car and goes locking for expensive houses to see if he can find someone he can convince to sign a check for a few 100k before sunrise.

This feels profoundly agentic because 1) what they are trying to do springs from a profound inner vision (they want to pull a steam ship over a mountain in the Amazon) and 2) he just won't take no for an answer.

Kaparthy says that agency is more important than intelligence. I think even in research focused places, there is an essential role to be fulfilled by people that bring the right people together and push projects forward. In an imagined world where artificial intelligence, actual intelligence, exists, agency as an orchestrator of intelligence resources will perhaps become even more important than it is today.