1. Head tube should aligned with the fork. GPT-5.6 Luna is particularly terrible here.

2. Bikes are almost always depicted going rightwards so you can see the drivetrain. 4 failures in this.

3. Only ~7 of the 23 models achieved a diamond bicycle frame shape

4. Almost all the models have the bird directly over the crankset; as little as 4 or 5 have the birds sitting on a set-back bicycle seat.

> The Astra pelicans are much better.

It's absolutely in the training data now.

Time to retire this (previously mildly amusing) benchmark as frontier models are now pelicanmaxxed.

What training data? Is there a big library of excellent SVG pelicans to train of?

Yes, all the people talking about this benchmark since OP created it.

Is it just me or is it kind of remarkable that all the remotely competent pelicans on bicycles kind of look the same?

I assume that these have been in the training set for quite a while, but that they’re still a bit hard for the models because it’s not covered by the RL pipelines.