Given that this is a well known example, presumably OpenAI included a training set using letters.

Not everyone got the same results.

https://news.ycombinator.com/item?id=41530647

Which is to be expected, the answer isn’t deterministic.

can you ask it:

9.11 and 9.9, which number is larger

> Evaluating Number formats > I'm considering that the question might be about comparing two numbers, 9.11 and 9.9, in different contexts such as different locale formats or representing dates. 9.11 is larger than 9.9

Well, it's kind of right because we left it ambiguous what kinds of numbers these are. September 11 is a greater date than September 9. However, I would want the model to have the street smarts to assume without further context, I must be just talking about numbers as in numbers.

Was it trained on Excel code?

it claims 9.11 is larger, hah.

https://postimg.cc/KKKLZNjW

That's odd. Tried several times with the same prompt and always got the correct answer with step by step explanation.

Those models are just not deterministic.

4.9 is larger than 4.11.

Explanation:

• 4.9 is equivalent to 4.90. • 4.11 remains 4.11.

When comparing the numbers:

• 4.90 (which is 4.9) • 4.11

Since 4.90 > 4.11, 4.9 is the larger number.

Try it with 9.11 and 9.9. Some have proposed that it thinks "9.11" is larger because in a social context 9/11 has much larger significance.

From o1-mini:

To determine which number is larger between 9.11 and 9.9, let's compare them step by step:

Understanding the Numbers: 9.11 can be read as "nine and eleven hundredths." 9.9 can be read as "nine and nine tenths." To make the comparison easier, you can also think of it as 9.90 (adding a zero doesn't change the value).

Comparing the Decimal Places: Ones Place: Both numbers have 9 in the ones place. Tenths Place: 9.11 has a 1 in the tenths place. 9.90 has a 9 in the tenths place.

Conclusion: Since 9.90 has a larger value in the tenths place compared to 9.11, 9.90 (or 9.9) is larger than 9.11. Therefore, 9.9 is larger than 9.11.