"The final point I need to make, which does flatter the closed labs, is that the likes of Anthropic and OpenAI moved on from caring about these indices months, or even years, ago. Their focus is to a large majority on supporting internal uses – like AI R&D automation – and the use-cases of customers who are spending billions of dollars."
Is this true? If yes it makes a lot of sense. Time and time again, when I try some newly released open weights model I am disappointed. Even some older Claude models are beating them on quality of answers, even though they score less on intelligence in benchmarks.
"The final point I need to make, which does flatter the closed labs, is that the likes of Anthropic and OpenAI moved on from caring about these indices months, or even years, ago. Their focus is to a large majority on supporting internal uses – like AI R&D automation – and the use-cases of customers who are spending billions of dollars."
Is this true? If yes it makes a lot of sense. Time and time again, when I try some newly released open weights model I am disappointed. Even some older Claude models are beating them on quality of answers, even though they score less on intelligence in benchmarks.