Before February 2025, will a Gemini model exceed Claude 3.5 Sonnet 10/22's Global Average score on Simple Bench? | Manifold

Before February 2025, will a Gemini model exceed Claude 3.5 Sonnet 10/22's Global Average score on Simple Bench?

Basic

11

Ṁ501

Feb 2

9%

chance

1D

1W

1M

ALL

https://simple-bench.com/ Claude 3.5 Sonnet 10/22 achieves 41.4% whereas the best Gemini model scores 27.1%

Update 2025-22-01 (PST): - Resolution Date: The market will now be resolved on February 1st, 2025 instead of the previously stated date. (AI summary of creator comment)

This question is managed and resolved by Manifold.

#New Year's Resolutions 2025

Get

1,000

and

3.00

Sort by:

bought Ṁ50 YES

Can we go ahead and resolve this one?

@rogs Won't resolve it until February 1st

bought Ṁ5 NO

Now I'm looking at my comment above and wondering what I was thinking. Did I think this was a post about OpenAI models vs Claude rather than about Gemini vs Claude? Why did I think it was resolvable already?

Related questions

What will be the best score on Cybench by December 31st 2025?

What will Claude 3.5 Opus's reported 0-shot performance on GPQA Diamond be upon release?

How long until one of Gemini, Claude, etc... match the capabilities of O1?

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Math Evaluation

Will Gemini 2 be released before EOY 2025?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

Will Gemini exceed the performance of GPT-4 on the 2022 AMC 10 and AMC 12 exams?

What will be true of Gemini 2?

Will "Gemini [Ultra, 1.0] smash GPT-4 by 5x"?

Related questions

What will be the best score on Cybench by December 31st 2025?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

What will Claude 3.5 Opus's reported 0-shot performance on GPQA Diamond be upon release?

Will Gemini exceed the performance of GPT-4 on the 2022 AMC 10 and AMC 12 exams?

How long until one of Gemini, Claude, etc... match the capabilities of O1?

What will be true of Gemini 2?

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Math Evaluation

Will "Gemini [Ultra, 1.0] smash GPT-4 by 5x"?

Will Gemini 2 be released before EOY 2025?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules