
Each year category resolves to the percentage of people worse than the best AI at predicting the future.
To measure who is better or worse at predicting the future, I will look at how the best AI system did in the ACX prediction contest. In summary, it is about placing a bunch of predictions at once on relevant topics. You can read more about it here: https://www.metaculus.com/tournament/2023-contest/.
I might use a different contest if I find a better one.
Similar market about problem solving: https://manifold.markets/patrik/how-good-will-ais-be-in-solving-com.
@MalachiteEagle Very interesting. Tho I think something like they used for openai o1 and o3 models but with questions that are after the training data could be even a better way.
I don’t understand the resolution yet, is it that you will use some AI systems to forecast the ACX forecasts and then compare the best performing one to the distribution of human results?