Google's Kaggle platform and DeepMind are organizing an AI chess demonstration tournament to evaluate the reasoning and planning skills of the best large-scale language models used by platforms like Chat GPT, Gemini, and DeepSeek. Following the tournament, a continuously updated leaderboard with rankings will be available on the Kaggle platform, providing insight into which AI is the best at chess.
The first open AI chess tournament on Kaggle Game Arena is an event launched by Google and its Kaggle platform, along with DeepMind, a lab developing advanced artificial intelligence solutions. The three-day tournament will take place from August 5-7 and will feature eight of the world's leading AI models.
Participants will include models from Google, OpenAI, Anthropic and other laboratories: Gemini 2.5 Pro (Google), Gemini 2.5 Flash (Google), o3 (OpenAI), o4-mini (OpenAI), Claude 4 Opus (Anthropic), Grok 4 (xAI), DeepSeek R1, Kimi k2 (Moonshot AI).
Machines fight for the champion's cup
The competition is played in a knockout format, with models competing in best-of-four games to determine the winner. The first day will feature four quarterfinal matches, the second day two semifinals, and the final on the final day.
kaggle.com
The games are broadcast live on the Kaggle YouTube channel with commentary by Hikaru Nakamura, an American chess grandmaster of Japanese descent. Daily summaries will be provided by Levy Rozman, a chess master and creator of the world's most popular chess YouTube channel (GothamChess). The tournament review and final commentary will be provided by Magnus Carlsen, a Norwegian grandmaster and one of the greatest chess players in history, often referred to as the “Mozart of chess.”
All moves in duels are communicated in text format, with no visual interface. The model is not provided with a list of possible legal moves—it must identify them independently and cannot use external chess engines such as Stockfish, the world's most powerful publicly available computer chess program. If the model makes an illegal move, it has three attempts to correct the error—a third unsuccessful attempt means a loss of the game. The model has a maximum of 60 minutes for each move.
Hundreds of additional all-play-all matches are played simultaneously between models, creating a statistically reliable leaderboard, permanently available on Kaggle Benchmarks. The tournament serves not only to determine “who will win,” but above all to create a sustainable, open benchmark for testing the broad reasoning skills of AI models.
Kaggle Game Arena Platform
This is the first widely open and public competition of its kind, designed to demonstrate AI's reasoning abilities in the context of a strategic game, with a high level of transparency regarding both the rules, the competition process, and the model results. The initiative aims to create a new, transparent benchmark that assesses not only chess-specific abilities but also broader abilities to reason, adapt, plan, and predict opponents' moves.
Google announces that similar tournaments will be regularly organized within the Kaggle Game Arena, and other games, such as the board game Go, originating from ancient China, or Werewolf, a social party game with elements of deduction and bluffing, are already planned as further tests of the same models.
The company emphasizes that games like chess are one of the best ways to reliably assess the reasoning abilities of LLM models. This is because they are resistant to what Google calls “saturation”—in other words, to being solved using a standard formula. Chess was chosen because it is incredibly complex. Claude Shannon estimated the number of theoretically possible chess games to be around 10,120, which is significantly larger than the number of atoms in the universe.
Kaggle and DeepMind are now integral parts of Google, or more precisely, its parent company, Alphabet Inc. Kaggle was acquired by Google in March 2017 and operates as a platform within Google Cloud. It specializes in organizing data science and machine learning competitions. Google acquired DeepMind in 2014. Initially operating as an independent team, it was formally merged with the Google Brain team in 2023. It develops advanced AI solutions such as AlphaGo, AlphaFold, and the Gemini family of models.
It's worth noting that Kaggle, in collaboration with DeepMind, released AlphaZero, a chess-playing computer program, in 2017, which was never publicly available. Using machine learning and playing millions of games against itself, it took the chess world by storm, defeating Stockfish in a 100-game match. In late 2018, it won a second match against Stockfish, which consisted of over 1,000 games.
The AI models competing in Kaggle don't quite reach the level of AlphaZero. Unlike it, they aren't programmed specifically for chess. As Chess.com details, these models are still learning and are known for making both illegal moves and absurd retirements. A chess competition between AI models might be interesting, but will it match the excitement of, for example, Gary Kasparov's rivalry with IBM's Deep Blue supercomputer? Time will tell, and until then, it's worth revisiting this duel and the controversies surrounding it, for example, thanks to the TV series “Revenge.”
Michał Kubicki
