Elite Banks Predicted World Cup With AI Models. They Weren’t Even Close.

When Goldman Sachs tweaked its predictive models halfway through the tournament, the results were even more wrong.

Elite banks' World cup predictions weren't even close.
Goldman Sachs’ predictive algorithms have improved significantly since 2014, but the results were still wrong. Srdjan Stevanovic/Getty Images

Few things could slow down the world’s busiest investment bankers like the World Cup. Over the past three weeks or so, instead of fine-tuning artificial intelligence algorithms to help make investment decisions, several large banks diverted these expensive and sophisticated tools to predicting who would win the World Cup.

Sign Up For Our Daily Newsletter

By clicking submit, you agree to our <a rel="nofollow noreferer" href="http://observermedia.com/terms">terms of service</a> and acknowledge we may use your information to send you emails, product samples, and promotions on this website and other properties. You can opt out anytime.

See all of our newsletters

But sadly, none of them were even close.

Goldman Sachs, which crunched countless data points about teams and individual players into four machine learning models and picked the most likely results out of one million simulations, had France, Brazil, Portugal and Germany in the semifinals. And then Brazil was supposed to beat Germany 2-1 in the final match.

“For the doubters out there, this final result was cross-checked in excruciating detail by our (German) Chief Economist Jan Hatzius!” The investment bank said confidently in a report.

UBS made similar predictions about the contenders in the final rounds. Based on “an objective skill-level measurement,” called an “Elo rating,” a team’s scores in the pre-qualification, and other factors, the Swiss bank regarded Germany as the most likely champion with a 24 percent of chance of winning, followed by Brazil and Spain.

Dutch bank ING Group adopted a rather different approach by measuring the teams’ market value, assuming that a team’s market worth is closely correlated to its success. As such, ING predicted that Spain, the most valuable team in the world, would beat France, the second most valuable, in the final match.

Of course, except for France, none of the teams mentioned above made it into the semifinals.

Aware of the inherent randomness of soccer, Goldman Sachs actually updated its statistical models throughout the tournament. In an updated bracket projection published on July 9, one day before the semifinals kicked off, the bank predicted that Belgium would beat England in the final.

If anything, algorithms did a great job getting everything wrong this time. Neither of the teams made it into the final.

What’s even more embarrassing is the fact that Goldman Sachs’ prediction this year was further off than its first attempt to forecast the World Cup in 2014. The projection four years ago used a much smaller data set, working with only team-level data, such as the teams’ rankings and the number of goals scored in the last 10 international games. The simulations failed to predict the champion, but they accurately named three of the four teams in the semifinals.

“It is difficult to assess how much faith one should have in these predictions… But the forecasts remain highly uncertain, even with the fanciest statistical techniques, simply because football is quite an unpredictable game. This is, of course, precisely why the World Cup will be so exciting to watch,” wrote Manav Chaudhary, Nicholas Fawcett and Sven Jari Stehn, the three economists behind Goldman Sachs’ project.

Elite Banks Predicted World Cup With AI Models. They Weren’t Even Close.