Here's a quick post-tournament update on our Elo Rating for BJJ. This post is an analysis of the statistical model, not the matches or fighters.
We calculated the predictive accuracy by totaling the number of correctly predicted wins for fighters that had a higher probability of winning, divided by all matches with a prediction probability.
Out of 121 matches, the Elo Rating for BJJ correctly predicted 87 matches, or 72%. Seemingly high, there were 16 matches where the fighter did not show up to fight or the match ended in a disqualification. Fighters that were no-shows included Hiago George, Rudson Mateus, Helton Junior, and Yan Cabral.
Our Elo Rating for BJJ posted on January 20, 2020, is reflective of fighters' performances in 2019 (refer to the article here for full methodology). While we will explore updating the rating system, the Elo model is self-correcting over time. In the final section below, we've listed the biggest changes as a result of the tournament.
By way of comparison, the IBJJF seeding system had a predictive accuracy of 64% for the European Championships. The IBJJF seeds fighters based first on the number of points earned at an IBJJF Grand Slam tournament, then with overall points, and finally taking into account past world champions. We define the IBJJF's predictive accuracy as the number of matches won by a higher seeded fighter over a lower seeded fighter, divided by the total number of matches.
From previous studies, we found that collegiate wrestling rating systems have on average produced results with a predictive accuracy of 67%. Our model has improved on both of those systems.
Highlights:
We accurately predicted a few noteworthy first round matches with our Elo Rating system.- Thalison Soares was slightly favored to win over Bruno Malfacine, with a win probability of 54.48%, and beat the long-time reigning Rooster weight world champ.
- Gabriel Sousa went up a weight class to Feather, but was still favored to win against Gianni Grippo, with a win probability of 65.64%.
- Manuel Ribamar edged out Espen Mathiesen, with a win probability of 58.80%, and won his first European title.
- Italo Moura beat Gabriel Figueiro, with a win probability of 54.39%.
- We correctly predicted 20 first round match ups out of 24 matches, or 83% first round predictions (excluding no shows).
Our Elo Rating for BJJ had a predictive accuracy of 72% overall.
We calculated the predictive accuracy by totaling the number of correctly predicted wins for fighters that had a higher probability of winning, divided by all matches with a prediction probability.
Elo Predictive Accuracy = matches won with higher probability
total matches Biggest Upsets:
- Patrick Gaudio over Felipe Andrew in Super-Heavy - Andrew had a 71.72% win probability.
- Felipe Andrew's win against Keenan Cornelius in the Open Class - Cornelius had a pretty high scoring for the absolute division.
- Leonardo Saggioro over Gabriel Sousa in Feather weight - Sousa had a 66.68% win probability.
- Bruno Santos Lima over Gabriel Almeida in the Medium Heavy division - Almeida had a 63.04% win probability.
- Pedro Ramalho over Alexandre Joaquim in Middle weight - Joaquim had a 60.53% win probability.
Tweaking the Model:
There is always room for improvement in any statistical model. The most notable limitation was that past world champions that only compete at the IBJJF World Championship were rated lower than others for not competing very much in the previous year. The Elo rating system rewards fighters that have more matches. Names like Marcus Buchecha, Bruno Malfacine, and Mikey Musumeci, received lower scores than intuition would lead us to believe. Again, Malfacine was rated lower than Thalison Soares before the tournament because he only had two matches in 2019, which was at the World Championships and he lost one of those matches to Musumeci. In addition, fighters like Keenan Cornelius jumped around to 5 weight classes, and thus his rating was lower than expected in all weight classes but rather high in the absolute division.Our Elo Rating for BJJ posted on January 20, 2020, is reflective of fighters' performances in 2019 (refer to the article here for full methodology). While we will explore updating the rating system, the Elo model is self-correcting over time. In the final section below, we've listed the biggest changes as a result of the tournament.
By way of comparison, the IBJJF seeding system had a predictive accuracy of 64% for the European Championships. The IBJJF seeds fighters based first on the number of points earned at an IBJJF Grand Slam tournament, then with overall points, and finally taking into account past world champions. We define the IBJJF's predictive accuracy as the number of matches won by a higher seeded fighter over a lower seeded fighter, divided by the total number of matches.
IBJJF Predictive Accuracy = matches won with higher seed
total matches From previous studies, we found that collegiate wrestling rating systems have on average produced results with a predictive accuracy of 67%. Our model has improved on both of those systems.
Noteworthy Changes on the Elo Rating for BJJ