With the IBJJF's new seeding system at tournaments, we can now analyze the accuracy of fighter seeds and its ability to predict who will win a division. The fun in this is we get to see where upsets occur. What makes sports tournaments exciting is when an underdog comes out on top or a heavy favorite is able to win again and again.
Highlights:
While the methodology for the IBJJF's seeding system is not public knowledge, we can speculate that past tournament performance plays into the seeding. It is hearsay that players are penalized if they switch between age divisions and weight classes. And when a fighter hasn't competed in a while, it works against them. Now I wouldn't personally use the seeds as betting odds, but hey it is cool to analyze. Let's see what I was able to uncover with the available data.
First, let's talk about the data with which we are working. At the 2019 IBJJF No Gi Pans tournament, there were 1,180 recorded matches (note: the juvenile blue belt male/female opens were not recorded by IBJJF, nor was the purple belt adult open class division. These are missing from the data set).
Of those 1,180 matches, 639 matches resulted in the higher ranked fighter winning the match and 541 matches occurred where the lower ranked fighter beat the higher ranked fighter, resulting in an upset.
Next, I segmented out the masters divisions for comparison, since the masters divisions can be hit or miss on seeding and some masters do not compete regularly. This affects the accuracy of seeding for the masters divisions. With adults and juveniles only, there were 596 total matches, 329 correctly predicted victories, and 267 upsets.
Overall, the predictive accuracy at No Gi Pans was 54% with the masters divisions and 55% with the masters taken out.
Predictive accuracy is defined as the number of matches correctly predicted to win (higher ranked fighter beats a lower ranked fighter) divided by the total number of matches. Generally, collegiate wrestling seeding systems render about a 67% predictive accuracy(1), so the seeding at No Gi Pans was on the lower end. We will examine this again around No Gi Worlds after several other major tournaments take place this fall, including the European and Pan Pacific No Gi Championships.
Next, let's take a look at the number of predicted victories by belt. More upsets occurred at purple belt, compared to other belts. Out of 166 total purple belt matches, 47% resulted in upsets. Upsets at brown belt followed closely at 45%.
The Medium Heavy Division, followed by Super Heavy, had more upsets than correctly predicted victories. The Heavy weight class was the most accurately predicted with 62% correctly predicted victories compared to upsets.
In reference to age, the adult division had the most number of matches overall. Its overall predictive accuracy was 55%. The Masters 3 division had more upsets than correctly predicted victories.
Mat 6 had the most matches at 100 adult matches over two days, followed by Mat 6 at 97, and Mat 5 at 95. Mat 8 only had 52 adult matches in two days. Overall, Mat 6 had almost as many upsets as it did correctly predicted victories. Mat 3 had more upsets than victories and Mat 4 and 7 had a great deal more victories than they had upsets for both days of the event.
The teams with the most upsets compared to correctly predicted victories included:
Thanks for reading!
(1) Stefani R., 2011. The methodology of officially recognized international sports ranking systems, article, Journal of Quantitative Analysis in Sports 7(4), 9. doi: 10.2202/1559-0410.1347
Highlights:
- There were 541 upset matches compared to 639 correctly predicted victories at the IBJJF No Gi Pan. For adults divisions, there were 329 correctly predicted and 267 upsets.
- IBJJF seeding system has a lower predictive accuracy than collegiate wrestling
- The most upsets occurred at purple belt
- More upsets occurred in the Medium Heavy division than other weight classes
- The Masters 3 division had more upsets than victories
- More upsets occurred on Mat 3 than any other mat
- The female divisions had almost as many upsets as victories
- More upsets occurred on Sunday than Saturday
- Most upsets occurred by a fighter that was 4 seeds lower
While the methodology for the IBJJF's seeding system is not public knowledge, we can speculate that past tournament performance plays into the seeding. It is hearsay that players are penalized if they switch between age divisions and weight classes. And when a fighter hasn't competed in a while, it works against them. Now I wouldn't personally use the seeds as betting odds, but hey it is cool to analyze. Let's see what I was able to uncover with the available data.
First, let's talk about the data with which we are working. At the 2019 IBJJF No Gi Pans tournament, there were 1,180 recorded matches (note: the juvenile blue belt male/female opens were not recorded by IBJJF, nor was the purple belt adult open class division. These are missing from the data set).
Of those 1,180 matches, 639 matches resulted in the higher ranked fighter winning the match and 541 matches occurred where the lower ranked fighter beat the higher ranked fighter, resulting in an upset.
Predictive Accuracy = correctly predicted matches
total matches Predictive accuracy is defined as the number of matches correctly predicted to win (higher ranked fighter beats a lower ranked fighter) divided by the total number of matches. Generally, collegiate wrestling seeding systems render about a 67% predictive accuracy(1), so the seeding at No Gi Pans was on the lower end. We will examine this again around No Gi Worlds after several other major tournaments take place this fall, including the European and Pan Pacific No Gi Championships.
Mat 6 had the most matches at 100 adult matches over two days, followed by Mat 6 at 97, and Mat 5 at 95. Mat 8 only had 52 adult matches in two days. Overall, Mat 6 had almost as many upsets as it did correctly predicted victories. Mat 3 had more upsets than victories and Mat 4 and 7 had a great deal more victories than they had upsets for both days of the event.
The female division's seeding was further from the predictive accuracy norm, where upsets occurred 52% of the time. It is likely that female entrants competed less frequently in no gi leading up to the event and thus the seeding system could not take this into account since there was no data on those fighters. Thus, upsets were bound to occur if some seeds were assigned at random.
Overall, there were more matches on Sunday than Saturday, 382 and 214 respectively. It is interesting to see that there was a greater gap between upsets and victories on Sunday.
So, how big were the upsets? With the masters divisions included, most upsets occurred when a fighter of a lower rank by 1 or 2 seeds won. For the adult divisions only, the most upsets occurred when a fighter of a lower rank by 4 seeds beat a higher ranked fighter. Also noteworthy, there were 29 matches where a fighter seeded 8 positions below upset a higher ranked fighter. Upsets with a scoring higher than 20 usually occurred at the blue or purple belt levels. Upsets in the teens often resulted when a previously unranked fighter won the division and upset all their opponents.
- GF Team with 25 upsets to 8 victories
- Brazilian Top Team - 23 upsets to 17 victories
- Balance - 13 upsets to 10 victories
- Nova Uniao - 8 upsets to 3 victories
- Open Mat Mixed Martial Arts - 8 upsets to 3 victories
Thanks for reading!
(1) Stefani R., 2011. The methodology of officially recognized international sports ranking systems, article, Journal of Quantitative Analysis in Sports 7(4), 9. doi: 10.2202/1559-0410.1347