Add Category
Updated
The statistical justification for proposed AFL rule changes does not stand up to scrutiny.
The league's football operations manager, Steve Hocking, this week trumpeted a 15 per cent increasing in scoring by teams involved in recent VFL trial matches.
But, when further analysed, those trials actually produced on average nine points fewer than games involving those sides played this season under the existing rules.
Moreover, the breadth of sample data is so small that definitive conclusions are impossible to draw.
In less than two weeks, the AFL Commission will meet to consider radical rule changes for next season. Among the proposals put before it will be one to enforce starting positions at every centre bounce, and another to double the length of goal squares to 18 metres.
In announcing the AFL competition committee's recommendations on Thursday, Hocking highlighted data gathered from trials conducted in three recent VFL matches.
That's right: three matches. Played at the end of the season. By four of the competition's weakest teams.
"I feel like what's been put in front of the Commission is well-researched," said Hocking.
"It's been trialled well beyond any of the trials that have ever gone on previously. The centre diamond was plugged in for one game and it's now a square."
The centre square was introduced in the 1970s though, several decades before the benefits of big data analytics had been brought to bear.
Put simply, Hocking, who was approached for this article but declined to comment, has a sample size problem.
Ask any statistician what they can infer with any significance from three data points and the answer will invariably be "nothing". Trends can't be identified. Factors such as weather, accuracy and team style can't be controlled for. Causation can't be proved.
Yet none of that has stopped the AFL from trying it on.
"All the stuff that we're putting forward is well-researched," insisted Hocking. "I'm very, very comfortable and satisfied with the work that's been done and the response today from the comp committee I couldn't be happier with."
Hocking said the VFL trials of the rules had resulted in an increase in scoring for the teams involved based on their season average.
Taken at face value — and ignoring the obvious sampling issues — that number seems to present a persuasive case for change after what has been the lowest-scoring AFL season in half a century.
| Average score in non-trial matches | Average score in trial matches | Difference | Percentage difference | |
|---|---|---|---|---|
| Werribee | 81 | 105 | 24 | 29.1 |
| Northern Blues | 67 | 91 | 24 | 35.6 |
| Sandringham | 74 | 52 | -22 | -30.1 |
| Coburg | 61 | 69 | 8 | 12.9 |
| WEIGHTED AVERAGE | 68.5 | 79.3 | 10.8 | 15.7 |
*Averages weighted for number of trial games
As the above table shows, three of the four teams scored more in the rule trial games than they averaged for the rest of the year.
However, it needs to be noted that those clubs were also four of the VFL's five worst. They were explicitly chosen to take part in the trials because they were no longer in finals contention.
Their higher-than-average team scores can hardly be considered surprising when they were up against equally porous defences. Weak teams will generally score more against each other than they do against top sides.
In fact, the ABC's analysis suggests the trials resulted in lower-than-expected scoring.
The three trial games produced an average of 159 points per game, whereas all other games involving the four competing clubs this season resulted in an average of 168 points.
Two of three games saw less scoring than either participating team's other unaltered fixtures.
| Match score | Trial game total | Rest of season game total | Trial game points difference | |
|---|---|---|---|---|
| Werribee | 105 | 163 | 176.8 | -13.8 |
| Coburg | 58 | 163 | 170.8 | -7.8 |
| Northern Blues | 96 | 176 | 161.6 | 14.4 |
| Coburg | 80 | 176 | 170.8 | 5.3 |
| Northern Blues | 86 | 138 | 161.6 | -23.6 |
| Sandringham | 52 | 138 | 163.1 | -25.1 |
| Average | 159 | 168.1 | -9.1 |
*Teams which played twice are only counted once
Looked at this way, scoring was actually down 5.5 per cent in the VFL trials.
If a similar trend was carried over to the AFL, it would see scoring drop to 157 points per game (78.5 points per team).
The fact that overall scoring in the trials was below average even when some of the competing teams' scores were above average is instructive. It shows us that those sides failed to score as much as expected against the defences they faced.
For example, Coburg conceded about 110 points per game this year, which was 136 per cent of the average VFL team's 81-point score. Therefore, rival teams should have been expected to score 136 per cent of their own season average score when facing Coburg.
Werribee's 81 points per game in non-trial games was right on the VFL average, so it would have been expected to score 110 points against Coburg if the match was played under normal conditions.
Under the new rules, however, it only managed 105 points.
A real trial producing valid statistical information about scoring would involve collecting large numbers of data points: an entire season's worth at a minimum
When overseas sporting bodies such as the NBA, NHL, and Major League Baseball want to test rule changes, they often spend years conducting trials in minor and reserve leagues to observe teams' responses and to uncover any unintended consequences.
Three late-season dead rubbers involving relatively weak VFL teams is not a real trial. It is a pretext to a rubber stamp.
Topics: victorian-football-league-vfl, australian-football-league, sport, vic, australia
First posted