This is a critique, not a criticism. Short history lesson: In the early months of this blog I likened the then UCP to Logan's Run and suggested the system (and the league) was unnecessarily driving players up and out of competitive paintball. Which was true. Since that time the classification system has been adjusted a number of times in ways that (intentionally or unintentionally) fell in line with VFTD's original criticisms.
In a recent post about Social Paintball I took them to task a bit for a contributor who simply regurgitated the old claim--first made here--purely to engender some controversy. It has since occurred to me that the system was tweeked again and that I hadn't reviewed in detail the latest changes and if I was going to give Social some grief over it I oughta make sure nothing major had changed. And nothing major has changed--but a situation that hadn't been addressed before has been now, sorta. That is the relative merit of a score received depending on the size of the division.
One of the score modifiers is # of teams in a division. Any number 10 and above doesn't modify the received score. For divisions less than 10 teams there is a modifier that reduces the overall score received based I presume on the idea that it's more difficult to win out in a bracket of 20 teams than it is in a bracket of 8 teams and regardless there is necessarily a higher likelihood of competitive intensity in the larger bracket. Conceptually I'm okay with that but where I think an issue exists is at the other end of the team numbers spectrum.
Given the current scoring system every team, other than first and last, receive a score that accounts for the number of participants--at least in part. For example in a division of 12 second place receives 92 points (rounded) while the same second place finish in a division of 8 receives 87 points. The # teams modifier goes further and will reduce that base score even further--which is okay--but that's not where the score inflation problem lies. Score inflation produces disparate values when the divisions get bigger.
Btw, the reason this is a critique and not a criticism is because, despite the score inflation, a quick crunching of the numbers demonstrates that, unlike in times past, this won't effect the upper divisions where the problems of the past primarily resided but it might have an impact on the lower divisions, principally D3 & D4.
Here's an example: 6th place in a division of 8 is 35 points (rounded); in a division of 10 it's 50 points & in a division of 20 teams it's 75 points. In a division of 30 teams 6th place receives 85 points. Each result is a 6th place finish but also results in a swing of 50 points between the lowest and highest score. Of course in one sense the swing reflects the degree of difficulty in arriving at that placement; 6th out of 30 is considerably more difficult--at least statistically--than 6th out of 8 and while I happily concur that's not the issue either.
The issue with score inflation is that the classification system determines a player's status based on multiples of a base score. If your division nevers exceeds 12 or 15 teams it own't ever be a concern. In divisions that routinely reach or exceed 30 teams it could be a problem. And when I say "problem" I mean produce a result where players are bumped up when they shouldn't be. Fortunately the present system has sufficient safe guards--and most D4 & D3 teams don't play a full season--so that score inflation alone shouldn't push players (& teams) up a division.
A simple way to address the issue would be to reconsider the size of the smallest max score division and adjust the reducing multipliers--or leave it as is accepting score inflation because most of time it probably won't be a problem.
(And of course it ought to go without saying, but I'm saying it anyway, I may differ with APPA on occasion but only because it is a serious and legit effort--unlike the revenue stream mockery used by the other national league.)
Friday, January 27, 2012
Subscribe to:
Post Comments (Atom)
Posts
11 comments:
Another way to look at it is if you get 6th of 8 teams, 68% of the teams beat you. If you get 6th of 30 teams, you beat 80% of the teams. 6th of 8 is just a bit better than bothering to show up; 6th of 30 is narrowly missing the semi-finals at a national-level tournament after either winning your prelim bracket or at least one playoff match.
The original idea was that scores should reflect overall performance - if you place 3 of 5, or 6 of 11, or 16 of 31, those are all middle-of-the-pack performances, and should get middle-of-the-pack scores. Similarly first is first and last is last - it doesn't make much sense for last place at a 6-team tournament to yield a different score than last place at a 30 team tournament; either way you lost all your games.
After some experience, it became apparent that there were people doing well at smaller events and getting the scores to be bumped, even though their wins might have been as much a reflection of the lack of competition as the necessary skills to move up. 10 teams and up you should have a good representation of skills; less than that we adjust points down to reflect that smaller events are less likely to prove top-level performance.
The important consequence is you can win 5-team events all the time and not get bumped; you have to dominate a 6, 7 or 8 team series to get bumped, and when you start getting up to 10+ team events we're back to "consistent performance" (targeted at consistent top-20%) will earn you a promotion.
We looked through a LOT of players, especially players around the division cut-off points, and while nothing we do is going to be perfect for everyone, I'm very confident this is the best set of classification rules we've put together yet.
I'm also sure that with another season of information, we'll make some more tweaks next year, although each year the changes seem to be getting smaller.
I never saw the need for modifiers.
In the league we run, it's 1 point for 1st, 2 for second, and so on.
It's the same system they use in sailing.
Generally, it's the best teams that show up every time, and the less experienced ones that show up occasionally.
So, to me, a 1st is just as hard with 8 teams, as it is with 20.... because it's not team 9 to 20 you are competing with anyway.
- if you need modifiers becuse different regions have different quality, insert that season to season, from the get go.... so that 1 region gets 2 qualifiers, another 4, etc.
They do that for the soccer world cup, where a LOT more european teams qualify, than asian ones, because there is much tougher competition in europe.
It also allows the organisation more control, and you don't have to bother with complicated points systems.
As I said, a critique, not a criticism--and I would agree the system is the best its ever been.
And as in the past my primary interest is in seeing that the rules produce consistently competitive divisions without unnecessarily pushing players up the ladder--and the current classification rules are a great improvement over the original ones. :)
No worries, was mainly just trying to provide some additional perspective on how things worked and why.
I love it when Baca and Raehl get along. The birds seem to sing a little louder, the grass is a little greener, that trees grow taller......harmony in the universe. ;)
Reiner
Thanks. I threw up in my mouth a little.
Nick, the class sailing i've done first place has been .75 points, and then 2nd is 2 points, and on down the line. anyway, minor detail, giving the first place sailor a slight advantage out of a series.
anyway, continuing to tweak the appa system out is a good thing. Taking the data, being forthright with the players by informing them that this is an ongoing effort to have the best system hands down (which in my opinion - it already is, by far, but nice to know more improvements are working out)
So Dan:
Do you think the sailing system is better - or that sailing could be improved by adopting the system we have in paintball?
Hey Nick,
i've often thought about the difference between first and last place in paintball and the spread between 100 points and 10 when you only have (like in some regional tourneys only 4 or 5 teams in a division) However, in sailing, and I do have tons of sailing racing regatta experience, the objective is to have the fewest points and at .75 the "winner" has a slight advantage but not a huge gulf of point spread between boats, even if there is only 5 boats.
I think Tampa Bay Damage in 2011 is a great example, two wins, a third place in Galveston, but one event in Chicago, only capturing 40 points just put them out of the Series Title.
But let me put out this caveat please, with the pros with 10 teams, i think it works out okay because the point spread be it .75 down to 10 percentage wise BASICALLY works out the same (however, if this system was used, Dynasty still took the series champs and Damage takes second and third would be Russians)
NOW, after all that being said, I was looking at teams for series (treating them like boats) and not the individual player stats/standings because that doesn't exist in yacht racing.
The match racing I've done, typically your worst finish is discarded and because there are (just like in paintball) a different amount of boats competing it makes it more challenging to finish at top - and i do like the new revisions by APPA with regards to calculating player "points" based on number of teams.
All that being said, i actually like the NEW appa system because it does divide out the players points based on team attendance and i think they need to do it for series points for teams too.
You hit the nail right on its head, when talking about how 1 poor finish can put you out of contention.
That's what I dislike the most about the current system.
I do not think, with only 4-5 events in a year, that you should be able to discard your poorest result... but I do think we need a lesser points spread between first and last place.
Post a Comment