Monday, April 30, 2012

The Classification Curse

Regulars may recall that in posts past VFTD has had issues with player classification as defined and cataloged by the only real (functional & utilitarian) data base service of this sort in the paintball universe; APPA. Which isn't to say that APPA is bad. It isn't. Numerous changes made in recent years have improved it greatly. And in the grand scheme of things competitive in paintball it serves a useful purpose. (If you're a glutton for punishment follow this link.) But an important problem remains and it is to (once again) highlight that problem I'm writing this post. (That, and I was reminded of it yesterday when I spent a free--from TBD commitments--Sunday hanging out with some old paintball buddies and some new tourney players. For some additional background look here.)
In the Spring of 2010 a number of my former teammates decided to play a local event for fun--and to see if the game reignited old passions. Pretty much across the board it did--reignite old passions--and within weeks they had bought new gear, planned a practice schedule and decided to play as many of the rest of the season's events as possible.
Next week they'll be competing in the third event of their second CFPS season. Two former teams have been reconstructed and a brand new team of mostly young players has been formed as a result of ex-players rediscovering the tourney game they loved a decade or more ago.
The problem is APPA. (Sorta.) Some of the first posts on VFTD le blog in 2008 dealt with my concern that the classification rules in force then had the unintended consequence of pushing D1 ranked players out of tourney paintball. (If you're curious--and you know what curiosity did to the cat--start looking in the July 2008 archives.) And here we are again except the situation is happening on the local level. As it turns out my former teammates, the old guys, have had some success and it appears that by the end of the season they will be bumped up in classification by APPA and out of local competition. Theoretically the CFPS could offer D3X Race 2-4 but have had difficulty fielding a division in the past and if next year is like this one the old guys will be out of tourney paintball again. Okay, but they could play PSP. Or maybe a different regional series, right? Well, yes, at least by classification rule they could but they are all older with families and jobs and real responsibilities and never intended or wanted to travel around the country to events or rise through the ranks. They are regular guys who enjoy playing competitive paintball for fun--not to rejoin the grind--and once again the unintended consequence of well-intentioned and useful rules is gonna push players out of the game. And this time it will be exactly the kind of players the game needs at the grassroots level to provide leadership, resources and knowledge to help grow the next generation of tournament players.

34 comments:

nickgibson said...

I play the cfps series and would hate to see voodoo get bumped out of the series.

Anonymous said...

I've argued this point to Chris many times.

But, one could not even convince this man his name is "Chris".

Why do we have these standards when/if they do not reflect the true market?

By all technical means, I'm a rookie "0"/"N/A". In reality, I and those who are familiar with your favorite Anon, should never be placed below a D1 ranking- it just wouldn't be fair. "We" haven't played a tournament in ages, but we're no "less good" and yet all of us (some who played Semi/D1 on a regular basis) could walk into a tournament and "own" as the kids say based solely on our APPA's- and it would be legal.

I could sandbag- but what's the point in playing for cheap prizes? Or playing less talented or skilled players? None of those benchmark tag lines the APPA has been designed to "protect" against apply in reality. I'm not going to spend my money on playing less experienced players so the concept is null. I'm not alone- I've seen others argue this too. Especially considering the whole sponsorship "standard" we see.

I think in general, the whole idea of a system which "stops" "sandbagging" not only creates an artifical problem, but negatively affects the tournament market as a whole. It is no doubt a self-sustaining, self-fufilling prophecy.

But let's hand it to Chris- What better way to make money?

Then again, this is coming from a man who would rather "RaceTo" some imaginary answer then let a game play out...

Listen, I get that stats allude to specific information, but when one uses that data in recourse to determine the game, where does it end?

Anonymous said...

I'd really like to see them fold a few divisions together. At least put 3-4 together. Your pretty much d3 at the end of the season if you podium 2 events and don't play anything else (depending on the number of teams in the event ect) I've seen many players get bumped the d3 that were barely cutting it at 4 and it makes me question the point.

Anonymous said...

I think they should fold all divisions below D2 at the PSP. They can hold a D3 event but the points earned go towards your local series. If your not ranked D3 yet according to APPA your intelligible to compete. Same goes towards D2. This would put less emphasis for teams to be pushed to compete nationally which burns most players out in one to two seasons. It would also put more emphasis on the regional leagues which is far more affordable and sustainable for most players to continue competing.

Dave Painter said...

Let me start by saying I believe in what Chris is doing. It’s heading in the right direction. Ranking players can’t be left up to a committee to decide each person’s ranking. The best method for doing so is by taking the results of events and applying points and creating rankings. This the APPA does very well.

The issue the first Anon points out is simply due to limited/no data in the system for that particular player. Unfortunately everything has to start somewhere and when you start out with no data you get those issues. If he wants to play lower divisions and “own” the competition he can, but the system will catch up with him and he will get bumped up the ranks as he gathers 1st place trophies in lower divisions. Instead of complaining you could always go out and create a better solution, do you have a better solution? Should we just let your friends decide what your rank is?

As for the issue Baca points out – I agree. And have been talking with Chris about this issue as well. I’ve always felt that Local tournaments should have less importance (points) then a regional event and the regional events should have less points then national events. Chris/APPA has continued to make changes/improvements to the system to address this and other concerns. There is a weighting on the points earned based upon the number of teams in a division. This starts to answer the Local vs Regional vs National concern. However, as a local event producer I don’t want to eliminate anyone from playing if at all possible. One solution I see would be to offer say a D3 5-man event but award D4 points (at a local level) – this would allow D3 players to continue to play locally unless they played regionally or nationally at a D3 level and earned enough points to “graduate” to the next level. For my x-ball league I’ve simply made an open division which has no roster limitations at all, unfortunately since I run a season, not individual events, the APPA system doesn’t work well for me and I have elected to not use it for that division.

I feel Chris and team have always looked to improve the system year over year. While the changes may seem slow to some I believe the idea is to make sure the corrections don’t go too far and cause other problems. Slow and steady wins the race.

Dave Painter said...

I don't claim to know the finances of the national leagues being run today, but wouldn't "folding" all divisions below D2 cause a rather large reduction in Revenue for the leagues. I have to believe that to some degree it is the lower division numbers that are helping pay for the overall event. How would the leagues replace that revenue? Would they be able to remain afloat if they dropped those divisions?

Anonymous said...

baca's team sounds like he teams that were actually participating in Am open or World Cup events circe 2000. They had money, and played for fun, and for the experience of a big event. They entered whatever division they thought they could hang in. I miss teams like that.

MikeMfromPrime said...

As long as there is a discernible and impactful difference in skill level between divisions, they should be separated. The difference between D4 and D3 teams is apparent. The difference in use (or purchase) of skill players in D1 is one noticeable from D2. D5 is "true beginner" level. Teams want a reasonable expectation of success.

I've been told the real problem is that it takes at least 3 seasons to evaluate the impact of rule changes. How many series are there using APPA compared to 5 years or even 3 years ago?

The sea change in recent years is going from a mere # of tournaments played, to a performance-based rating. Going off Mr. Painter's suggestions it sounds like it should be performance within a certain "grade" of series. Maybe even a classification system for different series built in.

Sandbagging was a serious problem is the early 2000s. I don't think it's a remotely artificial problem. But I'd listen to an argument that says they're overcompensating and why.

At all levels, For starters, 5man should never count as much as Xball format. Budget, number of players, practice-time, staff, all reasons why.

The question is, does allowing those higher ranked players that win local events to play discourage new players/teams from joining/continuing? Just as much as winning is fun, losing is not.

I think these teams that can't keep playing are more victimized by a lack of team generation/economic reasons (and their own success) than the ranking system. A healthy pb setting should at least have a D3 outlet of play within driving distance. I don't know how the system can compensate for such an environment. Random draft by D4 teams of D1/2/3 players?

Also, if the scene is local enough and the effects so negative, they always have the option to not use the system. It's a product.

Anonymous said...

Being ranked D2 leaves you will only 2 choices:

1. Get on an airplane and play PSP.

2. Break up your team and have the D2 players drop-in on other local D3 teams (if your market is lucky enough to provide a D3 bracket).

Raehl is blind to this as he can't imagine these situations from a players perspective. He (and the other powers-that-be) continue to build from the top down and it is KILLING competition ball.

Reiner Schafer said...

Last anon, you've summed up the problem, what would be your solution? Or what should the APPA's solution be?

Nick Brockdorff said...

The whole notion of ranking players, in a team sport, is silly.... and going about the real problem ass backwards.

The real problem in paintball, was always that paintballers are fickle and jump ship, often times several times a season.

The simple - and best - solution would be to just put a limit to how many teams a player can play for in a given timeframe (I would make that limit "1").

Granted, that would then hurt the players that play for one team locally and another nationally, but really, it should be up to the local tournament organiser, if he wants to have an "open division" at his event, which would make it possible for anyone to play.

You don't see this whole individual ranking in other team sports - at least not to an extent, where it determines at what level a particular individual can play.

- and really, if Fedorov wants to go play for a Div. 2 team, instead of a Pro team, why shouldn't he be able to?

raehl said...

I'm not blind to that at all.

MikeFromPrime hit the issue. We have two choices:

1) Give a team that has won a division this year the option of moving up or splitting on to different teams in the same division next year.

2) Let the team stay together and keep winning in the same division next year.

If we choose 1), some players on the team may not want to play if they can't keep winning the same division with the same team.

If we choose 2), some teams in the division may not want to play if they have to keep losing to the same team that already won the division.

If the concern is "losing players", then option 1 is the better option.

If anyone has an Option 3, I'm all ears.

As for Baca's particular case, if he's talking about a D4 RT2 team, I would guess CFPS would be able to support a D3 RT2 division next year. If he's talking about a D3 team, I would suggest that a team winning D3 is not just a buncha guys playing for fun between work and family time.

Dave Painter said...

Nick,

I imagine the reason why it doesn't happen in other sports is because as you move up the ladder in other sports the rewards are significantly better. There is limited/no value in sandbagging and staying in club ball when you can play pro.

Why play local club ball for peanuts a year when you're good enough to play pro and earn 100's of Thousand (or even millions). The reward drives the players behavior.

In our choosen sport there is not that huge delta in reward between D2 and PRO. I know my guys would prefer to play pro because they want to be the best and don't care about the reward. However, not everyone sees it that way.

Baca Loco said...

Nick
While I am sympathetic to elements of your argument there are clear issues involved like competitive balance and the expectations of competing teams as to what constitutes a fair competitive environment.

Chris
As usual your blinders are 20/20, providing an excellent view of only that which you choose to see. :)

As to the specific problem in the post, yes, a D3 RT2 option is the only one--within the current system--that might work.
As to the old guys themselves only 2 of them are under 30 and both them are closer to 30 than 20 and at least two others are over 40. Btw, having fun doesn't preclude the desire to win.

You have only two issues within the system you created! That is NOT the sum of all options available.
A perfect viable option 3 is stay out of the system. :) Or take from it only that which is useful--except there's that pesky Big Picture plan.

MikeMfromPrime said...

Other sports are also based on purely physical attributes and therefore, by age. Or by weight class. PB is a mental game. You wouldn't separate chess players based on age.

Out of curiosity I breezed through the player ranks for the US Chess League. At numbers 134 and 141 players were 16 and 14(!) years old. Their classification rules are also strikingly similar. 10 individuals per team. Cap on allowed performance points total per team. Plus higher ranked players have to play others in the higher slot. Those higher "boards" count more than others. http://www.uschessleague.com/LeagueLeaders.html

Anonymous said...

Outgrowing the local competition is not a paintball specific thing, it happens in other sports too. Even if there is no ranking system to slot teams, it would get old for a strong team to dominate local competition constantly.

To Baca's OP, is the real problem getting ranked-up/bumped-out of the local scene, or is the problem that the local scene is not robust enough to have enough/any teams at higher levels?

Baca Loco said...

Mike,
That explains alot doesn't it? ;)

Anon
That is a fair question and I think a fair answer is thatn in the current environment is doesn't appear robust enough--even with the increases made the last year or two.

The question doesn't however deal with the very real issue that there are lots of former competitive paintball players, some percentage of whom, would very likely be interested in playing again.

Anonymous said...

"The question doesn't however deal with the very real issue that there are lots of former competitive paintball players, some percentage of whom, would very likely be interested in playing again."

For the sake of argument, say the you get rid of APPA ranking and let teams self-select. These former competitive guys come in and destroy the competition. Then what? They are better than the local talent and they aren't willing to travel to play better competition. Do they just keep beating up on the overmatched local teams tourney after tourney? How long would they have fun doing that?

I think it really boils down to local talent pool if the team is limiting itself to local play. If there are other similarly skilled teams in the area, things will work out fine.

Maybe being forced by APPA rankings to split up would be best for the local scene. They could provide leadership and experience to other teams to help grow them into higher ranked teams.

Baca Loco said...

Gee Anon, it's curious how you and Raehl share some habits. Like making bald assertions and creating hypotheticals that seem to suit the answers you're looking for. :)
But that's okay.
You do realize tournament paintball existed prior to APPA, right? And that APPA isn't a panacea for all paintball's ills, right? (Note to Chris: apparently APPA doesn't protect overmatched local teams from being beaten up either. See MAPL.)
I think it boils down to local promoters deciding what's best for their area. What APPA and the UPC (and to an extent the PSP) are looking to do is build a top to bottom competitive paintball structure from the top down.
The fact that it causes dislocutions isn't a reason to trash it but the fact it causes dislocutions isn't a reason to ignore them either.

Yes, that's what a team of friends want--a system that tells them if they get too good they have to find other people to play with. Now that is a raehl solution!

Nick Brockdorff said...

Mike:

I can't think of a single team sport that ranks players by physical attributes?

- chess btw is not a team sport, they compete individually, and the scores are then compiled into an atriticial team system... it's wholly different from our sport.

Also.

Nobody is telling a soccer player he can't play on the 2nd team, because he scores too many goals.

Nobody is telling a hockey goalie in a farm club, he can't play there, because he had too many saves

Or, for that matter, nobody is telling a pitcher on a beerleague softball team, that he can't pitch anymore, because he is getting too many strike outs.

A LOT of sports though, have team relegation and promotion, to ensure a particular division is not perpetually dominated by the same team.

However, that is a fairly alien concept in american sports, I know, so would probably be hard to institute in the PSP and affiliate leagues.

Dave Painter is right though, our sport is curious, in the lack of reward until - at least - Pro level, and as such, I would say the top division at local/regional level, would always have to be an open one, so that people are not forced out of the sport for financial reasons.

I still find the whole individual ranking thing odd, and would much prefer a team ranking system, with promotion and relegation.

Not least because it is the healthiest for the sport, it gives small teams a valid reason for ambition, that today is solely reliant on finances and connections - anyone can play Pro today, if they are prepared to pay for it.

A promotion/relegation system, will mean that the finances automatically follows skill, so that the higher rank you are, the better support you will attract.

Dave Painter said...

Nick,

No one has to tell that goalie or soccer player to move on to the next level because when they're offered a spot at the next level it comes with significant rewards. The beer league pitcher is an interesting one and happens all the time. The same team wins the league year after year - but the other teams keep showing up - because no one is expecting to win money, it doesn't really cost that much to play and lets face it - everyone is there for the beer (I mean love of the game).

As for the comment "in the lack of reward until - at least - Pro level" - I'd disagree with you. Specifically regarding upTon 187 cRew - our overall sponsorship hasn't changed materially from last year to this year. With the exception of the fact that we most likely won't win 2 out of 4 events (as we did last year) and earn more reward from our sponsors.

I believe the league (at the pro level) is split into two groups - teams with large budgets and teams with small budgets. Look at Damage - they are playing all three "Pro" series' this year - PSP, NPPL and Millennium - I can tell you that we can barely afford to play the PSP. I can't speak for other teams, but I assume there are at least a few others in our shoes too.

I'm not complaining - just pointing out the facts. We're new and need to earn it. Just because we got here doesn't mean we're swimming in cash now.

Nick Brockdorff said...

That's why it said "at least" - because "Pro" is a very different thing, whether you are on a factory (or russian banker) owned team, or at least the factory's main team - or just a team of guys playing Pro.

I pretty much know where that division is, and especially the year to year struggle to become a factory's main team, is very interesting for me to follow... and also how smaller/newer factories, live and die with the success or failure of their main team.

For instance, I have the utmost respect for Planet Eclipse, but I also believe, irrespective of the quality of their workmanship, that the first Ego coinciding with XSVs rise to power, was very mutually beneficial, and something Eclipse has been very skilled at building on since then.

Either way, I still think it's an unantural and silly way to manage the level of play in sub-Pro divisions, to rank individuals, rather than teams.... and I think the sport could benefit immensely, with all the major leagues operating under a promotion/relegation system.

Anonymous said...

@Baca,

Say your team of friends is allowed to play down to the top locally available division: would they dominate it or simply be pretty good in it?

If they would dominate, that would get old pretty quick. If they would just be pretty good, then that would mean the APPA rankings are inflated.

Baca Loco said...

Anon
I have no idea. Last year when they got back together they were pretty bad. They have improved event to event. Their concern is that if they maintain or continue to improve they will end up classified out of a place to play.

There is a separate issue with regards the APPA ranking. One I will post about tomorrow.

Anonymous said...

Please forgive my ignorance, seeing as how this is my very first tournament (CFPS Event 3). I do think understand the theory behind this discussion. Why doesn't the APPA rank "the teams" as a whole based on winning % and points earned from wins / places, Then rank the players individually based on their particular set of skills / accomplishments (whatever standards those may be ie; kills, % of gun fights won, etc)that would have to be decided upon by the powers that be. This is directed back at the goalie / pitcher analogy which referred to the their individual accomplishments (stike outs / saves) They have to have a supporting cast or they will not (normally) dominate a league. This way if you have a superstar or two on a team they can play against a team of average guys that gel well together and both teams can be competitive (one or two superstars doesnt necessarily make a winning team). If the team as a whole reaches a specific level based in player skill rankings and team wins / places then they make the choice to stay together as a team and move up the rankings or those players that have higher skill rankings move from that team to a higher ranked team and would probably be welcomed with open arms because of what they bring to the table. Now if they wanted to stay at their current level then allow them to, but limit the number of high ranked players that can play on a given team. This would put an element of choice back on the teams and players as to what level / division they end up playing. Hope this makes some sense. Thanks

Baca Loco said...

Last Anon
In one sense what you suggest kinda happens within roster rules--who is allowed to play on what level team, etc.
In another sense it is impossible for APPA or any other system to do more than register who played what event and what the team result was. Attempting to evaulaute the contribution of individual players simply isn't possible.

Anonymous said...

Baca,
Is that because of the lack of irrefutable evidence?

Basiclly meaning no video of the games. I didn't mean for a player to be arbitrarily evaluated, but rather refer back to something like video of the game for hard evidence as to number of total kills, run throughs, one on one gunfight wins. You know the kind of stuff players find exciting that either happened or didn't and the video would prove it. I know that it would take time and energy to make something like that happen. Would it help? Maybe / maybe not. It's just a suggestion.

Thanks for lending the Noob an ear.

Baca Loco said...

No, it's because of the manpower & expense issue. The PSP is spending quite a lot of money just to hire guys to watch the Pro field in order to try and keep track of basic stats. Imagine how unworkable it would be for APPA to try and man and staff all the events that use their ID database in order to collect stats on everybody.
And even if that hurdle were overcome then you'd have o decide if the stats you're collecting mean what you think they mean.

Ed said...

I'm on the team that Baca mentioned in this post. Many of us had stopped playing for 7+ years before we decided to play tournaments againin Feb. 2011. Most of us started out with 0 points since we played before the APPA ranking system. In just over a year, we are on the verge of becing ranked D3. When that happens, we won't have a place to play next year since the CFPS doesn't have as many D3 teams entering. Our only choice would be to play national events, which is out of the question for most of us. Frankly, our wives would revolt.

And I would think we are just the type of players that paintball needs. For example, we started a D5 squad of mostly teenagers just so we can teach our brand of paintball and pass on the brotherhood we have shared through paintball. We also have 2 father-son combinations on our team. Bet not many teams can say that.

At our core, we are just a bunch of old-school players who love playing tourney ball with our friends. We love this sport and would love to continue playing local tournaments, but that won't be likely next year because of the ranking. Most of us were surprised how fast our points started adding up.

If you want to know more about us, check us out at http://www.facebook.com/floridapaintball

Nick Brockdorff said...

So, wouldn't your problem be solved by local events having an open top division?

Seems to me it is going about it backwards, to want to change the national ranking system (which is flawed for other reasons), to solve a local problem?

Sounds like this "problem" lands squarely in the lap of the local tournament organiser.

Ed said...

Maybe so.. I don't pretend to know the answer. I'm not nearly has enlightened about the ins and outs of the ranking system as Baca and the rest..

We were just surpised at how fast we went from having 0 points to becoming D3 ranked players and wondering how we could keep playing local tournaments next year if we get bumped up.

We could have really sandbagged and played D5 since most of us had no points, but we didn't feel right about doing that. So we entered as a D4 squad. In less than a year, we suddenly found ourselves on the verge of being D3 and without any real option of playing at the local level any more. It just seems counter-productive to the overall goal of continuing to build the sport.

Ed said...

Also -- and this is for another post I'm sure -- but the cost of playing a national event is just prohibitive.

Even if we had a few extra bucks to pay for the event, try convincing our wives that it's okay to spend $1,000 or more to play one tournament. We'd all find ourselves divorced before the end of the season...

TJ said...

An open division is not a solution for anything. It's just asking for abuse.

Dan said...

TJ, we use an open/rookie division here in Michigan to great effect. Podium 3 rookies, and your no longer new, time to move to open. We also don't use APPA. for all of those problems listed...

. national events should count for more than a local event, and that five man should could less than xball. And that getting moved up shouldn't be equal across the divisions. Each subsequent promotion needs to be harder to obtain.
Personally I'd suggest tha the thresholds stay roughly the same, except that d2, d1 both need to be increased.

Then change what each winning player earns per type of event.
On the national level
D3 xball 400
D3 5man 300
D4 xball 200
D4 5man 150

Locally (even the feeder series)
D3 xball 300
D3 5man 200
D4 xball 175
D4 5man 125
Winning locally can't out rank you for the future the event, but one really good national with locals can.

There is no reason that a winning d3 5man team should be required to play d2 xball.