All of a Sudden, We're All Talking CSM

I don't talk about the CSM that often. It's not out of any specifically strong feelings. In fact, I've recommended that folks vote for CSM in the past, and mentioned how important I thought it was. But I typically don't follow their exploits with bated breath, or even read CSM summary posts by other bloggers.

Part of that is my firm believe that we as a society apply democracy a bit too liberally. Not everything needs or deserves a vote. Sure, I think it's the finest system in the world for organizing a government for citizens who are location-locked into a specific country. The key, though, is that everyone involved in that country and in that democratic system has equal amounts vested into the success of the enterprise, and equal amounts to lose if they get it wrong. Democracy needs seriousness and a deep familiarity and awareness of the issues, as well as no readily available and easy-to-access escape plan.

That said, it's a terrible idea for a lot of other things. While you may take your kids' feelings into account, you make decisions about them bilaterally (or unilaterally in single-parent homes). They don't get a vote, because they don't have the context and knowledge necessary to make an educated decision. Nor do you give airplane passengers a vote on the route the pilot should take.

When all parties aren't equally vested, you also don't utilize democracy. My mother wants to redesign her living room. I used to live in that house, and have somewhat of an interest in keeping it looking good. I also want her to eventually move out to where my family is located now, and I want to keep the house in good repair for them to sell it at a profit. But while I am slightly vested in how the living room looks, she's the one who owns the house, and she's living in it day-in and day-out. Her level of vestment is much higher than mine. It'd be ludicrous of her to give me an equal democratic vote with her in how the living room looks. Nor am I going to make career decisions because two of my friends - outnumbering me - feel I should.

Then, there's the "bail option". When you're tied to the success or failure of an endeavor, you'll take it much more seriously than when you're passing through. I shouldn't have an equal vote as a hotel owner about how their room looks. I have to live there for one night; that hotel owner's very livelihood depends upon making design decisions that ensure profitability for decades, potentially. She's not going to let my wife and I out-vote her about how her hotel is designed.

Along the same lines, democracy is a ludicrous method for a gaming company to choose a trusted user group.

This isn't a consistent belief; it's one I've come to gradually over time. part of it rests on my own experiences with focus groups and user groups in my career. When I'm working on a focus group, my goal is to get a variety of opinions that accurately reflects the landscape of user experiences. I don't want to get the loudest voices. I don't want to get the most popular or personable voices. I specifically want diversity and those hard-to-interact-with customers who I won't hear from normally. I don't want clients I'm speaking to on a daily basis; they're going to make their opinions known already.

I may not want the same group of clients to talk to me about their opinions on all aspects of my business.  At work, we have a couple different products we offer, which appeal to very different audiences. One group isn't enough. I want a large, varied pool I can pick and choose from according to my needs.

We've recently heard about a lot of drama and animosity between the CSM and CCP. I say "recently" because it really hasn't gotten much press, despite multiple members being thrown off, leaks of critical information covered by the NDA at multiple points, and serious issues of trust. Leaking NDA-covered information is most certainly the fault of the guilty CSM members, and represents both a breach of agreement and poor faith. That certainly damages the role of CSM member.

But the primary fault, ultimately, lies with CCP. The CSM was a reaction to a particularly asinine set of decisions that demonstrated tone-deafness about customer preferences and desires. And at the time, it was a big, bold symbol of the new CCP. It served its purpose to bring players further into the fold. But let's consider... is the CSM set up for success?

Consider: players are elected from among the player base, who fail a few of the key requirements of a successful democracy. They can leave and find another game at any time. They aren't equally vested as CCP employees are devs are; their livelihood isn't on the line. And they have varying degrees of knowledge about the game. Sure, some players are deeply knowledgeable, but a vast majority know a few names, and nothing more. Those players who ARE knowledgeable are likely heavily biased towards one area of play or another. None of that bodes well for the selection of the best, more knowledgeable CSM members who have their finger on the pulse of the game and the greatest aptitude for predicting player reactions and sharing varied and well-reasoned feedback. It doesn't even encourage a variety of viewpoints, only the selection of members who come from the loudest parts of the game; in its worst moments, this process results in famous folk being selected, which is almost universally bad (Sugar Kyle and Ripard Teg probably being the exceptions).

Moreover, look at the "platforms" of players who run for CSM. They all talk about wanting to DO things; to see certain changes take place. Year after year, this is the case, despite the fact that the CSM are player advisers meant to provide a reality check on how players will react to CCP decisions. The CSM is not tasked with coming up with plans or solutions; that's CCP's job. And they certainly don't exert influence on the agenda. They're meant to share the player perspective.

A democratic process for electing the CSM is misguided and doomed to result in failure, because it sets expectations and follows models - ie. the political process of many countries - that are wholly, fundamentally divergent from the objectives of the CSM. It's a mismatch of method and purpose.

CCP is clearly seeing the mistakes it made in the format and structure of the CSM, as well.  CCP employees have taken to Twitter actively encouraging players to circumvent the CSM. And devs have been openly hostile to players by taking feedback given via non-controlled media (ie. reddit, in particular) personally and lashing out. The cracks are showing with the relationship between CCP and its players.

Let me take an aside here to briefly talk about some of the feedback devs have given players. In particular, I've read reddit posts where devs express offense at the work they do not being appreciated and being responded to with poor manners and hostility. To those devs, I must say: you wanted to force discomfort and change on your players, and you need to be prepared to take your lumps in the process. Ultimately, we are your customers - not just players - and if your work isn't being appreciated by us, it's your fault, either for poor decisions, not clearly communicating the value, or by ignoring your customers' desires. You providing a product does not entitle you to our adulation; only our money if we choose to participate in it.

That's a harsh stance, but CCP isn't getting anything players don't dish out to each other. They've curated a culture - through its policies allowing players to be the villains to each other - that is very engaging and raw to players. When players are angry with each other, they hit each other with both barrels. Sometimes, they go too far and give offense. CCP can hardly protest when that same behavior is directed at them, particularly when they've engaged upon a drastic series of disruptive, painful changes for players.

And make no mistake, the changes CCP is making ARE painful. They're meant to be uncomfortable, to shake up the game and force players to change their thinking. Jump fatigue, revised sov mechanics, endless new abilities of ships, and a host of other changes are all making players feel uncomfortable. The pan on input broadcasting via ISBoxer created more friction. CCP almost certainly expected all of these changes to mkae players uncomfortable, and even angry. To now be offended because players who have always expressed their frustration in impolite ways release this frustration at you is disingenuous in the extreme. Berating your customers isn't the way to do it (in full disclosure, some devs have taken very thoughtful, respectful ways of addressing the hostility along the lines of, "We value your feedback, but please consider the effect your tone may have on the people who are sacrificing time with their families to improve the game." Well done, there.)

Anyways... back to the CSM, which was doomed to be a popularity contest with members making promises they couldn't hope to keep, from a too-small pool of members to fully represent a variety of informed opinions about all aspects of the game. In the process, they turned CSM members into personalities and firebrands, whereas they should be beta testers at best. That's a lesson I think CCP is now learning; they set the CSM up for failure - failure to meet player expectations and failure to provide the insight they really need.

I'd also argue that CCP is bringing the CSM in far, far too downstream to be effective. Right now, CCP has a vision of where they want to go, and they present it - well-baked - to the CSM for their feedback on "do you like it", effectively. While this gives a clear impression of what CSM members think about this specific plan, it "poisons the well" and destroys any chance CCP may have to understand what players want, what players are experiencing, and what players' challenges are.

When you put together a focus group, you want to ask general questions about their experiences, challenges, and perceptions first, before you present any possible solutions to them. If you do it in the wrong order, everything that follows will be tainted by the solution you propose. Folks will respond either for or against it. You effectively override their opinions with their reactions.

Right now, when a new CSM is elected (ugh...), they have access to the work done by the previous CSM. Some initiatives are continued, some new ones are introduced. They have lots of reading to do of "what has gone on before". All of this is nice if your CSM is intended to develop solutions themselves, but it torpedoes any effort to solicit their honest feedback. The well is spoiled from well before they have their first meeting; they're cooking with tainted flour. No wonder why everyone's getting sick. You call focus groups freshly each time you have a question for a reason; each time, you can set the table again and start fresh. You can't do that with the modern CSM.

Whether the capital rebalance focus group was a fall-back strategy because of a poor CSM relationship or an unrelated, new approach to soliciting customer feedback brought in by new employees, I can't say. But, I think focus groups are a better way to go. They tend to ask questions about experience, not present solutions for feedback. They pose possibilities, not serve as a confirmation method for a decision already made. Focus groups can inject the customer-focus that the CSM process - with members being given in-development solutions far downstream in the process - will never be able to provide.

The only change I'd make to the focus group process is to keep the names of the participants and the results of the focus group silent. What is gained by publishing this information? If the changes that come out of the focus group are terrible, they'll be held up for ridicule. If they're wonderful, no one will notice. CCP's obligation is to ensure that a variety of knowledgeable viewpoints are represented; they aren't responsible for the results of that focus group (beyond structuring question as to not lead participants). Is it announced for some sort of escape clause?  "Well, the focus group members wanted this, so it's not our fault that it was terrible," will do nothing to deflect anger in case of a bad rebalance. But publishing the names or affixing the names to specific comments will result in less honest opinions, and that can kill the value as well.

The gameworld CCP faces in 2015 is very different than the gameworld they faced in the past. They have other means of gathering player feedback, have instituted advance discussion threads in the forums that have yielded good feedback, and are beginning to utilize focus groups that get a variety of experience-relevant feedback about specific topics without "poisoning the well" as the CSM does.

The CSM isn't to blame for the failure of the CSM (in most cases, though spreading NDA info falls squarely on the shoulders of the guilty members). The fault rests with CCP for choosing to create a screwdriver when it needs binoculars. Even the best CSM member cannot provide what CCP needs, because the CSM itself isn't structured to provide it. They've become celebrities, lightning-rods, and firebrands. Some have done their very best. But, in the end, the goal is impossible because it's been inadvertently sabotaged by there very construct chosen to achieve it.

Sion put out a proposal to modify the CSM. I don't think modification is valuable, because I don't believe any number of fixed members are valuable to CCP in the way ad-hoc focus groups could be. Democracy isn't the answer; the wise and unbiased selection of competing viewpoints is. Ultimately, that falls on CCP, not us. If they fail to secure that variety, they'll be living in the same echo chamber we claim the Goon leadership is, and their game will fail because of that poor business decision. But theirs is the risk, so theirs must be the control.

We all need to be okay with it when CCP decides to do away with the CSM. I'm curious to see where these focus groups go from here, and whether CCP's new customer experience executive can clean up some of the victimism among some of the CCP devs.

Let's judge them on their overall approach to soliciting customer opinions, not the specific method in which they do it.