17 August

Customer Support: Past, Present and Future

It's been around a year since EVE Online’s Customer Support Team last published a dev blog and so we felt it was time for an update. This blog is all about Customer Support and as such it will not include any game content or release information. With that being said let's talk about tickets. 

Before we talk about ticket reply times, queue status and address some of your feedback, we want to highlight some of the events since last we spoke.

New Ticket System

Last June we went live with a new ticket system, a much needed improvement on the old system which had been in use since EVE launched back in 2003. We got to play with lots of new tools which help track ticket submissions metrics, measure reply time averages (more on this later) and lots of other fascinating stuff which quite probably only the Lead Game Masters find interesting but which ultimately benefit you, the players.

Perhaps the most obvious benefit to players of the new system was the Help Center which contains around 190 articles and gets over 5000 views per day. We love communicating directly with you all, but I think you would all agree that it is preferable if a player can have his question answered without the need to submit a ticket.

CSM X Summit

For the second year in a row we scheduled a session to meet with the CSM during their summit here in Reykjavik.  One of the major concerns they raised was transparency with regards to what is and what is not an exploit. Because of this feedback, the Known and Declared Exploits article was produced. It's one of the most "liked" articles and we aim to keep it updated as much as possible. We're looking forward to meeting with CSM XI in September.

Player Submitted Art

Last March the Operations and IT team expanded their territory and erected a new wall between themselves and Customer Support team. The bittersweet moment of the final part of the wall going up left us with a sense of loss (<3 the ops team) but also a whole lot of blank wall to cover. Despite our best efforts to decorate the wall it was decided that a couple of cat meme pics just weren't going to be good enough. So we asked you, the players, to create some art for us to hang on the walls. You responded admirably and the winning art was printed out and now proudly hangs for all Game Masters (GMs) and visitors to see. 


As well as the above highlights for the department we must acknowledge that these past 6 months have had their difficulties, and to be honest the support we have provided for the last few months has not been ideal.  Twice in the past six months, mostly due to issues outside of our control, our ticket queue has gotten so high that we made the difficult decision to send out mass replies to the majority of tickets in the queue asking players to update their tickets if their issue persisted.  We were able to reduce the queues by about 70% using this emergency measure but it’s not exactly great support and we fully acknowledge that. However, as of writing, our level of support is high, our reply times low, and we aim to keep it that way.

In mid-June we made some changes to our email routing which resulted in replies to tickets by players not getting into our support system. New tickets were able to be submitted but any replies submitted by email to tickets did not make it into our system; this affected 1701 players. We should have detected the problem earlier but for a while it went unnoticed. Once we discovered this issue we contacted the affected players, apologized and provided a ship skin as compensation.

Perhaps it would be a good idea to give a quick overview of how the Customer Support ticket queue is setup before we go any further. Players submit tickets which are filtered into various categories Billing, Gameplay and Technical being the three main ones with each category having sub-categories. It’s not a case of all tickets being in one big queue and then handled on a first come first served basis. We can't adopt a "one big queue" approach with all GMs working out of it because different GMs have different training and specializations. Also some tickets are more time sensitive than others and they need to be addressed as soon as possible.

All tickets are important to us but we do filter some tickets into a queue in which a GM will reply to the ticket is less than an hour--usually within 15 minutes.  In this queue we handle the aforementioned tickets which are time sensitive such as offensive behavior, player harm reports (broadcast for reps), stuck characters, and some tickets which can be quickly solved with the click of a button such as agent mission issues.  

We also tend to handle most account information tickets in this queue. The reasoning behind this is that if a player wants to come back to the game and does not remember his account details then we want to be able to provide them as quickly as possible rather than have him wait for a few days during which time the urge to rejoin may fade or when they might not have time set-aside to make their grand return.

One of the most common points of feedback from players is that we need to be more transparent on how long a player can expect to wait before their ticket is replied to. We would love to be able to say to a player that they can expect a reply to their ticket in "x" hours, but this is simply not possible with how our support system works and because there are too many factors to consider when estimating reply times. 

One factor which affects reply times is how many tickets we receive. It should be simple in that the more tickets we predict to receive the greater the reply times will be. However, predicting the amount of tickets we expect to receive is not that simple. With our new system we can get a fairly good estimate of how many tickets we will receive in an average week. With EVE though there is rarely such a thing as an average week. The graph below shows daily tickets received from July 2015 to July 2016. As you can see there are frequent spikes in tickets submitted each of which would throw off any estimate of reply times we could provide to players. 

Some spikes represent expansions, which we can take into account because they are planned, but some spikes in ticket submission spikes come pretty much out of the blue. The two biggest spikes we saw in the last year came from issues which we did not expect to cause a huge number of tickets. For instance, one issue relating to players encountering a black screen when logging in was completely unexpected.


(click to enlarge)

The approach we are aiming towards is that players will receive the first reply to their ticket within 36 hours of the ticket being submitted. By aiming towards this goal we remove the need to provide what would likely be inaccurate estimates for reply times. 

Below you can see on average for July 2016 of how long a player waited before they received the first reply to their ticket. The 0-1 hours reply typically relates to those tickets which are handled in the queue mentioned earlier which deals with time sensitive or easily resolved issued. Getting the average first reply time down to 36 hours is going to be tough to achieve and it won't happen overnight but we feel that its attainable and a level of support that you deserve.



When talking about reply times with players we often get asked why we cannot simply hire more GMs when reply times get too high. EVE is a beautiful, vast and challenging game for players and GMs alike. It simply takes time to find the right GM for each position we want to fill. This is especially true for non-English language support. Its further complicated in that we would generally need a new employee to relocate to a barren, volcanic rock in the middle of the north Atlantic, or Iceland as its more commonly known. 

For other games a new GM may be fully trained after a few weeks, whereas with EVE it takes around six months to fully train a GM and expect them to operate with any degree of independence. This means that if the ticket queue suddenly spikes we can’t just hire a dozen more GMs to handle the load.  Hiring 100 extra GMs is just not viable – ultimately we are a business and we need to try to maintain a staffing level that provides the support we want to give you guys but is cost effective.

With that being said we have recently expanded our team both in Iceland and abroad. Indeed, at the time of writing this blog we expect to welcome a couple of new GMs to the team next week.

If you have visited the r/EVE subreddit over the past few months, you may have seen some threads concerning the reply times for tickets relating to account security. It's worth taking a moment to provide some insight into these cases and why they take so long to resolve.

Account security cases are perhaps the most complicated tickets that we deal with, due to this complexity they tend to need to be handled by our most experienced GMs who have years of experience. 

These cases often involve ISK and assets being laundered by other players or Real Money Trading (RMT) operatives, using all manner of tricks that need to be tracked in order to reverse damages or to take punitive measures against those involved. During this process a GM may find more hacked accounts, each of which need to be investigated. This further complicates and extends the amount of time spent per ticket.

Since the introduction of skill extractors we saw a large increase in the amount of RMT motivated cases. This increase has led to our security team, GMs, and dedicated software teams collaborating to produce specialist tools to aid GMs in identifying and resolving these cases. It took time to research, produce and implement the tools but they have since made a great impact on how quickly we process these cases. 

Unfortunately delays in developing these tools lead to long waits for replies to tickets from players with compromised accounts--delays which we failed to communicate transparently to our players at the time.  Those affected players have our sincere apologies and hopefully should have received some additional game time as compensation for the delays and lapses in communication.

We plan to do better in future, but ultimately need our players to help address these issues by asking that they make use of two factor authentication for their accounts and refrain from involvement in account sharing or funding RMT by ensuring they only purchase PLEX and other EVE services from CCP or our authorised resellers.

Earlier we mentioned that we aim to reduce our first reply time for all tickets to within 36 hours. We have some upcoming changes to Customer Support over the next few months, some of which will be visible to EVE players and some of which are internal to the department.

Most of the improvements needed to provide better support are on us to complete. But there are some things that you can do to help us out:

• Provide all the basic information you think may be useful to a GM. We regularly get tickets from players who want a reimbursement for a ship loss. But they don’t include the name of the character who lost the ship or the ship type. This means that instead of being able to investigate the loss the GM has to ask for the information, which means a delay in resolving the ticket. 
• Please don't submit duplicate tickets about the same issue. We genuinely understand that if you don’t hear back from us then the natural inclination is to ask for an update, sometimes through submitting another ticket. Submitting more than one ticket about the same issue can impact reply times for you and other players. We humbly ask that only one ticket about an issue be submitted.
• If you have an issue which affects multiple members in your corporation or alliance, then please get a representative (preferably a director or CEO) to contact us on behalf of the affected players rather than having all affected players submit a ticket. Generally, we have to speak to a director or CEO anyway when dealing with corporation or alliance issues so it just makes sense for one person to contact us.  
• Bumping tickets does not improve a ticket reply time. We just wanted to mention this because we do see players regularly send replies to their ticket with the intention to ‘bump’ it to the top of the queue. This effectively does nothing to the tickets place in the queue.

Finally, on behalf of every GM in the department, we just wanted to thank you the players for making the game what it is. We look forward to talking to you via tickets or in person at Fanfest or one of our awesome player gatherings.

Fly safe!
o7

GM Ender
Director of Customer Support 

8 August

PATCH NOTES AVAILABLE FOR 118.7 AUGUST RELEASE!

The 118.7 release will bring contract functionality to citadels, as well as iterations on the visual damage system, PBR for in space assets, improvements to graphics performance in areas that have clouds and more intuitive UI feedback for ship holds.

Check out more information on what's coming on the EVE Updates website and in the patch notes for 118.7 which can be found here.

Information on any additional downtime that might be required for this release will be available in the coming days.