The Dublin Diaries

On the morning of March 18, Andrew Jon, and Hussain took flight for Dublin, Ireland in order to to present DVation's project idea to the people at SAP’s Service and Support Center.  We arrived in Dublin on March 19th and spent 4 days roaming the city.  From the surrounding suburbs to the city center we explored parks like Phoenix Park and St. Stephen’s Green, tasty brunch spots like Brother Hubbard, and pubs like Temple Bar and Fitzsimons.  We also sipped some coffee from a few cozy cafes and did some reading and work.

After the few days of exploring Dublin, we were ready to get down to business and present to the folks at SAP.  Our two hosts, Chris and Manish, kindly put together a great agenda for the day.  First, Chris met us around 9:45am, took us to coffee, and prepped us for the day ahead.  Then we headed to our conference room for the day where we gave our presentation at 10:15am.  The presentation went extremely well and by the end we had the folks at SAP jokingly making offers about investing in our company.  Additionally, we received some great questions about the product that we had yet to consider.  For example, one person asked “how does our product work handle the case of a globalized company? Will it cater to companies that extend across time zones and work in multiple languages.”  Another person alerted us to the fact that “displaying a worker score may have negative effects if the score is visible to all, but the score would is extremely useful from a managers perspective.”  We definitely plan to keep the things we learned in this meeting in mind as we continue to grow our product throughout the next quarter.

For part 2 of the day, Chris and Manish, had set up a lunch for us and meetings with folks who work with trouble tickets daily.  These meetings really helped us understand the lifecycle of a ticket.  Because of the NDA we signed, we cannot really expose any of the details specific to SAP that were mentioned but we can say these meetings led to multiple insights that will be extremely valuable in shaping our product.

As our time in Dublin came to a close, we had had a great time and left with some big takeaways.  First, the reaction following our presentation affirmed the great value in our product.  Second, there was a lot of insights to be garnered from truly understanding how a company handles trouble tickets.  Lastly, Dublin is a fantastic place to do business and some team bonding.

DVation greatly appreciates the help of Jay Borenstein for setting us up with the visit to SAP and Chis, Manish, and the other people at SAP that met with us and showed us amazing hospitality.

Stanford Remedy Demo

On Monday, February 23, Santi, Jon, and Jai met with Tracy Neil, the Remedy ticketing system administrator, Chris Lundin, Director of the IT Service and Operations Center, and Adam, an employee at the Tier 2 IT Service Desk.  The purpose of this meeting was to further discuss the Remedy ticketing system as well as to present and gain feedback on another iteration of prototypes.  As a double bonus, we also got a brief walkthrough of the Remedy ticketing system and Chris offered to send some sample ticket data our way!

Probably the single biggest annoyance that Adam faces daily is user error when filling out and submitting a ticket.  This user error could be improper categorization of the issue which leads to a misrouted tickets.  Additionally, deficient user-written descriptions can make it nearly impossible for IT admins to determine what the issue is.  Even though issue reporting falls outside of our intended scope (ticket resolution), this observation gave us an idea.  A chat feature or at least an fast and easy way to contact the user reporting the issue would be convenient for the problem solver to find out more about the problem.

Another problem that occurs when using Remedy is that users forget to check the audit log, a log that keeps track of all the departments a ticket has been routed to, before re-routing a ticket.  Because of this, tickets sometimes get re-routed to the same department twice.  This made us think of providing a pop-up feature that warns when a ticket is about to re-routed to the same department.

When walking through our prototype, Chris and Adam provided some good feedback.  One important point was that routing a ticket to a specific person is usually not desired as it is against “etiquette”.  Routing a ticket to a group and then letting the group decide where the ticket should go is generally the desired behavior.  Additionally, many admins at Stanford use canned templates they send back to the user to help the user resolve an issue, so including this could be a helpful feature.  They also were not fans of our use of stars to denote priority, which we agreed with.  The last huge insight from the prototypes was that in order to get admins to post useful information and links for solving an issue, there needs to be some tangible benefit.  Or at least, they need to know that their work is helping others.  Chris suggested creating a good citizen score that would based on the number of people that found information an admin posted as helpful. We liked this idea a lot.

We left this meeting feeling good about our next iteration of prototypes while also ready to actually start some of the software development.  Oh and finally, we were going to have our hands on some data.

Prototype V3, V4 Demo

On Wednesday, February 18th the whole team headed to VMWare to meet with TJ, a Principal Architect associated very closely with the sales side, with the intent to show him our newest prototypes and get his feedback. Although in his position didn't put him close to the IT Admins we aimed to serve, we were able to learn a bit about the enterprise sales cycle. TJ also let us know that he thought out 'Integrated Search' feature was very useful and lets us know of some of the troubles the clients he had contact with had dealt with in the past. We walked away from the meeting not changing our prototype much but knowing we added value with this product and met a well-connected individual at VMWare. 

The next day, we returned to VMWare to meet with Vivian, Amit, and Denis again. This was the first meeting since our pivot, so we wanted to not only inform them of our new vision, but get some feedback on the prototypes we had developed. We focused many of our questions towards Denis, as he used to be an active IT Admin who would deal with ticketing systems.

The feedback we got was mixed. We felt that we had a hard time explaining to them our audience- we wanted to focus on those who have to deal with end-users who are often not technical submitting IT help tickets while it seemed that they believed we wanted to make a bug reporting system. This led into a large digression into Bugzilla and Helpzilla, which they walked us through. However, we did find that some features were definitely useful. Denis especially liked our Integrated Search feature which would allow for searching of internal documentation and Google simultaneously, as well as the Predictive Resolution, which would help to suggest ways to solve tickets. Amit, on the other hand, seemed very skeptical and asked questions about our user acquisition, infrastructure concerns, and other business aspects which we were not yet even thinking about. Amit later reflected his doubts about our direction in an email. 

This interaction left us with the conclusion that our application had the potential for great improvements for the IT ticketing space, but perhaps we were not presenting our value propositions clearly and concisely enough. 

Prototype V2 Demo

On Tuesday, February 17 at way too early in the morning (9:00 am) we met with Stephen Wong, the Director of IT at the Graduate School of Education.  Despite the team’s collective sleep deprivation, this was an extremely fruitful meeting as we learned about the ticketing system in place at Stanford (Remedy) and its annoyances, we discovered that there is a need for a smart ticketing system like the one we were prototyping, and we were given another contact within Stanford to discuss our ideas with further.

After introductions, Stephen briefed us on IT in academia and its difficulties by touching on the many different layers as well as the discrepancies in IT departments due to budgetary differences.  After this, Stephen told us about Stanford’s ticketing system — Remedy.  There were many important takeaways from Stephen’s discussion of Remedy.  First, the most common problem in issuing tickets is the routing of tickets to the wrong group.  Once assigned to a group, tickets are hand assigned to a particular person.  During our earlier brainstorming, the team had come up with the idea of auto-allocating tickets.  Stephen pointed out that this is an extremely difficult problem as there are hidden personal and political factors that go into assigning a ticket.  For example, tickets from certain faculty members who are known to be difficult to work with are assigned to IT personnel who are very patient.  Another idea that we thought of during brainstorming was automatically assigning a priority to a ticket based on learned features extracted from the ticket.  Stephen grounded us once more by mentioning the difficulty in this task.  With ticket prioritization political factors heavily influence the order in which a ticket is resolved.  For example, if the person who submitted the ticket is the dean or someone known to complain a lot, the ticket is handled quicker.

Despite finding out that two features we were excited about after brainstorming would be too hard to implement, we were not discouraged.  After elaborating more on our ideas and beginning to show the prototypes that Santi drew up, Stephen mentioned that to his knowledge, a system that learns from previous tickets does not exist and it would be pretty awesome for a system to automatically suggest resolutions.  This was extremely good news to hear!

After walking Stephen further through the prototypes, he pointed out some key insights.  First, many IT workers rely heavily on Google when troubleshooting so providing the ability to search Google and retain helpful links would be a great feature.  Additionally, Stephen mentioned that issues fall broadly into two categories: problems that are quick fixes taking less than 5 minutes to resolve and problems that the IT worker doesn’t immediately know how to fix and require more than 5 minutes.  If a problem fell under the 5 minute or less category, Stephen was a firm believer that the ticketing system should be as unobtrusive as possible allowing the user to dedicate his time to solving the problem and not navigating some feature-filled pages.  For problems requiring more than 5 minutes to solve, Stephen believed more information could be extremely helpful out streamlining the resolution process.

At the end of the meeting, Stephen put us in contact with Tracy Neil, the Remedy Administrator, another excellent person we could pick the brain of to advance our project.

Prototype V1 Demo

On Friday, February 13, Hussain and Jon woke up bright and early for the first meeting with VMware since we pivoted ideas.  We met with Terry, a user endpoint support guru, and Nate, an executive support employee.  The goal of the meeting was to learn more about the troubleshooting process from their perspective and to get some initial feedback on our first prototype.

The first thing we discussed was their day-to-day in order to better understand the work they performed.  Both Terry and Nate solved the issues assigned to them by directly working with users to solve the problem.  Nate specialized in working with executives while Terry focused on the average user.  One key insight we learned was that that 30-40% of the issues they encounter are quick fixes.

Next, we asked some questions to test some of the underlying assumptions to our idea of a smart ticketing system.  From this, we learned that many of the issues are repetitive – meaning that support is asked to resolve the same issues multiple times.  Additionally, under the current system used at VMware an article is written for repetitive issues to be added to the knowledge base.  However, Nate and Terry mentioned that these articles are frequently not used as they tend to be poorly written.  We also asked about the usefulness of logs when diagnosing a problem.  Nate and Terry said they do sometimes use logs to assist their work, but that the logs come from many sources, such as from logs from iOS devices, desktop computers, servers, etc.  The many sources of logs present a difficulty if we wanted to create software that could manage these logs and then diagnose problems from them.

Lastly we showed them a very rudimentary POP (Prototype on Paper).  Although the prototype was quite simple, it helped Terry and Nature better understand our idea.  From this, they provided a very helpful suggestion — filling out a report to resolve an issue needed to be as unobtrusive for the person resolving the error.  If this process was too cumbersome and there was no apparent benefit to completing it, they were confident workers would quickly disregard filling out reports.

At the conclusion of the meeting we had some good information that would be helpful in creating the next iteration of prototype.

Needfinding with Denis

Wednesday the 4th Santi, Andrew, and Jon found themselves on the VMWare campus again. This time we would be meeting Denis, the dogfood environment IT Admin at VMWare to talk about the issues he had with the current products VMWare and competitors offered. 

Denis generously gave us an hour of his time to walk us through the current IT management solutions offered by VMWare. This was a great exposure to what the current products looked like and behaved, as well as how IT Admins traversed these products. Some of what we saw was not too surprising; boring, unflashy software. We were often waiting extended periods of time for widgets to load and render. Also Denis seemed to have to jump around from application to application to get all the information he needed- it gave the impression that there was no great dashboard for all of the information Denis wanted in one place. Another interesting thing was that Denis seemed to be annoyed by many of the visualization widgets that were scattered throughout the various applications, his preferred dashboard to monitor key statistics was very simple and consisted of three bar graphs for CPU Usage, Memory Usage, and Storage Usage. 

We asked Denis typical needfinding questions, some resulting into very specific digressions, others bearing great insight. In particular, when asked "What is the most frustrating part of your job?" Denis responded "troubleshooting." He explained that when there is a failure in his IT infrastructure it is often very difficult and time-consuming to resolve the error. Reminiscent of the first project Karthik pitched us in our initial meeting, Denis explained that there are many points of failure in these systems and some of these are outside of the scope of his job/access. If he believed there was a problem with the storage, he would have to talk to the storage guys (who might be busy). If he believed there was a problem with the network, he would have to talk to the network guys (who also might be busy). While this was happening the users he supports would be out of service. He explained that this process was very frustrating, time-consuming, and occasionally meant he would be up late in the office troubleshooting in hopes that he could fix the issue before the next work day. He also explained that all the "predictive" systems to anticipate failures that he had dealt with were "total crap." Altogether, this was incredibly valuable insight into the life of an IT Admin. 

We also gave Denis an idea of what we were hoping to build. His reaction was not very enthusiastic and it didn't seem like something he would ever use and definitely was not something he currently needed.