all

Pivot to Prototype

We met in the loft on Thursday mostly for a travel-related SGM with Jay. But while we had the chance, we told him of our new product direction. Jay said he liked the idea, noting that it was much more clear and well-defined. He lauded our group on its flexibility but warned us not to celebrate yet, as we had a lot of catching up to do- in terms of benchmarking this new space, needfinding, and prototyping. 

On this front, the good news was we had a meeting scheduled for the next morning at VMware to meet with two IT Admins there to get feedback on our new product. We didn't want to show up empty-handed, so after our SGM we made some brief whiteboard prototypes to help determine the features we wanted to incorporate. 

Pivot

We knew we had to pivot. Plain and simple. Our last product proposal was both infeasible and its usage was unclear. We had to make something more concrete and implementable in less than 15 weeks. 

We met in the d.school at 10pm. Drawing from the conversation we had earlier that day with Jay and the teaching staff, we began brainstorming with emphasis on: 

  • Clear use cases. We knew we had to define its usage in a very concrete way. Although our original goal was to make very 'agnostic' software, making concrete use cases would probably mean making ourselves more niche and honing in on the IT space due to our relationship with VMWare. 
  • Actionability. We wanted our product to not only to have clear use cases but be a service to actually do something in the real world. Focusing on the IT world (see above bullet) this meant we want to make the jobs of either VMWare or VMWare's customers easier and more effective. 
  • Data Driven. We wanted to play to our machine learning background and allow the power of the algorithms we know to deliver value to new verticals. 
  • "Under-promise and over-deliver". Out product had to be something with a valuable core but with room to add on all sorts of sexy features. This way we could give ourselves reasonable goals and allow for the possibility to exceed expectations. 

We began our brainstorm. Working with our skills, we asked "What is modeling good for?" We talked about reactive alerts. We jumped around to a lot of different topics and many different users. The one person we had met with regarding needfinding, Denis the dogfood environment IT Admin, kept coming to mind. Denis found the task of troubleshooting IT environments to be a tiresome task often laced with the trouble of communicating with many silos... 

With some teamwork and iteration, we thought we had it. We would once again deal with log data, creating tickets from anomalies and errors found within the logs. These tickets or alerts would be intelligent though, giving IT Admins like Denis advice on how to solve tough errors based on errors statistically similar to it. We would allow for easy or even automatic collaboration with these tickets, routing tickets to those who really can address the issue at hand. Once again this was a very data driven product, but with clearer use cases, actionable results, and features easier to rapidly prototype. Plus, we were excited about the idea! We felt that we had successfully pivoted, and couldn't wait to pitch the idea to our users, IT Admins. 

Show us the way, Teaching Staff

Tuesday after our VC presentation and we show up at the loft ready for our SGM with Jay and the teaching staff. Feelings are mixed as we know we have a product idea that according to the VCs can "bring a lot of value" (so we are excited) yet is "highly dependent on our modeling layer" (so we are scared and unsure). We are going in to the SGM looking for some guidance. 

Thankfully, our CS210 staff gave us just that. One of the TAs led us through some procedural/infrastructure considerations, leading to creation of this blog and usage of our GitHub to track and assign issues. This was very helpful, but the more meaningful insight came from Jay who had further feedback from the VCs and plenty of personal opinions regarding our "Big Data to Small Data" exploration platform. 

Basically, it sucked. The use cases were unclear, the data was unavailable, the models didn't exist, the team was unclear if we could pull it off... the list goes on. Basically Jay told us that all our creeping suspicions and fears were justified and that we needed to better define our users and the actionable ways in which they could benefit from our product. He also thought our product was maybe too ambitious and that we needed to remember "it is better to under-promise and over-deliver than over-promise and under-deliver." With this input, we realized we had a lot of work to do, and planned an emergency pivot meeting that night in the only place we felt appropriate- the d.school. 

under-promise and over-deliver
— Jay Borenstein

VC Day

Thanks to Jay's connections in the industry, the entire class was able to pitch their project proposals to a panel of Venture Capitalists: John Lilly of Greylock, David Hornik of August Capital, and George Zachary of Charles River Ventures. This was an amazing opportunity to get industry opinions about our proposed projects from individuals who know what works and what doesn't. 

It was very entertaining to see our peers present their respective products, but eventually we had to face the music and present our product. Equipped with our slide deck we pitched our "Big Data to Small Data" product with the title "Explore Big Data." Our presentation went smoothly and it was time to hear the million dollar input from the VCs. 

After joking about how we wanted to take on Splunk and Domo (as two of the VCs were affiliated with these companies) we learned two main things: 

  1. A product like this is extremely valuable, predicated on the proper function of our models. 
  2. Creating models that scale and continue to give insight is very hard.

We found this input to be very valuable and would like to thank these assumably very busy men for their time. Unfortunately we began sensing a theme: did we bite off more than we could chew with this ambitious product? 

Needfinding with Denis

Wednesday the 4th Santi, Andrew, and Jon found themselves on the VMWare campus again. This time we would be meeting Denis, the dogfood environment IT Admin at VMWare to talk about the issues he had with the current products VMWare and competitors offered. 

Denis generously gave us an hour of his time to walk us through the current IT management solutions offered by VMWare. This was a great exposure to what the current products looked like and behaved, as well as how IT Admins traversed these products. Some of what we saw was not too surprising; boring, unflashy software. We were often waiting extended periods of time for widgets to load and render. Also Denis seemed to have to jump around from application to application to get all the information he needed- it gave the impression that there was no great dashboard for all of the information Denis wanted in one place. Another interesting thing was that Denis seemed to be annoyed by many of the visualization widgets that were scattered throughout the various applications, his preferred dashboard to monitor key statistics was very simple and consisted of three bar graphs for CPU Usage, Memory Usage, and Storage Usage. 

We asked Denis typical needfinding questions, some resulting into very specific digressions, others bearing great insight. In particular, when asked "What is the most frustrating part of your job?" Denis responded "troubleshooting." He explained that when there is a failure in his IT infrastructure it is often very difficult and time-consuming to resolve the error. Reminiscent of the first project Karthik pitched us in our initial meeting, Denis explained that there are many points of failure in these systems and some of these are outside of the scope of his job/access. If he believed there was a problem with the storage, he would have to talk to the storage guys (who might be busy). If he believed there was a problem with the network, he would have to talk to the network guys (who also might be busy). While this was happening the users he supports would be out of service. He explained that this process was very frustrating, time-consuming, and occasionally meant he would be up late in the office troubleshooting in hopes that he could fix the issue before the next work day. He also explained that all the "predictive" systems to anticipate failures that he had dealt with were "total crap." Altogether, this was incredibly valuable insight into the life of an IT Admin. 

We also gave Denis an idea of what we were hoping to build. His reaction was not very enthusiastic and it didn't seem like something he would ever use and definitely was not something he currently needed. 

Pitching "Small Data" to VMWare

On Tuesday, February 3rd Karthik came to Stanford and met us in the Gates Computer Science. The purpose for this meeting was to get feedback from VMWare's perspective about our "Big Data to Small Data" proposition. 

While preparing this pitch, we prototyped an infrastructure schematic we believed would exist when building this product. We felt this would make it clearer to Karthik what we intended to build, and would also help to distill our thoughts so our pitch was more coherent. The result of this effort was the following hand-drawn schematic. 

While presenting to Karthik, we explained our proposed product would take your large, inoperable log data, pass it through a "modeling layer", resulting in a small "parameterized" representative set of data with which you could "play" with your data and gain the insight you needed. Karthik seemed pleased with our buzz-word filled pitch, but expressed two main concerns: 

  1. This product was mainly useful for C-Suite executives and managers, and was less intended for the closer-to-data employees such as IT Admins. Karthik stressed that although these executives were the "check writers" in his experience with the enterprise sales cycle the people we need to appeal to are the end-users (the "influencers"). In the case of VMWare this would be IT Admins, and our product failed to address their needs. 
  2. This product schematic was very dependent on the "Models" node. We could make everything else perfectly, but if our modeling fails the product fails. 

This input from Karthik was very valuable. Noting the large dependence on modeling, we stressed the need for VMWare to supply us with example data so we could begin modeling. Additionally, we planned our next meeting- a needfinding talk with the dogfood environment IT Admin at VMWare, Denis. 

Post-Meeting Brainstorm

Saturday morning after meeting our liaisons for the first time, we reconvened at the d.school to brainstorm solutions to the task that Karthik had laid at our feet. Once again, Santi led us by moderating.  

We quickly came to the conclusion that we wanted to work with unstructured log data. We felt that this helped keep our product more "data-agnostic" as simple logging techniques are both ubiquitous and information-rich. We began brainstorming about methods to extract insightful information from these logs. We came up with a variety of solutions from easy querying and visualizations to summary statistics to alerts and report generation. We felt like we had hit a jackpot of great ideas and started to dive into more detail about each one. 

At some point, Splunk came up. We knew Splunk was a large company with amazing technology to access your "machine log data." So we googled them. We found that all the sticky notes we had on our board were already on Splunk's sales page. We had spent the last hour reinventing Splunk. 

This was both saddening and encouraging. On one hand, our "new innovative ideas" were already implemented, and were surely to be better in basically every way than something we would make in 6 months. On the other hand, we realized there was value in what we were talking about. In about an hour we had thought up the core products to a billion dollar multinational company. We knew we were onto something, but we had to change direction. We had to distinguish ourselves from companies like Splunk. 

Differentiating from Splunk ended up being great for our brainstorming productivity. All of a sudden, we had a flurry of ideas. Synthesizing and compiling them, we found an ambitious product with a catchy, buzzword-filled tagline "Turning Big Data into Small Data." Inspired partially by the Stanford coursewe postulated that with proper unsupervised modeling techniques we could turn the large, intractable data companies had on their servers into smaller representative sets that users could interact with in their browser or mobile app. We envisioned that with smaller scale data we could allow for sweet visualization and interactivity without sacrificing the integrity of the overall data. Our main users would be executives and managers (decision-makers) who wanted to make data-driven decisions quickly without a lot of technical expertise. Additionally, we felt our product would be useful for employees that had to directly deal with the data at hand, such as IT Admins or Data Scientists, as they could get a summary of the data before using more technical tools like Splunk for deep-dives. Our envisioned product could give you an overview of your data, a "feel" for your data in some sense, with the potential to shed light on problematic or interesting trends automatically and without waiting for external reports. 

Altogether, we felt like we had successfully ideated a product that: 

  • Was different enough from other business intelligence solutions such as Splunk, Tableau, Domo, etc. 
  • Was capable of giving valuable insight to our users 
  • Helped to address the "3 month" reporting issue

As such, we compiled our notes and prepared to pitch this product concept to Karthik, who was to meet us the coming Tuesday to hear our proposal. 

Meeting VMWare

On January 29th, we were able to meet our liaisons at the beautiful VMWare campus. 

After introductions, coffee, and gelato we got down to business. Karthik and Vivian, our liaisons, along with a more technical employee named Amit, explained that they had envisioned two main project ideas for our class. 

The first idea was a product that would intelligently identify the source of an IT infrastructure failure. Amit explained that there were many components to these large scale IT deployments, and at any given time one or more might fail, causing disruptions of service for users and headaches for IT admins. The software product would essentially be an API-of-APIs that upon request would give predictions as to which component of the infrastructure was failing. All the VMWare employees avidly expressed that this product would be extremely valuable to the company. Unfortunately, this didn't align with our expectations or hopes for our project that we brainstormed the night before at all. 

The second idea was essentially a reporting tool for IT Admins and executives such as the CTO or CIO who wanted information regarding their IT infrastructure. Karthik explained that when VMWare's customers asked for this sort of information a long, time-consuming, and expensive process would have to occur. Karthik explained that in this scenario he would have to ask another employee to retrieve and compile the requested data, who might have to ask other employees, who might have to ask other employees. Basically, the pipeline for this sort of request was long, convoluted, and not efficient. In the current ecosystem, customers have to wait as long as 3 months to receive reports regarding their data. This was one of the most remarkable and important facts we learned that day, and helped us choose to pursue this project. The "3 month" figure made it obvious to us that there was a need for a better solution, but this project direction also more closely aligned with our expectations and goal and allowed for much more flexibility in the way of implementation. We immediately expressed interest in this project. 

In the current ecosystem, customers have to wait as long as 3 months to receive reports regarding their data.

To give them an idea of what direction we wanted to take the project, we let them know a brief synopsis of our thoughts from the night before. Karthik let us know that despite the typically slow-moving and unsexy nature of enterprise software he thought our ideas were both extremely valuable and unique. With that, we ended the meeting optimistic of our future and partnership with VMWare. 

Pre-Meeting Brainstorm

The night before our first meeting with our VMware liaisons, all members of DVation met in the famed Stanford d.school to have a preliminary brainstorming session. The point of this brainstorm was to go into our first meeting with some broad ideas of what we wanted to accomplish and what sort of products we wanted to make. Our Project Theme had hinted at applications of Big Data and Data Visualization, so these two themes guided our discussion. 

In preparation for this brainstorming session, we all researched different aspects of the space we were to be in. Some of us researched exactly what VMWare does and the products they offer, while others researched the data visualization competitive landscape and other business intelligence solutions. These insights proved to be invaluable during our brainstorm, 

Santi, our design thinking messiah, led the brainstorming activity. In true d.school fashion we proceeded with "yes, and..." statements, writing and rearranging sticky notes. We focused on thinking of new and innovative ways to visualize data and give insight into one's business. We made guesses regarding the nature of the data we would have access to via VMWare. We thought of ideas which would leverage our skills and backgrounds, mainly in the area of machine learning. 

After a few hours of deliberation, we encountered a few main themes that we all agreed on: 

  • We wanted to make a data-agnostic product. Sorry VMware, we don't want to make something that only applies to your space. 
  • We were all allured by the idea of quantifying productivity. 
  • The idea of interesting info coming to you was consistent. We liked the idea of not having to make queries, but having analytic solutions give a business owner info "before he/she even needs it."
  • We wanted to make some sexy visualizations. 

With this collection of broad ideas, we felt prepared to meet our liaisons at VMWare the next afternoon. 

DVation, the beginning.

 DVation is a group of motivated Stanford seniors working alongside VMWare via CS210. To learn more, check out our About page. 

Over the next 20 weeks we will be working alongside VMWare and the wonderful CS210 teaching staff to create a groundbreaking software product. This blog will serve as a journal of the entire process, from brainstorming and needfinding to development and deployment. 

Hope you enjoy our journey.