Full-Stack Analytics: The Next Wave of Opportunity in Big Data

This post orginally appeared on KDnuggets in May of 2014 and came out of a panel discussion at Analytics Week in Boston that was moderated by Gregory Piatetsky of KDnuggets.  On the panel, I was asked to discuss where we see investment opportunities in the Big Data landscape and this post will expand on my comments.  The lens through which I make these observations is from our role as a seed and early stage venture capital investor, which means we are looking at where market opportunities will develop over the next 3-5 years, not necessarily where the market is today.

Over the past few years, billions of dollars of venture capital funding has flowed into Big Data infrastructure companies that help organizations store, manage and analyze unprecedented levels of data.  The recipients of this capital include Hadoop vendors such as Cloudera, HortonWorks and MapR; NoSQL database providers such as MongoDB (a Flybridge portfolio company where I sit on the board), DataStax and Couchbase; BI Tools, SQL on Hadoop and Analytical framework vendors such as Pentaho, Japsersoft Datameer and Hadapt.  Further, the large incumbent vendors such as Oracle, IBM, SAP, HP, EMC, Tibco and Informatica are plowing significant R&D and M&A resources into Big Data infrastructure.  The private companies are attracting capital and the larger companies are dedicating resources to this market given an overall market that is both large, ($18B in spending in 2013 by one estimate) and growing quickly (to $45B by 2016, or a CAGR of 35% by the same estimate) as shown in the chart below: 

While significant investment and revenue dollars are flowing into the Big Data infrastructure market today, on a forward looking basis, we believe the winners in these markets have largely been identified and well-capitalized and that opportunities for new companies looking to take advantage of these Big Data trends lie elsewhere, specifically in what we at Flybridge call Full-Stack Analytics companies.   A Full-Stack analytics company can be defined as follows:

1. They marry all the advances and innovation developing in the infrastructure layer from the vendors noted above to
2. A proprietary analytics, algorithmic and machine learning layer to
3. Derive unique, and actionable insights from the data to solve real business problems in a way that
4. Benefits from significant data "network effects" such that the quality of their insights and solutions improve in a non-linear fashion over time as they amass more data and insights. 

A Full-Stack Analytics platform is depicted graphically below:

Two points from the above criteria that are especially worth calling out are the concepts of actionable insights and data network effects.  On the former, one of the recurring themes we hear from CIOs and LIne of Business Heads at large companies is that they are drowning is data, but suffering from a paucity of insights that change decisions they make.  As a result, it is critical to boil the data down into something that can be acted upon in a reasonable time frame to either help companies generate more revenue, serve their customers better or operate more efficiently.  On the latter, one of the most important opportunities for Full-Stack analytics companies is to use machine learning techniques (an area my partner, Jeff Bussgang, has written about) to develop a set of insights that improve over time as more data is analyzed across more customers – in effect, learning the business context with greater data exposure to drive better insights and, therefore, better decisions.  This provides not only an increasingly more compelling solution but also allows the company to develop competitive barriers that get harder to surmount over time.  In other words, this approach creates a network effect where the more data you ingest, the more learning ensues which leads to better decisions and opportunities to ingest yet even more data.

In the Flybridge Capital portfolio, we have supported, among others, Full-Stack Analytics companies such as DataXu, whose Full-Stack Analytics programmatic advertising platform makes billions of decisions a day to enable large online advertisers to manage their marketing resources more effectively; ZestFinance, whose Full-Stack Analytics underwriting platform parses through 1000s of data points to identify the most attractive consumers on a risk-adjusted basis for its consumer lending platform; and Predilytics, whose Full-Stack Analytics platform learns from millions of data points to help healthcare organizations attract, retain and provide higher quality care to their existing and prospective members. 

Each company demonstrates important criteria for success as a Full-Stack Analytics company:

1. identify a large market opportunity with an abundance of data;
2. assemble a team with unique domain insights into this market and how data can drive differentiated decisions and have the requisite combination of technical skills to develop and;
3. manage a massively scalable learning platform that is self-reinforcing.

If your company can follow this recipe for success, you will find your future as a Full-Stack Analytics provider to be very bright!

Comments

[Alex]

Well, right now even focusing on the new startups you could get a sense of how big this market is. And will grow for sure even more, because big data is for the future.
And even a more complex trend is to have smarter and smarter AI to process it plus to "understand" what it means.

[Nadim Hossain]

Really well articulated. This is exactly what we are going after at BrightFunnel, for our specific area of domain expertise - predictive revenue analytics for B2B marketers. I wholeheartedly agree with your points above. While actiomable insighta, in the form of attribution and prediction are critical for our audience, ultimately network-wide benchmarking amd imtelligence is what's most disruptive in the longer term.