How are developers building the next big app? What’s this next big app look like? What are the factors that will transform Big Data-infused apps into game-changers?
Actuate recently invited four industry experts to discuss the future of creating the next generation of data-driven applications and how they are influenced by open source and freemium software.
Allen Bonde (@abonde), VP Product Marketing & Innovation at Actuate, moderated the panel which included:
Here is Part 2 of the conversation:
Bonde: I’ve said this at a couple recent presentations that “fast is the new big.” When it comes to data, do you guys agree with that or not? Is it worth more to have more data or to do things more quickly?
Kolsky: The problem here with Big Data is that you can call it whatever you really want to. I don’t really care. But I mean yes, fast — the whole concept of big data stems from the fact that we can process, store and manipulate data hundreds of times faster than we could before, because of evolutions in data management, data storage and processing power. So it’s not big data, it’s the same thing that we had before. We can just manage it better and faster. And that’s what — when people tell me big data, the first thing I say is that you need real-time, because that’s what it means to me.
O’Grady: Yeah. And I would add to that. I think on the one hand you have the experiences — there’s an engineer from Wal-Mart that came out at one point and said, “You know what? More data beats better algorithms.” And that’s something you hear data scientists say over and over again. Whether or not it’s true I think depends on the situation. But in general, yes, you want more data rather than less. That being said, to the point earlier, a lot of the data that many of the companies we deal with are looking at is not even remotely big data.
We’re not talking about petabytes, we’re not even talking terabytes. In some cases you can actually have data sets that are hundreds of gigabytes in size that actually give you huge input. One of the guys who — was it Paulson? There’s a hedge fund manager, and he was essentially trying to glean insight in terms of market performance by trying to correlate market ups and downs to taxi dispatch data in New York City, specifically on Wall Street. This guy’s got tons of data.
But it was the insight to say, “This is something we can do with it.” So faster, big or better, I don’t know, but I think it varies.
Milinkovich: Personally, I don’t think it matters whether it’s fast or big or anything else. It’s what you do with it. Anytime you put together charts where you have a bunch of squiggly lines that follow each other around, the first thing you have to think about is correlation is not causation. Correlation is not causation. Matching lines up, there’s a whole website that’s dedicated to correlating completely bogus things.
Bonde: You mean like the shark bites and ice cream sales?
Bonde: Turns out that eating ice cream is dangerous.
Kolsky: You’re making a great point. It’s not the data, but what you do with it. I think that’s the most critical aspect, because one of the reasons we see things like this emerge is because most users don’t know what the data does is what to do with it. You know that you have customers, you know they have problems. You know that you have stores, you know that the stores are selling products to your customers.
You don’t know then that SCA-slash– ID – slash– 22. It’s your customer name. And you don’t know need to know that. That’s what the idea is behind these improvements. It’s like, you know, we have much better databases that allow anybody to understand what the data is, and you can do it the first time. But again, it goes back to what he said. If you don’t know what you’re doing, it really doesn’t matter what data you see.
Bonde: Let me build on this idea of the interface. There’s a lot of buzz around the consumerization of IT, for example, and the influence of consumer trends. When we have our consumer, our shopping role and how that sets expectations when we start to work, for example. We saw the global adoption of tablets, with the sort of merging and blending of consumer and business. I’ll address this to you, Loie, because again, you have this perspective of the consumer perspective.
How much is the consumer world influencing everything we’re talking about? And not just pretty pictures and ease of use, but is it more than that, or is it mostly that?
Maxwell: Well, I want to add on to what they were discussing. It’s the visualization of data, because you used to have to have an analyst that you paid hundreds of dollars an hour to analyze the big data, break it down, tell you what you need to be doing. Now I think with the way we’re building tools, you can actually create a visualization of that data and/or allow individuals to carve it up however they need to, or their business needs.
I think that there’s a democratization of that, essentially, with these apps coming out, and being so much more user-friendly. It is kind of picking up on what the consumer population has to have in order to adopt consumer products, now that there are actually advances in the design of technology, and that it’s no longer okay to just have graphs and charts and numbers and code. We actually are seeing a change, a shift.
Bonde: How much do you think the open source movement and the community has driven this innovation towards ease of use and ubiquity?
O’Grady: Well, I don’t know… And actually, I think it would not be fair of me and not accurate of me to say that open source has necessarily driven good design or usability. What I will say is that I think open source has really driven a bottom-up adoption cycle, and a bottom-up adoption cycle has profoundly influenced in terms of the way technologies are adopted, the way technologies are procured. Where, look, if I’m an enterprise company ten years ago, I could sell to a CIO, I’m always basically selling to one person.
What the product looks like isn’t all that important, because at the end of the day I just need to get this one person, and that person doesn’t care about things like ascetics usability and so on. Today it’s a lot different. Today we a market that’s really looking at evaluating products by month, and very much selling an iPhone, I don’t sell an iPhone to the CIO and he rolls that out. I sell an iPhone to each individual in the company. And therefore it’s a very different process. Things like design matter. Things like availability matter. Things like installation, ease of use, etc., all those things matter.
But it all, to me, comes back to the bottom-up adoption cycle, which has been driven in part by open source.
Milinkovich: I agree with that. I forgot to say that I agree with you. But one other auxiliary point about the role of open source has played in innovation in big data, is that big data’s really one of the first major industry movements that came from open source first. Because the whole market segment called big data really started with the Apache Hadoop project, and Apache’s done a fantastic job of growing a community of projects around this particular domain.
And we see companies like Cloudera and Waterworks coming out of this sector. That entire innovation cycle was driven out of open source. So I think it’s really the first example that I can think of where a major new and novel innovation in the enterprise software market came first from open source and became mainstream directly from open source. So it’s a subtly different point than what you’re making. And a lot of that, again, it was bottoms-up acquisition. It was developers realizing the power of what they could do with these tools that made it happen. But it was definitely driven by open source.
For more insight, here is a separate conversation between Bonde and Maxwell.