Data Informed have published interview with Doug Cutting on the future of Hadoop.
He makes a number of observations.
On MapReduce…
MapReduce is on its way to being legacy.
On important Hadoop developments…
I think Kudu is very exciting; a new storage engine that offers a lot of low-latency, random-access capabilities that HDFS doesn’t while still permitting the fast analytics that you can do on the flat files in HDFS.
On deployment…
…providing a vendor-neutral cloud, so you don’t have to be locked into an Amazon or a Microsoft or a Google but retaining the option to move your data and operations, we think is important.
On hardware developments…
it’s pretty clear what the next hot hardware area is: memory technologies that are coming out that give you orders of magnitude faster access to persistent storage and also, combined with that, hardware that lets you access that memory over a network without involving the remote CPU, so basically every machine on a cluster can have micro-second-level access to all the memory in that cluster. And that’s going to be a game changer.
