The recent Tidewater Big Data event was an incredibly informative day with a number of experts providing perspective and sharing experiences on how they are using Big Data and the technologies that are enabling them to get value out of that data. As I listened to the sessions, three big themes jumped out to me.
1. Hadoop is a unifying platform.
Now more than eight years old, Hadoop has become the defacto standard for Big Data processing. Facebook, IBM, Yahoo, Amazon, and others all use Hadoop. More than 50% of the Fortune 50 companies are now using Hadoop for data storage and analysis. With this ubiquitous use, the Department of Defense and the Intelligence Community are increasingly looking to Hadoop to provide new capabilities for handing the information explosion we’re experiencing in the 21st century. Gus Taveras, former CTO of the Defense Intelligence Agency, gave a great talk on the future of intelligence sharing. The day also included some great primers on what Hadoop is, how it works, and how we got to where we are in terms of using Big Data for better decision making.
2. Distribution for Hadoop is key to accelerating innovation.
Simplifying the process of bringing data in and managing it is critical for increasing the use of Hadoop, especially in the budget and resources-constrained federal market. The Lightening Talks centered on “Hadoop-able” problems and how our participants effectively used the technology to find new insights in existing data
3. Cloudera remains the thought leader in Hadoop.
The event was sponsored by Cloudera, so some self-promotion was to be expected, but in this case it is earned. Doug Cutting, the original creator of Hadoop, is now the Chief Architect at Cloudera. Through Doug’s leadership, Cloudera has sponsored continued development of the open source Hadoop project. The case studies offered by the Cloudera team throughout the day really made the case that the promise of Hadoop can be realized in an affordable and efficient way.
For a look at all of the archived sessions, please click here.