Among the many trends slamming into IT today is the idea of Big Data, and while it is no doubt the buzzword of the day, there is definitely something happening around the idea of using data to understand your business better.
A couple of weeks ago I spoke to MIT professor Andrew McAfee about this notion of big data. McAfee is best known for being the guy who coined the term Enterprise 2.0, the idea of bringing social tools into the enterprise. He is currently principal research scientist at MIT’s Center for Digital Busines and recently wrote a book called Race Against the Machine with Center for Digital Business colleague, Erik Brynjolfsson.
McAfee likens Big Data to the invention of the microscope in that the microscope allowed scientists to see things that were previously invisible. By the same token, Big Data exposes information that you couldn’t necessarily find before. “We now have massive amounts of low-level details and pretty powerful tools to get abstract meaning,” McAfee said.
And he believes this is going to be as big for the world of business as the microscope was for biology. That’s because with a bigger body of data, the more you can learn if you are careful and rigorous in how you handle that data, he explained.
And in this brave new world of big data, we are going to need subject experts to ask the right questions. That could be highly trained data scientists or it could be just people who know a lot about a given subject because of their background.
McAfee says in the new enterprise where Big Data is going to be playing an increasing role, knowing what questions to ask is going to be key. He sees new jobs for people who can ask the right questions because as he says, “Computers are great question answerers and fairly lousy question askers.”
He adds, “I haven’t seen a machine that knows what questions to ask. That’s a key issue. This is where good old fashioned experts still have a role to play,” he said.
As IT Pros, you are going to have to learn to process this big data and find tools for the non-technical experts and suits in the C-Suite to mix and match the data. The big difference between this and traditional business intelligence is that with BI you were looking back where you were at a given point in time, whereas with Big Data, you can analyze data in real time and begin to make more intelligent decisions about where to put your resources at any given moment.
As McAfee points out this is not going to reduce jobs. If anything, it’s going to require more people to work with this growing pile of data. “Big data isn’t going to drive out people. It’s going reduce the market share decisions we make by gut and that would be a good thing.” And IT pros are going to be on the front line of this transformation.