Big Data hype still here, but maturity beckons

IDC Analyst: Big data is about decision-making, not just technology for technology's sake

Few tech industry buzzwords have gotten as vigorous a workout as 'big data', but while the hype remains plentiful, it is starting to give way to real-life successes as well as formal ways companies can develop big data strategies, according to a number of IDC analysts.

"We're at this period now where there's certainly a lot of hype, a lot of promise," said IDC analyst, Dan Vesset, during a panel session at the research firm's Directions 2013 conference in Boston in March. "The question is what's reality, and what can and should companies do in the near and short term?"

IDC, which is owned by IDG News Service's parent company International Data Group, defines big data as information-driven, tactical decision-making resulting in new economic value, derived from large data volumes from a wide variety of sources, Vesset said.

The first point is crucial, according to Vesset.

"We can be installing all kinds of technology but the important thing is to improve the decision-making," Vesset said. "You can have the greatest Hadoop deployment in the world but it's not going to be enough," he added, referring to the popular open-source data-processing framework that is synonymous with the big data movement.

Progressive is one example of a company using big data projects to transform their business, said IDC analyst, Michael Versace, who presented with Vesset. Using detailed information about customers' driving habits, the insurer has created a usage-based model that defines a policy's price down to the individual, he said.

Progressive gets the data through a device a driver plugs into a car's diagnostic port, according to its website. It can track how often customers slam on their brakes, drive late at night and other possibly risky driving habits. If the data shows a customer is driving safely, they can get significant discounts on their insurance.

Meanwhile, a number of challenges stand in the way of companies wishing to launch successful big data projects of their own, according to Vesset. These include the dilemma of what business data should be stored and what should be discarded, the cost of acquiring needed technologies and a lack of IT professionals with the necessary skills.

The last problem will likely become more severe in the near future, according to Versace. "That gene pool is growing pretty shallow right now."

Some misperceptions also persist with regard to big data, Vesset said. For one thing, "it's not all about social media," he said. "That's one of the big fallacies out there, that big data is all about clickstream analysis."

Nor is Hadoop a sole solution, since it's oriented to large-scale batch processing and not real-time monitoring, such as a rail company tracking the performance of specific components in their train cars as they deliver shipments, he said.

The analyst firm has created a 'maturity model' for big data, which Vesset and Versace described in their session. It spans five subject areas: Data, people, process, technology and intent; and five stages of deployment: ad hoc, opportunistic, repeatable, managed and optimised.

A first step for companies just starting out with big data is to identify opportunities to use their existing technology and data in new ways, evaluate public cloud and open-source options, and start experimenting with proof-of-concept exercises and prototypes, according to the analysts.

Over the following one to two years, those companies should look to use early successes with big data projects to justify funding for larger efforts. In the same period, it would be wise to also look for sponsors within business departments who will champion big data projects, they said.

Some 80 per cent of what can be considered big data is unstructured or semi-structured information, said IDC analyst, David Schubmehl, during another presentation at the event. These sources could include anything from clickstream data to patent records, research archives and even video, he said.

This diversity will give rise to what IDC is calling unified information access technology, evidenced by products such as Oracle's Endeca and IBM's Vivisimo, as well as specialised vendors like Attivio.

Big data's challenges will also continue influencing the database industry, with growing prominence for technologies like graph and in-memory database platforms, said IDC analyst Carl Olofson during the presentation.

Traditional relational databases will also change, with capabilities expanding "to the point Ted Codd wouldn't recognise them," Olofson said, referring to the 'father' of the relational model.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Latest Videos

More Videos

looking for the best quality of SMM Panel ( Social Media Marketing Panel ) is a website where People Buy Social Media Services Such as Fa...

Kavin kyzal

How to manage social media during Covid-19

Read more

Thank you for sharing your knowledge. Definitely bookmarked for future reading! Check this website https://a2designlab.com/ with lots of ...

Pierce Fabreverg

Study: Gen Z are huge opportunity for brands

Read more

Thanks for sharing. You might want to check this website https://lagimcardgame.com/. An up and coming strategic card game wherein the cha...

Pierce Fabreverg

Board games distributor partners with Deliveroo in business strategy pivot

Read more

Such an important campaign, dyslexia certainly need more awareness. Amazing to see the work Code Read is doing. On the same note we are a...

Hugo

New campaign aims to build understanding around scope and impact of dyslexia

Read more

Great Job on this article! It demonstrates how much creativity, strategy and effort actually goes to produce such unique logo and brandin...

Pierce Fabreverg

Does your brand need a personality review? - Brand vision - CMO Australia

Read more

Blog Posts

Ensuring post-crisis success

The COVID-19 pandemic has exposed brands’ CX shortcomings and a lack of customer understanding. Given ongoing disruption, customer needs, wants and expectations are continually changing, also causing customers to behave in different ways. Just look at hoarding toilet paper, staple and canned food, medicinal and cleaning products.

Riccardo Pasto

senior analyst, Forrester

A few behavioural economics lesson to get your brand on top of the travel list

Understanding the core principles of Behavioural Economics will give players in the travel industry a major competitive advantage when restrictions lift and travellers begin to book again. And there are a few insights in here for the rest of the marketing community, too.

Dan Monheit

Co-founder, Hardhat

Predicting the Future: Marketing science or marketing myth?

Unicorns, the Sunken City of Atlantis, Zeus: They are very famous. So famous in fact, that we often think twice about whether they are real or not. Sometimes if we talk about something widely enough, and for long enough, even the strangest fiction can seem like fact. But ultimately it is still fiction - stories we make up and tell ourselves over and over until we believe.

Kathy Benson

Chief client officer, Ipsos

Sign in