Top 2016 Big Data Challenges: Skills Shortage

In the coming year, companies across industries will see an unprecedented need for more data science talent. Additionally, there will be an ever-increasing demand for departments across organizations to align objectives and leverage the data needed to achieve those goals.

Making big data a priority for 2016 means addressing these issues as they arise so you can realize the value of big data analytics.

Demand for data scientists overwhelms the supply

A report from McKinsey found that the American economy will experience a shortfall of between 140,000 and 190,000 people with the expertise required to make big data campaigns successful. Data scientists have varying areas of expertise, however, a keen analytical mind is a requirement. Whether you need data scientists who are trained in mathematics or any social sciences, it’s becoming increasingly difficult to staff teams.

This isn’t a new problem. Looking back even a few years ago, when it became clear that data was essentially currency, people predicted significant shortfalls in data scientists. In 2015, companies throughout the industry felt the sting acutely.

Every day, new job postings go up looking for qualified data scientists. It takes time to find candidates and not every data genius is the perfect fit for every company. There may be some relief coming, with specialty certificate and alternative education programs for big data popping up from universities and other educational institutions, but it’s not an immediate fix for 2016.

If companies want to fill their teams with more data scientists, my advice is to hire people in accordance to the nature of the problems companies want to solve, not all problems require advance data science. In addition, often overlooked are aspiring but non-conforming data scientists who have good math and technical disciplines but without formal training.

Getting complete alignment takes time, effort

Ask any member of a C-level what he or she needs out of big data, and you’re going to get a number of different answers. The analytics that matter to marketers varies greatly from the insights a network security analytics wants to see.

The simple lesson is that every department needs analytics, so they all need data. How, then, can a data science team – likely understaffed, mind you – make everyone happy? The answer is to align goals across the company. Even if the desired analytics seem entirely disparate, encouraging data sharing throughout the company will help each department get what they need.

Marketers trying to nail down customer sentiment need data from IT, finance and sales. IT security putting together baseline activity for users on a network need to monitor every users’ behavior, as well as information on transactions. Everyone needs to be aligned and capable of sharing data throughout the enterprise for a company to get the analytics it needs.

Without proper alignment, big data and analytics projects degenerate to little more than interesting science projects. Siloes are the bane of successful big data projects. Even the slightest interference from one department or employee can deem an entire initiative useless.
Alignment needs to start at the top of a company, with every executive mandating his or her teams work with other departments to share relevant data. Big data and analytics are long-term, ongoing projects that need new data daily and must be improved over time.

Achieving full buy-in from all stakeholders and a commitment to provide resources are both significant challenges. Once they’re overcome – and only then – can big data change the enterprise. We’ve seen retail and marketing verticals revolutionize their industries by making big data work.

Segmentation, upsell opportunities and personalization are all common among Web retailers in 2015. All of those strategies came about because marketers and retailers committed to understanding their audiences and let big data show them the way.

Everything was about big data in 2015 – none of that is going to change moving forward. Despite the nearly universal demand for better data and analytics, many companies are struggling to get there. The challenges are numerous, however, getting creative and committing to solve them will pave the way for analytics to change business forever.

Information-Management: http://bit.ly/1K8lDD9