Data Analytics Governance Gets More Important

Everywhere you turn these days there are headlines about the rise of big data and how predictive analytics will usher in a new era of corporate innovation, one that upends the norms for how companies interact with their customers.

According to Bloomberg News, data scientists are in such high demand that they’re commanding salaries upwards of $200,000. The article quotes a McKinsey study projecting that “by 2018, the US alone may face a 50 percent to 60 percent gap between supply and requisite demand of deep analytic talent.”

But despite all the hype over predictive analytics, we’re still very much in an environment “where data science cowboys operate in a wild west of innovation and deployment,” at least according to Michael Goul. Goul is an association dean for research and professor of information systems at Arizona State University’s W. P. Carey School of Business, and he’s spent the last few decades studying artificial intelligence and business analytics. While he agrees that data science has the potential to revolutionize commerce, he also thinks too many companies are rushing headlong into the field without putting proper governance systems in place, and in some cases this might lead to disaster.

Goul recently met a consultant at a conference who told him about a guy he worked with at a bank 20 years before. “He discovered that the customers they were throwing out because they were having late fees and not paying on time, those turned out to be some of the most profitable customers. So he went to the IT department and asked if they could implement a change in the policy that would not get rid of these customers.”

But the IT department claimed it was too busy to tackle those changes, so the employee learned how to code in COBOL and implemented the changes himself. While this may seem like the perfect case study for the benefit of using analytics, consider the fact that the new code remained at the bank long after that enterprising employee left, which means there’s a policy in place that’s based on 20-year-old data. “That code has been embedded for 20 years and probably everybody who knew about it has long since left the company,” said Goul. The potential consequences for such an outdated system are tremendous.

Goul knew companies needed a series of frameworks with which to approach data analytics in a holistic, responsible manner. In 2014, he teamed up with the Society for Information Management’s Advanced Practices Council, a program for CIOs who learn from the world’s leading researchers and each other how to successfully leverage digital technology, to conduct research, and develop these frameworks. He did this by seeking out companies that would be willing to speak openly about their use of data analytics. “There’s a good chance that if someone is speaking at a conference then they’re probably more willing to share what they’re up to,” he explained. “And so I would target those companies that were being open and presenting things at conferences, and if I had to follow some of these people around to ask them questions, that’s what I would do.”

What were those questions? “Who are the stakeholders involved in your projects? Is your corporation looking at this from a centrally managed viewpoint or some hybrid in which there are some projects you manage at the corporate level and others that you let individual departments manage? And how does that tie with your resource acquisition or the technology capabilities in the vendor suite applications that would allow you to build your analytics?”

Goul began to see patterns emerge in how various stakeholders answered these questions, and from that he was able to develop frameworks for how data analytics should be approached.

Designing the analytics process

At any company, there are a number of stakeholders across many departments ranging from customer service to sales to marketing. Deploying any analytics solution doesn’t occur in a vacuum, so one must ensure the entire organization is fully informed of the proposed approach. “That’s where there’s typically a possible friction, because you’re running an independent department, and you’re doing well on the projects you have but you want these things embedded into the core functions of the organization,” said Goul. “You’re getting down into the logic of the production systems, and this really becomes a broader corporate issue as to how to manage that.”

One important metric to establish during the design stage is the rate of decay. Any algorithm will eventually become obsolete once the outside environment changes, so there should be a predetermined “sell by” date in which the data team should reevaluate the systems in place and determine if they should be updated. “It’s like programmers who hate to do documentation — I think data scientists aren’t so prone to capturing all the details of their model by the time they leave it and move on to the next project,” said Goul.

Establishing a testing system

Data scientists, try as they might, aren’t always objective observers of their own work, so there needs to be an independent assessment of the model’s efficacy. This might mean having multiple data analytics teams that test each other’s work or hiring a third-party consultant. “One of the companies that’s doing this well is State Farm,” said Goul. “It has an independent group that comes and does tests of different analytics programs that have been embedded and see if they’re as effective as they were intended to be.”

Empowering your employees

Many data analytics programs require the participation of employees, whether it’s a salesperson entering information into a CRM or retail employees carrying around a handheld tablet they use to try to upsell a client. In most cases, these employees might not even know of the data scientist’s existence, so there needs to be an effective training system in place to ensure they’re making correct use of analytics.

Accounting for streaming and big data

Some of the largest companies deal with giant datasets that are streaming in real time. Because there are so many factors at play, any underlying assumptions about the data can be quickly upended the moment the environment changes by a tiny degree. Because of this volatility, data teams must be constantly testing the data models and periodically calibrating it. “Let’s say your model is analyzing the stock ticket markers coming in and it’s looking at the weather data, which is constantly changing and fluctuating,” said Goul. “You’ve got models looking at this streaming data and implementing rules, and yeah it’s so messy and complex that it’s hard to even put some parameters around it.”

We’ve seen when big data has backfired. Whether it’s Target angering parents by informing them of their daughter’s pregnancy or the flash crash of 2010, misuses of data can have unintended consequences. So while it’s good that companies are wading in and testing the potential of data analytics, it’s also important that they understand the limitations of those analytics and how, if not deployed properly, it can lead to unforeseen outcomes that will hurt, not help, your business.

Information Management: http://bit.ly/1Nqnaf7