Why Do IT Projects Often Fail?

Despite new methodologies and management techniques meant to head off spectacular failures, critical technical initiatives still fall flat at an alarming rate.

In the age of agile development, DevOps and related management techniques, is IT project failure even a thing anymore? The answer, sadly, is yes.

In the past, IT failures often meant high-priced flops, with large-scale software implementations going on way too long and way over budget. Those failures can and still do happen. Case in point: IBM’s never-completed $110 million upgrade to the State of Pennsylvania’s unemployment compensation system.

But IT failure today is frequently different than in it was in the past, as agile, develops, continuous delivery and the fail-fast movement have changed the nature of how IT handles projects.

These iterative management methodologies and philosophies are meant to minimise the chances of projects going spectacularly awry, but the fact of the matter is that IT projects still fail, just in new and sometimes more insidious ways.

Here’s what seven IT leaders and analysts say about the state of IT project failure today.

A Cautionary Tale

The Project Management Institute’s 2017 Pulse of the Profession report found that 28 percent of strategic initiatives overseen by survey respondents were deemed outright failures.

Some 37 percent of the more than 3,000 project management professionals who responded cited a lack of clearly defined and/or achievable milestones and objectives to measure progress as the cause of failure, followed by poor communication (19%), lack of communication by senior management (18%), employee resistance (14%) and insufficient funding (9%).

And speaking of money, the same report found that due to poor project performance, organisations waste an average of $97 million for every $1 billion invested. That’s better than 2016’s $122 million in waste, but still a significant amount of cash lost.

Factors for Failure

Despite new methodologies and management techniques meant to head off spectacular failures, many of the factors that traditionally put IT projects at risk of failure are still present in the enterprise, experts say. Inadequate resources, overly aggressive timelines, underestimated costs, overlooked requirements, unanticipated complications, poor governance and human mistakes such as bad code can all lead to project failure.

PwC’s 2017 Global Digital IQ Survey polled 2,216 business and IT leaders from 53 countries and asked them what hinders digital transformation.

Some 64 percent of respondents said lack of collaboration between IT and business is to blame, 58 percent cited inflexible or slow processes, 41 percent listed lack of integration of new and existing technologies, 38 percent named outdated technologies and 37 percent put down lack of properly skilled teams.

Meanwhile, the criteria used to judge whether a project is successful, or a failure, has been expanding to reflect how critical today’s technology initiatives are, experts say. PMI’s 2017 Pulse of the Profession report states, “the definition of success is evolving.

The traditional measures of scope, time, and cost are no longer sufficient in today’s competitive environment. The ability of projects to deliver what they set out to do, the expected benefits, is just as important.”

The study identified organisations with 80 percent or more of their projects being completed on time and on budget while also meeting original goals and business intent; it classified this group as “champions.”

The report also highlighted the fact that these champions had invested in several common areas, including the leadership skills of project professionals, benefits realisation management, project management offices and actively engaged executives.

Agile and Automation to the Rescue?

Some trends, notably agile and develops methodologies, help mitigate the potential for wholescale project failures in modern IT shops.

The increasing use of automation in development and testing also helps to mitigate the potential for failure.

This stuff is really complex, and mistakes are made. But as more and more automation comes through, there should be less human errors made, especially in scripting and applications deployments and networking.”

Changes in organisational hierarchy help mitigate risk, too. Executives from various units are expected to partner together, to move quickly and adjust on the fly; in fact, leading organisations allow for more autonomy for course-correct, to enable this culture, according to analysts and consultants.

Fast Failure as a Tool

Reed A. Sheard, vice president for college advancement and CIO at Westmont College, has seen how that cultural shift has a positive result. He says he and other CIOs understand that not all projects are equal; each one carries different potentials when successful and varying degrees of consequences if they fail. With that in mind, he says, “we make judgements on where it’s OK to fail and where it’s not.”

He cites two recent initiatives to illustrate his point. The first one featured the implementation of a new platform to manage the school’s admission process, a critical undertaking where failure to meet user requirements and specific deadlines would be devastating, as admissions is one of the organization’s core functions.

Sheard says he was actively engaged in the project, evaluating progress and the resources that went into it. It went live, successfully, in July 2015.

The second initiative centered on delivering a platform that would allow alumni to network virtually. Sheard says his team tried to build a secure, user-friendly platform, but in the end couldn’t accomplish those objectives. Staffers struggled with securing the system, authenticating users and populating user profiles with the right information.

Others see that mindset, the willingness to sometimes fail, as a critical component for organisations that want to innovate and remain competitive.

“If you’re constantly learning and constantly improving, then you could be constantly failing,” says Terry Coull, a principal at WGroup, a firm that helps organisations optimise business performance and create value.”

“For those in that continuum, it means you’re adjusting, adapting, you’re flexible, so you’re working in small teams and delivering. So your project failure isn’t as large as it used to be. In the old waterfall world, you could lose a year before you find a failure.”

Risks of Failure Remain

These newest corporate cultural trends and IT methodologies certainly don’t guarantee success or fully guard against project failure. In fact, some say that there are elements in the modern IT shop that could even exacerbate the potential for problems that could take down a project.

One expert pointed to potential problems with agile and DevOps methodologies. “You solve the smaller problems, but then you build these large integrated systems where the large problems aren’t visible until you hit scale,” says Marshall Van Alstyne, a professor and chair of information systems at Boston University’s Questrom School of Business.

Meanwhile, the breakdown of siloes between business units and IT can add to the risk of project failure as business unit executives embrace technologies and seek to capitalize on the latest and greatest, regardless of whether they fully understand or thoroughly vet their options.

Consider that more and more technology spend flows from business unit budgets rather than IT coffers, says Chris Curran, a PwC principal and chief technologist for the US firm’s advisory practice. He notes that PWC’s 2015 Global Digital IQ Survey found that 68 percent of spending for technology falls outside of the IT budget.

“That has encouraged leaders to do things rapidly and as focused as possible, and they’ll go out and engage SaaS and third-party vendors and cloud. Cloud and SaaS have both increased speed and the access to new technologies, but they have also contributed to project failure.

“Business leaders go out and engage these products directly but then realise they need access to enterprise data or other parts of the enterprise IT infrastructure that they didn’t know they would need, so the project gets stalled or detailed or canceled or re-scoped,” he observes.

He notes that that PwC has also found in other surveys over the past three years that the No. 1 reason cited for project failure is an “inflexible or slow process.” Other top reasons are the lack of a skilled team and problems with third parties.

Furthermore, while improvements in IT infrastructure, particularly in hardware, help reduce the risk of catastrophic failures, there are still legacy tools, technical debt and manual processes where mistakes small and large could lead to big-time failures as projects come online.

In the end, it’s important to remember that whatever the reason for the failure when it comes to projects that involve technology, IT is likely to get the blame if it doesn’t work out, whether it’s fair or not.

“Usually it’s IT that ends up holding the bag at the end of the day,” Stanger says, “which is why IT has to be very careful.”

CIO: 

You Might Also Read:

IT Jobs Are Bound For Extinction:

IoT Is Becoming A Nightmare For IT: