Project management and politics are once more in the news. This is partly due to it being August and a slow news week, hence the appearance of thinly-disguised listicles and boilerplate about why government projects tend to fail. Plus ça change. Ahead of the 2010 general election, and on the back of another list of government IT failures, David Cameron "signalled a move away from big IT projects, suggesting he will use technology to increase the transparency of government". In light of the NSA and GCHQ revelations, as much as the Universal Credit debacle, this is ironic.
IT projects regularly fail in both the private and public sectors. The supposed higher rate of failure in the latter is attributed to the extra complication of regime change (at both government and ministerial level) and the conflicting interests of politicians and civil servants (the Yes Minister trope). In fact, there is no good evidence that there is a higher rate of failure in the public sector. What is known is that the private sector is better at obscuring failure and redefining success. Conversely, public sector projects are (with a few exceptions) publicly audited and their success or failure is often a matter of political judgement, so there is usually an opposition willing to challenge the government's verdict. The Obamacare rollout, which for a while looked like the epitome of big government project failure, has dropped out of the headlines in the US and lost its political charge due to its gradual success, as could have been predicted.
The main reasons why IT projects fail are also well known, because they are common to most projects and have nothing to do with technology. It's mostly about people, and a little bit about process: a lack of genuine commitment to change, poor leadership, not involving the right people, poor supplier management, a failure to meaningfully define success, a failure to know when to stop or change course. The mitigations and contingencies for these weaknesses are also well known, and widely ignored. The root problem is that many projects are faith-based. The earliest records of project management we have are religious works. For example, the Old Testament is one project after another: build an ark, find the promised land, build a temple.
Project management is an attempt to provide a rational framework for goals that are too often inspired and driven by emotion. It is no accident that words such as "mission" and "vision" feature prominently in the language. The problem that public sector projects face is not that government is incompetent, but that these projects tend to be more emotionally charged, both in terms of their impacts and ambition. Replacing payroll system A with payroll system B may lead to tears and tantrums in one company, but this is trivial in comparison with the change to a national benefits system that will supposedly transform shirkers into strivers.
Many government IT projects are initiated for gestural reasons. This is not just about ministers being seen to be doing something, but about articulating and concretising a political worldview, a practice as old as the pyramids. This truth is usually ignored by project management "experts". For example, suggesting that UC could be better implemented using an incremental approach, avoiding a risky big bang and allowing for gradual adaptation, makes perfect sense technically but ignores the political importance of the scheme, which goes well beyond "making work pay". The commonly-understood goal is to cut the welfare bill (officially through greater efficiency and reduced fraud, unofficially through increased harrying of claimants). The gestural purpose is to introduce the idea of a citizens' basic income. It is the all-or-nothing approach, and IDS's associated zealotry, that constitute the symbolic message to the electorate: everyone must work, the poor must be subsidised (due to the incontestable needs of the market), and complexity leads to fraud. A low-key, gradual improvement in process efficiency is politically irrelevant.
This emotivism is a double-edged sword. Just as politicians will happily lead a project to failure so long as it continues to reflect the desired policy stance, so public opinion (or at least the media) may deem a project a failure because of unreasonable expectations. The e-Borders system has "failed" because it is popularly assumed to be a tool for "controlling immigration", which is not something that can be effectively done at passport control (and has arguably been a non-issue for some time now). As a policing tool - i.e. intercepting criminal suspects - it appeared to work fine. The downgrade to the "Go home or face arrest" vans earlier this year was an equally pointless gesture aimed squarely at xenophobes attracted by UKIP and a demanding press, not at people who had overstayed their visas.
Government also has a role as an advocate of technology and (increasingly) the "power of information" to both business and the wider population, hence the e-government frenzy that started in the late 90s. This partly explains its reluctance to consider non or low-tech solutions for public services (e.g. the insistence that job seekers must do online job searches, despite the well-known problem of fake jobs on job boards) and also partly explains the attractiveness of government as a client for large solution providers. But the higher risk of perceived failure (or at least the higher profile given to failures in the media as evidence of ministerial incompetence, big government ineptitude and supplier abuse) makes this a costly strategy.
Clearly then, the subjective rewards (or "soft benefits" in project management-speak) must be of considerable value to politicians. I think there are three worth noting. First is decisiveness. Launching a project (or any sort of initiative) is the end-game for many politicians who fear their ministerial tenure may be short. Unless you can arrive late and take credit for a successful implementation (e.g. Boris Johnson and the London Bike Scheme), few want or expect to be around at project end (IDS may have had a stay of execution till 2015, but he is likely to quit the scene long before UC exits the pilot stage, if it ever does).
Another attraction is that government IT projects encapsulate the neoliberal idea of managerialism: the belief that for every problem there is a solution that is amenable to generic management skills. The project management process itself encapsulates the tropes of measurement and monitoring, with reality easily giving way to the figment of targets and progress (the "reset" of Universal Credit and IDS's insistence that it is "on target" are examples of the lunacy this gives rise to). Similarly, e-Borders might appear a colossal waste of money, but it serves to further normalise the idea that society should be constantly surveilled and that we should be able to accurately quantify classes of citizen or resident. Even the growing role of Parliamentary committees acting as project auditors, rather than just spending watchdogs, reinforces the neoliberal trope of constant inspection.
The third benefit is the normalisation of failure, which is central to capitalism. One of the mantras of Silicon Valley, which was adopted from agile software development, is the Beckettian "fail fast, fail better". In reality, this is often little more than cant in commercial organisations, where the empirical method remains alien, while most media evangelists turn out to be self-serving "serial entrepreneurs". Ironically, the history of "the entrepreneurial state" encourages the belief that failure is intrinsic to government projects, albeit in an ultimately beneficial way. You can hear this in the words of Tony Hall, the Director General of the BBC (lumped with government when it comes to project management critiques) when he talks of the corporation's role: "We must be the risk capital for the UK. We have got to be the people who have enough confidence to be able to say that we are going to back things that may not work".
Government is unlikely to wean itself off gestural politics so long as it depends on elections, so a certain amount of project failure is an unavoidable cost of democracy. The state's role as the risk-taker and experimenter of last resort also makes it unlikely that it will ever become (or be seen to be) a better project manager than a private sector with selective memory. Finally, managerialism is bigger than neoliberalism and central to all flavours of government. Even the "night-watchman" state of right-libertarian fantasy (which is by definition a police state) will waste money on projects equipping the military and building courthouses.
This intractable reality leads the critics of government project mismanagement down the blind alley of personal responsibility: "we should guarantee that the ministers, senior civil servants and corporate CEOs involved will all be publicly sacked if the project fails". This is close to Stalin's approach to project management (JFDI) and project failure (the gulags), but as we have seen with the banks, pinning blame in complex organisations for activities that spanned many years is difficult, and project failures rarely produce a smoking gun like an email requesting that LIBOR be rigged. The dirty truth is that big IT projects, in both the public and private sectors, carry an implicit indemnity for the participants, much as banking does for executives who stop short of actual criminality. If this weren't the case, no one of any real talent would take the job on. Despite the ample scope for fraud and abuse, big IT projects remain largely crime-free zones.