Dana Scully and Fox Mulder, the F.B.I. team in The X-Files, show the value of clinical analysis of cause and effect. While Mulder is focussed on mixing it with various aliens, shape changers, mutant worms and other wonderful creatures, Scully is often back at the base dispassionately examining the numerous bodies that turn up in each episode. It is often Scully’s understanding of pathology and autopsy that provides the vital evidence in their search for the truth that is out there.
A bit like Mulder and Scully, over the past 18 years, our group has reviewed over 20 major projects that were in the process of failing or had failed. These reviews were not done as an academic exercise or a controlled experiment but, they were undertaken “in the heat of the battle”. Our clients wanted to know what they could do to fix the projects or what could be done to prevent other projects failing. The pathology of failed projects has aided us and our clients in understanding the major issues in computing.
What our group has learnt is that there is a common set of causes for project failure, a common pattern of project degradation and failure and common symptoms of projects that are failing. Our findings are in conflict with many of the articles and books addressing project failure.
What we have found is that most projects fail because of people and project management concerns rather than technical concerns such as development methodologies, technology platforms and tools. As Gerry Weinberg once observed there are three causes of failure “People, people, people”.
The failed projects
The projects we reviewed were in most business sectors including retail banking, merchant banking, insurance, credit, manufacturing, government departments such as education, health, social welfare and defence, system software, hardware, research and development and academic institutions.
The amount of investment either lost or misdirected in the 20 projects exceeded US $ 1 billion and we have evidence that the amount could be double that figure if the effort expended by business experts [users] and unpaid overtime and additional work by team members was included.
Interestingly, only 7 of the 20 projects received any press coverage [computer or general] and each of these 7 projects lost more than $100,000,000. We found no pattern of certain business sectors being more prone to project failure or that private sector organisations had a pattern of more failures than public or government sector organisations.
A redefinition of project failure
Most definitions of project failure focus on the traditional concerns of:
- meeting requirements [data and function];
- keeping within budget ; and
- delivering within estimated deadlines.
Within this limited definition of success all 20 projects had failed however, our group adopts a more detailed and expanded definition of success.
Our definition reflects the difference between an internal or software factory view of success such as the one above and an external or added value chain or service view of success. The fact that a project has met the requirements [objectives], budget and deadlines is simply a measure of the internal development process effectiveness and is not a measure of the product or added value effectiveness. We have reviewed a number of other projects that were considered successful [in terms of meetings objectives, budget and deadlines] by the IT development teams and management, yet these projects had not delivered any sustained business benefits or added value for the clients. In our terms [and in the business client terms] these projects had failed.
In addition, other projects we have reviewed had met the data and functional requirements, on time and in budget but with substantial degradation of the expected quality. Typically, the systems were not tested adequately, documentation and training was not delivered, screen design and usability was compromised and security was non existent. It is our group’s experience that covert degradation of project quality to meet deadlines and cost constraints is typical in the high-pressure and competitive business environment of the 1990’s.
As observed by Capers Jones  and Thomsett  many IT projects covertly “blow” budgets through unpaid and unrecorded hard work . Our research confirms Caper’s findings that projects are typically 30-50% over cost as a result of unplanned evening and weekend work and that the formal project cost tracking systems often do not record this cost. Our group has also identified similar covert cost “blowouts” in the effort required by business people in project-related activities such as requirements specification, testing, conversion of data, documentation and job redesign. As a result, many of the successful projects we reviewed had indeed failed to meet the estimated budget though the formal cost tracking process indicated that they had met budget.
Finally, our reviews revealed another critical, but often ignored, measure of project success. In all of the 20 major failed projects and in many of other successful projects we reviewed, team members were burnt out and demotivated. One project we reviewed had 90% turnover of team members in one year. The long-term cost to the organisations of key professionals who are so dissatisfied with how the project was or is being managed that they have “turned off”, lowered their productivity or, in extreme cases, have left the organisation is extremely high and in some cases had negated the benefits to the organisation of the project. The hidden long-term cost of demotivated professionals on other projects that they will undertake is rarely discussed in typical analysis of project risk and failure [see Jones, American Programmer, March 1995].
In other words, for a project to be successful, it must meet all the following criteria:
- satisfies stakeholder groups;
- meets requirements;
- meets quality expectations/requirements;
- within cost [paid, unpaid and business expert costs];
- within deadline;
- delivers sustained and actual benefits; and
- provides the team with professional satisfaction and learning.
Clearly, if these criteria were applied to IT projects, very few are successful. In our reviews, according to our criteria, an additional 20 “successful” projects were in fact failures!
In many ways, most analysis of project failures reflects one of the most serious issues in contemporary computing. Popular metrics and measures [for example Function Points and the S.E.I. C.M.M. model] are focussed on the internal efficiency of delivery [or the system development process] not the effectiveness of the delivered service [or product] in business terms. For example, the questions that metrics and process improvement models should be addressing are not only “How many function points were delivered and what was the development cost per function point?” but also “What was the added value or business effectiveness of the delivered function points?”.
Our expanded definition of project failure reflects this effectiveness view. Simply, a successful project must add tangible value to the organisation.
An interesting finding from the morgue
One of the most interesting and surprising findings from our reviews was that most organisations have a very poor understanding of how much money and effort was either lost or being placed at risk in the projects under analysis. We consistently found that a combination of unpaid hard work, un-costed business and other stakeholder effort and misdirected equipment acquisition coupled with poor project costing and tracking systems substantially understated the real costs of the project and lead to a delaying the recognition of the failure in progress.
For example, in one project, the hours expended by the computer team were being recorded and charged back at on-costed rates of $100.00 per hour. None of the hours spent by business experts in attending J.A.D. and other specification sessions, reviewing deliverables, testing the system and so on were not tracked and costed at all and the costs and time incurred by related specialist stakeholders such as Q.A., communications, operations and internal audit were also ignored. An initial estimate of these “hidden” costs revealed an understatement of 100% of costs. This pattern was common in all the 20 major projects we reviewed.
Common causes of failure
As a result of our reviews and consulting, we are convinced that there are a small number of critical factors that will inevitably lead to project failure. Perhaps surprisingly to some of our readers, these factors are all management-related.
While many other experts cite inadequate technology, lack of skills, inappropriate development methodologies and strategies and lack of development support technology such as ICASE, automated estimation tools, poor quality assurance and testing as causes of project failure, we would argue that when an organisation to allows a project to commence without these issues being identified and addressed is clearly a management failure [specifically a failure of executive project management]. This view reflects a contemporary view of project management [see Radical Project Management, Thomsett, 2002] that includes senior management involvement, stakeholder or client group involvement, risk management and quality planning as integrated components of project management.
We have developed a simple taxonomy of causes of project failure that is based on the potential impact of the factor on the project.
Level 1 factors – these factors guarantee project failure [or in street jargon, they are showstoppers]. That is the project will fail to deliver quality, added value and professional satisfaction on time and in budget.
Level 2 factors – as discussed by Capers Jones [American Programmer, op cit] and others, there are a number of other factors, which may not prevent the project from delivering on time and in budget but will generally result in substantial degradation of quality, added value and professional satisfaction.
In many ways, Level 2 factors are the direct result of the failure of Level 1 factors. However, with hard work, great pressure and, in some cases, good luck, project teams can manage to deliver with Level 2 factors absent or poorly managed.
Level 1 Factors or you’re dead in the water
- Lack of effective project sponsor or owner
Recent surveys by The Yankee Group and Deloittes and Touche [Johnson, 1995] on the high failure rate of business process reengineering projects report that lack of senior management commitment as a major factor in the failures. As discussed by Thomsett [1993, op cit], the contemporary role of project sponsors and steering committee members goes far beyond the traditional passive roles of project approval and review. It is critical that senior business executives who have sponsored an IT project must be actively involved, in effect, as an executive project manager in assisting the project manager in the following areas:
- stakeholder involvement, commitment and conflict resolution;
- benefits planning and realisation;
- quality requirements planning;
- risk management; and
- project change control.
These roles are in addition to the traditional roles of approval, funding and staffing the project.
It is typical in most projects that the project manager has responsibility without authority. In other words, the project manager is not given the requisite organisational or financial power to achieve his or her objectives. This lack of organisational power is the major cause of failure and, while Caper Jones and others term this type of factor as “soft”, our experience is that the impact of this and other soft factors is hard and tangible.
For example, a major stakeholder from a client business group who is organisationally senior to the project manager requests a small scope and objectives change. Without a powerful sponsor, the project manager has little choice but to document and accept the change which will lead to an extension of deadline and budget. In another typical example, in the initial planning sessions, a senior business stakeholder agrees to give the project manager access to key business experts for defining requirements. During the project, the senior manager reallocates the priorities for the business experts and effectively makes them unavailable to the project manager. This leaves the project manager without any effective options but to slip the schedule until the business experts are available.
These are simple examples of the need for an effective and powerful sponsor as, in both cases, the project manager would be able to raise these problems with the sponsor where he or she would use their power base and authority to address the issue at an executive level.
In every one of the 20 major failed projects, lack of an effective sponsor was common. As one project manager recently put it “To manage a project without an effective executive sponsor is to visit hell on Earth”.
- Lack of stakeholder buy-in
Project stakeholders are people who are outside the project manager’s scope of control and are service-providers [such as business experts who will be specifying requirements, data base and data communications consultants, computer operations and so on] and/or service receivers [such as internal or external clients, management, etc.]. As discussed by Thomsett [1993, op cit], these people are critical in achieving project success. If a project manager cannot gain effective buy-in and support from these people, then activities such as defining scope and objectives, identifying and managing risk, benefits analysis and realisation, quality requirements definition and change control are effectively compromised.
Without consensus among stakeholders as to the scope, objectives and quality requirements of the project, the project manager cannot effectively manage the project. At best, the project manager may be able to deliver a system that does not meet the needs of the majority of his or her stakeholders and does not add value. At worst, the project will simply remain in an “out of control” state [with constant changes in scope and objectives] until it is terminated or the team leaves in frustration.
In every one of the 20 major failed projects, lack of effective project sponsorship and stakeholder buy-in was common.
Level 2 Factors or when the going gets tough ….
Without the Level 1 factors in control, the project manager will not be able to effectively address the following critical factors in ensuring project success. It is important to note that, in some cases, the project may be delivered on time and in budget but without sponsor and stakeholder support, no project can deliver successfully [using our extended definition of success]. The following issues are discussed in more detail in Radical Project Management.
- benefits planning and realisation;
Benefits analysis, planning and realisation requires participation and commitment from project stakeholders [who will generally identify and realise the benefits] and the project sponsor [who is responsible for ensuring that stakeholders are held accountable for benefits]. For example, a project that is planned to avoid costs for the organisation through reengineering of a client area and reduction of staff in that area cannot guarantee those benefits without a senior executive sponsor and stakeholder managers who can ensure that the client group alters their work processes and that the cost avoidance is managed after the project has delivered.
- quality requirements planning;
Is is normal for stakeholders/clients to have differing views on the required quality of the system being developed. For example, Internal Audit may require the system to be auditable and secure, Computer Operations may want the system to be efficient, one stakeholder/client group may want the system to be user-friendly and another stakeholder/client group may be more interested in the reliability of the system. It is almost impossible for a project manager to resolve these differing quality expectations without some commitment and participation of the various stakeholders and the project sponsor in resolving these conflicts.
- risk management;
Contemporary project risk management [American Programmer, September 1992 and March 1995] not only examines and manages the risks inherent in the complexity and size of the proposed system but also includes risks associated with the project team and the stakeholder environment. Clearly, a project without effective sponsorship and stakeholder buy-in is exposed a number of high risk factors which, unless managed, will lead to project instability and delays.
- project change control.
The management of the inevitable changes in scope, objectives, quality requirements, risks, project staffing and so on require the project manager and his or her team to be able to negotiate with the various stakeholders impacted by the changes. As discussed earlier, a senior manager may ask the project manager to expand the project’s scope and objectives. This change leads to an extension of the project’s deadline that has a negative impact on another project manager who has planned to use some of the first project’s people on her project. In addition, the extension of the deadline delays a series of benefits that another stakeholder group has expected by a certain date. This ripple effect is common in all projects.
The dilemma that our project manager faces is simple but difficult. Does she agree to the senior manager’s request and harm her relationship with the other impacted stakeholders or …? A committed and powerful sponsor and supportive stakeholders will be able to apply pressure to the senior manager to withdraw or modify his request or will be able to support the project manager in minimising the impact of the new deadline.
Patterns of failure
As for most organisms, a project exhibits a predictable set of stages [with specific symptoms] during failure. To many people, a failing project simply degenerates quickly into a chaotic rabble of stressed people working long hours. In fact, this is usually not the case. In most of the projects we examined the following degradation pattern was observed.
Stage 1 – that will happen in Release 2
The project is submitted and approved as a monolithic or waterfall development strategy with the team planning to work on the total requirements, for example, Functions A,B,C & D. However, soon into the project it becomes apparent that the estimates are wrong and, in addition, a new requirement for Function E is added by a senior manager. Without any re-approval by the project sponsor and stakeholders, the project manager and team change the development cycle or strategy to a sequential release. That is, they unilaterally de-scope the project and continue to work on Functions A,B,C and E with Function D to be scheduled as an “enhancement”. It should be noted that in this Stage formal planning is beginning to collapse and the original Business Case and project plans remains unchanged from the original plan.
Stage 2 – I need more people
In many projects, Stage 1 simply delays the inevitable and further estimation errors, risk escalation or scope expansion leads to a realisation by the project manager that even with the de-scoping in Stage 1, the team cannot deliver Functions A,B, C and E [remember that the client and sponsor are still expecting D as well]. This Stage involves the project manager requesting additional people who are allocated to work on concurrent sub-projects [for example, some team members working on Function A and B and the new team members working on C and E]. Project planning has become informal and reactive and the risk has been raised by the adoption of covert concurrent development.
Stage 3 – who’s not working this weekend?
By this stage any small change [a specification change, a key team member leaving and so on] will usually result in the project further degrading into a covert quick and dirty strategy. Typically, the project manager and team begin to panic and unpaid overtime work becomes the norm in an effort to keep the project “in control”. In addition, development effort that is perceived to be unimportant such as documentation and testing is abandoned. A bunker-type mentality begins to emerge with the team seeing themselves as heroes and heroines struggling against great odds and any team members who express doubt about the project are treated as outcasts or negative people. It is important to note that the project is already close to complete failure at this Stage.
Stage 4 – we’re all crazy now
Finally, the project slips from an semi-organised quick and dirty strategy into a bizarre form of group-think that our group calls a “lemming rush”. In this Stage, all team members and the project manager have effectively lost control of the project. Cognitive dissonance begin to deny the desperate state of the project. Team members are often working 7 days a week and, in some cases over 10 hours per day. There is no project plan and all work is allocated verbally. Interestingly, many people in the broader organisation also deny the reality of the project and, in an example of the Bay of Pigs phenomenon, no one says anything as no one else is saying anything. This denial was well evidenced in a project where all team members and the project manager felt that the project could meet a deadline 3 months out and that our review was a waste of time. In one day, we identified that the project was over 12 months behind schedule and would deliver [at best] in 15 months not 3! Our findings were wrong – the project finally delivered 18 months later. By this stage, the project is totally “out of control” and can remain in this state for a long period. In effect, the team no longer understands the objectives or other project management issues – like lemmings -they are blindly migrating to the deadline as that is all that matters.
In all cases, the project is running two sets of plans – the plans that the Project Sponsor and stakeholders are given and the plan that the team is actually following [covertly].
We observed this pattern of failure in every one of the 20 major projects we reviewed. Fifteen of the 20 projects we reviewed had degraded to Step 4 and the remaining 5 were at Step 3.
Symptoms of failure
Like all organic failures, projects exhibit clear symptoms as the process of degradation proceeds. Lack of an agreed Business Case [scope, objectives, benefits, costs, risks, quality agreements and so on] and stakeholder commitment such as formal stakeholder agreements before commencement are obvious signs of a project that will fail. However, more subtle signs may emerge later.
- lack of project plan and Business Case updates
As discussed by Thomsett, Jones and others, most projects are subject to change and, as a result, the original Business Case and the associated project plans should be subjected to clear and managed project change control. The absence of a series of changes approved by the Sponsor, Steering Committee and stakeholders is a potential sign of communication breakdown between the project manager and sponsor.
- lack of stakeholder communication
As discussed by Thomsett [1993, op cit] in a well managed project, stakeholders are involved in planning the project and are regularly briefed on the progress of the project. Poor or no stakeholder communication on a regular [at least monthly] is a potential sign of failure.
- no external involvement in quality assurance
All formal quality assurance processes involve some independent external person being involved in detailed technical reviews of the deliverables and the project development and management process. As a project begins to fail, it looks inward and external reviews are avoided.
- excessive hard work
Teams that are motivated often work long hours. However, in a failing project, the working of long hours is not voluntary and, indeed, becomes expected and the norm. In general, a team working more than 60 hours a week for a sustained period is in serious trouble [see following signs].
- high staff turnover
Many people will respond to the challenges offered by a poorly-managed project with energy and enthusiasm. However, the impact of sustained and unnecessary pressure over a sustained period on personal standards and private lives results in many of the best people [i.e. those with other employment options or life choices] leaving the project.
- aggressive and defensive behaviour
In projects that are terminally-ill, any attempt by outsiders [managers, consultants, internal audit and other project managers] is met by a combination of aggressive and defensive behaviour. The bunker mentality of the team and the group cognitive dissonance effectively bonds the team into a delusion that the project will succeed and any indication by people outside the team that it may be in trouble is seen as a threat.
- no fun!
While some theorists may dismiss this symptom as irrelevant, our group’s experience is that well-run projects offer challenge, learning and enjoyment for team members. A well-managed project can stop and smell the roses. For example, regular team time-outs where planning, review and fun can be combined with team-building. Failing projects have no fun, a lot of challenge and a lot of bad learning. Simply, instead of fun and excitement, failing projects exhibit frustration and desperation.
The failing project test
Finally there is a clear and easy test for failing projects:
If a project is failing, a request to stop the project for formal planning sessions and time-outs will be met with the clearest indicator of a fatal disease:
“We can’t stop the project for planning, we have a deadline to meet !”
At this point, you may need to call in Mulder and Scully.