Background
What was to be the world’s largest automated airport baggage handling system, became a classic story in how technology projects can go wrong. Faced with the need for greater airport capacity, the city of Denver elected to construct a new state of the art airport that would cement Denver’s position as an air transportation hub. Covering a land area of 140 Km2, the airport was to be the largest in the United States and have the capacity to handle more than 50m passengers annually. A mechanized baggage system is at the heart of the New Denver Airport, as for all major new airports. In the case of Denver, this was to be something unique: the "Integrated
Automated Baggage Handling System", originally designed to distribute all baggage including transfers automatically between check-in, the aircraft and pick-up on arrival. Unfortunately, massive problems plagued this automated baggage system. Consequently, the New Denver Airport did not open in October 1993 as scheduled. After missing later opening dates in April and May 1994, the Airport seems as of January likely to be open in March 1995.
The delay would then be around 16 months. Expenditure to maintain the empty airport and interest charges on construction loans cost the city of Denver $1.1M per day throughout the delay. When opening day finally arrived, the system was just a shadow of the original plan. Rather than automating all 3 concourses into one integrated system, the system was used in a single concourse, by a single airline and only for outbound flights. All other baggage handling was performed using simple conveyor belts plus a manual tug and trolley system that was hurriedly built when it became clear that the automated system would never achieve its goals. Although the remnants of the system soldiered on for 10 years, the system never worked well and in August 2005, United Airlines announced that they would abandon the system completely. The $1 million per month maintenance costs exceeded the monthly cost of a manual tug and trolley system.
Design of the automated baggage system
The fully automated baggage system originally planned for the New Denver Airport was unique in its complexity, its novel technology, and its anticipated capacity. It was designed to deliver each bag, including transfers, individually from check-in or the unloading of the aircraft to the outward bound aircraft or baggage reclaim. “The delivery mechanism consists of about 9 km of conveyors and over 27 km of track on which circulate 4000 individual, radio-controlled carts, the so-called "destination coded vehicles"” (Auguston 1994) . The capacity of each track supposed to be 60 “destination coded vehicles” per minute.
Speed in handling baggage is critical to achieving acceptable boarding and transfer times at Denver, since the distances are much greater those at other airports. The space between the midfield concourses provides for two taxiways (one is standard) between the tails of the aircraft parked at the concourses, and the terminal building in which passengers check-in and pick up their bags is separated from the first concourse by an office block, a garage, and the Customs and Immigration facilities. Speed has been considered crucial to the commercial success of the New Denver Airport, which the owners have marketed to the airlines as a highly efficient platform for hubbing operations because of its multiple parallel runways and prospective ability to turn around aircraft flights very rapidly.
Despite the central importance of the automated baggage system, its design was largely an afterthought. This is a common practice, unfortunately. The Denver system was detailed well after the construction of the airport was under way and only about two years before the airport was to open. Being late, the design was thus subject to two important constraints. First, the geometry was tight. The automated system had to fit within the confines of the airport passenger buildings and the underground tunnel connecting the concourses and the terminal; in many instances it was shoe-horned in at considerable inconvenience. Second, the schedule was tight. The system was to be implemented within 21 months, since Denver executed the contract only in January 1992. This schedule precluded extensive simulation or physical testing of the full design.
Remarkably, the design of the fully automated baggage system at Denver did not include a meaningful backup system. The planners provided neither a fleet of tugs and carts that could cope with the level of baggage expected, nor even access roads between the check-in facilities and the aircraft.
Denver International Airport (DIA) Baggage System Development Timeline
Nov 1989 Work starts on the construction of the airport Oct 1990 City of Denver engages Breier Neidle Patrone Associates to analyse feasibility of building an integrated baggage system. Reports advises that complexity makes the proposition unfeasible Feb 1991 Continental Airlines signs on and plans on using Denver as a hub
Jun 1991 United Airlines engages BAE Systems to build an automated baggage system for Concourse A. BAE was a world leader in the supply, installation and operation of baggage handling equipment
Summer 1991 Airport’s Project Management team recognizes that a baggage handling solution for the complete airport was required. Bids for an airport wide solution are requested Fall 1991 Of the 16 companies included in the bidding process only 3 respond and review of proposals indicate none could be ready in time for the Oct 1993 opening. The 3 bids are all rejected Early 1992 Denver Airport Project Management team approach BAE directly requesting a bid for the project Apr 1992 Denver Airport contracts with BAE to expand the United Airlines baggage handling system into an integrated system handling all 3 concourses, all airlines, departing as well as arriving flights. In addition system is to handle transfer baggage automatically. Contract is hammered out in 3 intense working sessions Aug 1992 United Airlines changes their plans and cuts out plans for the system to transfer bags between aircraft. Resulting changes save $20m, but result in a major redesign of the United Airlines portion of the system. Change requests are raised to add automated handling of oversized baggage and for the creation of a dedicated ski equipment handling area Sep 1992 Continental requests ski equipment handling facilities be added to their concourse as well Oct 1992 Chief Airport Engineer, Walter Singer dies. Mr Singer had been one of the driving forces behind the creation of the automated baggage system Jan 1993 Change orders raised altering size of ski equipment claim area and adding maintenance tracks so carts could be serviced without having to be removed from the rails Feb 1993 Target opening date shifted from 31 Oct 93 to 19 Dec 93 and soon thereafter to 9 Mar 94 Sep 1993 Target opening date is shifted again, new target date is 15 May 1994 31 Oct 1993 Original target for opening 19 Dec 1993 Second target for opening Jan 1994 United Airlines requests further changes to the oversize baggage input area 9 Mar 1994 Third target for opening Mar 1994 Problems establishing a clean electrical supply results in continual power outages that disrupt testing and development. Solution requires installation of industrial filters into the electrical system. Ordering and installation of the filters takes several months Apr 1994 Airport authorities arrange a demonstration for the system for the media (without first informing BAE). Demonstration is a disaster as clothes are disgorged from crushed bags Apr 1994 Denver Mayor cancels 15 May target date and announces an indefinite delay in opening May 1994 Logplan Consulting engaged to evaluate the project 15 May 1994 Fourth target for opening May 1994 BAE Systems denies system is malfunctioning. Instead they say many of the issues reported to date had been caused by the airport staff using the system incorrectly Aug 1994 System testing continues to flounder. Scope of work is radically trimmed back and based on Logplan’s recommendation airport builds a manual tug and trolley system instead Aug 1994 City of Denver starts fining BAE $12K per day for further delays 28 Feb 1995 Actual opening Aug 2005 In order to save costs the system is scrapped in favour of a fully manual system. Maintenance costs were running at $1M per month at the time.
Key decisions that led to disaster
Key decision 1: a change in strategy
At the start of a project strategic decisions are made that set the project’s direction. In the Denver international airport case, a strategic error was made that resulted in “flip-flop” being made part way through the project. Prior to requesting bids for an integrated system in the summer of 1991, the airport’s Project Management team had assumed that individual airlines would make their own baggage handling arrangements. United Airlines had indeed proceeded with their own plan by engaging BAE (Boeing Airport Equipment Automated Systems Incorporated) directly. Continental Airlines had however not made any arrangements and given that the airport was not yet fully leased out, other sections of the airport were not being addressed. In the summer of 1991, the airport’s Project Management team changed their strategy and realised that if an integrated system was to be built, they needed to take responsibility back from the individual airlines and run the project themselves. This change in strategy came a little more than two years prior to the airport’s planned opening date and the timing of the decision was in large part the trigger behind the excessive schedule pressure the project was exposed to. In broader terms, the mistake made was a failure to link the airport’s overall strategy (the goal of having one of the world’s most efficient airports) with the sub-strategy of how to build the baggage system.
Key decision 2: the decision to proceed
Both the airport Project Management team and BAE decided to proceed with the full scale project despite clear indications that there was insufficient time left for the project to be completed successfully. Prior to entering into the BAE contract, there were at least three indications that the project required more than two years or was simply not feasible
- “The 1990 Breier Neidle Patrone Associates report indicated the complexity was too high for the system to be built successfully” (Neufville 1994)
- “Analysis of the three bids received indicated that none of the vendors could build the system in time for the October 1993 opening
- “Experts from Munich airport advised that the much simpler Munich system had taken 2 full years to build and that it had run 24 / 7 for 6 months prior to opening to allow bugs to be ironed out”(Cal Poly & M. Schloh 1996)
Reports indicate that the decision to proceed was based on the communications between the airport’s Chief Engineer (Walter Slinger) and BAE’s Senior Management team. While BAE had initially chosen not to bid for the airport wide contract, the rejection of the three official bids resulted in the airport team speaking directly to BAE about the possibility of expanding the United Airlines system that was already under development. Demonstration of the prototype to is said to have been the factor that convinced Slinger that the system was feasible. “Despite the fact that BAE was talking directly to Slinger about the possibility of building the system, some reports indicate that within BAE several managers were voicing concern. Again the issues related to whether or not it was feasible to build such a large system in such a short period of time. Reports indicate that several managers advised the BAE Senior Management team that the project was at minimum a four year project, not a two year project” ( Cal Poly & M. Schloh 1996)
Key decision 3: schedule, scope and budget commitments
The schedule, budget and scope commitments a team enters into are amongst the most critical decisions they will make. The seeds of project success or failure often lie in the analysis that goes into making those decisions and the way such commitments are structured.
In the DIA case, BAE committed to deliver the complete system under a fixed scope, schedule and budget arrangement.
The decision to give a firm commitment to scope, schedule and budget transferred considerable risk onto BAE’s shoulders. This move indicates strongly that those in the highest level of BAE’s management structure had completely failed to recognize the level of risk they were entering into. Had they been more aware, they almost certainly would have taken steps to limit the risk and to find ways to limit the scope to something that was more achievable in the time available
Again the finger prints of excessive schedule pressure can be seen in the commitments BAE entered into. The contractual conditions for the agreement and the scope of work were hammered out in just three “intense” working sessions . Although BAE had some level of understanding because of their contract with United Airlines, clearly the three working sessions will not have provided sufficient time for the different parties to develop an in-depth understanding of what was involved or for them to fully understand the risks they were taking.
BAE and the airport Project Management team made another major mistake during the negotiations. Although the airlines were key stakeholders in the system they were excluded from the discussions. Excluding stakeholders from discussions in which key project decisions are made is always a losing strategy. When previously excluded stakeholders are finally engaged, they usually ask for significant changes that can negate much of the previous work done on the project.
Key decision 4: acceptance of change request
Not surprisingly, as the project progressed the airlines did indeed ask for a number of significant
changes. Although in the original negotiations, BAE had made it a condition that no changes would be made, the pressure to meet stakeholder needs proved to be too strong and BAE and the airport’s Project Management team were forced into accepting them. Among the major changes were; the adding of ski equipment racks, the addition of maintenance tracks to allow carts to be serviced without being removed from the rails and changes to the handling of oversized baggage. Some of the changes made required significant redesign of portions of work already completed.
Accepting these changes into a project that was already in deep trouble raises some further troubling questions. Did the team fail to understand the impact the changes might have? Did they fail to recognise how much trouble the project was already in? Although answers to those questions are not available from the public record, the acceptance of the change requests again hints at the communications disconnects that were occurring inside the project. Clearly some of the people involved will have understood the implications, but those voices appear not to have connected with those who were making the overall decisions.
Further failure points
Risk management failures: The project encountered a number of major technical problems for which no allowances had been made. One of the most significant was caused by the fact that the electrical system suffered from power fluctuations that crashed the system. The resolution to the problem required filters to be built into the electrical power system to eliminate surges. Delivery and installation of the filters took several months, during which time testing was severely constrained. Such issues were likely predictable had the team focused on risk management activities. Again possibly as a result of the schedule pressure under which they were working, appropriate risk management strategies appear not to have been developed.
Leadership change:
In October of 1992 Walter Slinger died. Singer was the system’s de facto sponsor and his death left the project without much needed leadership. According to reports, Mr Slinger’s replacement lacked the in depth engineering knowledge required to understand the system. In addition the replacement manager retained their prior responsibilities and hence was stretched to the limit.
Architectural and design issues:
A number of reports indicate that the design the team chose to use was particularly complex and error prone. Among the issues noted;
- The system had more than 100 individual PCs that were networked together. Failure of any one of the PCs could result in an outage as there was no automatic backup for failed components,
- The distributed nature of the design (with PCs dotted around the different concourses) added to the difficulty of resolving problems when they arose,
- The system was unable to detect jams and as a result when a jam occurred, the system simply kept piling up more and more bags making the jam that much worse.
Again schedule pressure may well have been a factor in the design problems. When under excessive schedule pressure teams often settle for the first design they think of. In addition schedule pressure often forces teams to focus on the “happy path” design while spending little time thinking through how to deal with problems and how to make the system fault tolerant
The enormous size of the project: Three other airports have this kind of system at much smaller scales, San Francisco International Airport United terminal, Rhein-Main International Airport in Frankfurt, and Franz Joseph Strauss Airport in Munich. The DIA system would be 10 times larger and would use 12 times as many telecars as in San Francisco. The Frankfurt system runs on trays and conveyor belts rather than high-speed telecars and is three times smaller in size. Munich’s is far less complex. The DIA system would require 300 486-class computers, a Raima Corp. database running on a Netframe Systems fault-tolerant NF250 server, a high-speed fiber-optic network, 14 million feet of wiring, 56 laser arrays, 400 frequency readers, 22 miles of track, 6 miles of conveyor belts, 3,100 standard telecars, 450 oversized telecars, 10,000 motors, and 92 PLCs to control motors and track switches. Denver Major Webb once said: “This project is of the same magnitude as the Panama Canal or the English Channel Tunnel.” (Schloh, 1996)
Conclusion
Successful projects are projects in which people make effective decisions and making effective decisions require a number of ingredients. Chief among those ingredients are knowledge and expertise. Walter Slinger, the airport’s Project Management team and even the BAE’s Senior Managers did not have prior experience of a system of this scale. In addition, given that automated baggage systems were relatively new, even BAE’s Senior Management team only had a limited understanding of what was involved. That lack of knowledge, combined with the fact that expert advice was routinely ignored, is the epicentre of the failure. The initial
planning decisions, the decision to proceed with one airport wide integrated system and the firm contractual commitments to scope, schedule and budget all represented decisions that were made by people who lacked the necessary knowledge. The misjudgements resulting from those decisions were the sparks that ignited the fire.
We are often faced with situations in which we lack the prior experience to know how to proceed with certainty. The way in which we respond to those situations can spell the difference between success and failure. The first step lies in recognizing the situation and Slinger, The Project Management team and BAE’s Senior Manager seem to have fallen at that first hurdle. Had they recognized their lack of knowledge and the uncertainty they were facing, they could have taken a number of steps that would have reduced the risk. Chief among those steps would have been listening to those who did have the necessary prior knowledge.
The bright side of the story is that in Feb 1995 DIA did eventually open and despite using a largely manual trolley based system the airport proved to be an operational success . Fears that a manual system would be too slow to service an airport the size of DIA proved to be unfounded.