This paper presents an experimental survey conducted to examine how students act in a problematic computing project situation. We hold that the majority of all computing projects undertaken have problems of some kind . Whereas there are many different types of computing project problems, this research is about those computing projects that never seem to end–they become runaways . This is a dangerous phenomenon costing organizations millions of dollars each year; both in direct losses and missed opportunities. A project can be described as a runaway when more and more resources are invested despite information indicating that the project probably will fail in meeting the expectations of the stakeholders. In this paper it is assumed that it is possible to denote a project as a failure, which is not always apparent . However, we claim that the awareness of runaways are crucial for both information technology experts and end users who often are central actors in systems development projects.

A runaway project is an escalation situation and can be described in the following framework: All escalation situations entail some loss or cost as a result of an original course of action, the situations involve some continuity over time, a simple withdrawal is not an obvious solution, the decision maker must have a real choice in deciding whether to persist or withdraw, uncertainty of goal attainment and unambiguous feedback from previous decisions made. Escalation situations emerge through a complex compound of psychological, social, organizational, and project determinants.

Experimental research similar to the research presented in this paper has been done in the US . In this research, a scenario of a computing project was presented to two groups of undergraduate computing majors who were asked to act as project managers. As the scenario of the project progressed it experienced problems. The task of the subjects was to identify the problems and recommend to top management how to proceed with the project. The responses in the survey were analyzed through the escalation framework. The paper concludes that though most of the subjects stated that the project was in trouble, they were convinced that the right decision was to invest more resources and let the project continue. From the result of the study, we argue that runaway computing projects must get more attention in computing education. We do note that most organizational behavior courses, and similar courses given at for instance business schools, do include aspects of runaways and escalation. However, we question current traditional pedagogical principles of knowledge transfer as an appropriate mode of creating an understanding and eventually teaching how to avoid runaways. The paper closes with some suggestions on how escalation situations can be integrated in computing education.

The remainder of the paper is organized in the following six sections: The first section "Escalation situations" explains and discusses different aspects of escalation. The following section, "Subjects and research design" presents the subjects and the scenario used in this research. The next section, "Analyzing the survey" applies the escalation framework from section two to analyze the survey. The analysis is followed by a discussion section. In section six educational practice is discussed in the light of the findings in our study. Finally, in "Concluding remarks and further research" we show some practical implications for computing education and outline our further research.

Research on escalation situations during the past 20 years have mostly been experiments in psychology and sociology under the headings: knee deep in the big muddy , entrapment , too much invested to quit , escalation of commitment , knowing when to pull the plug , throwing good money after bad . The escalation research has only recently moved into a computing context . However, different types of failing projects and remedies against failures have been discussed and suggested since the beginning of computing . In the following we describe escalation situations and explain how they emerge.

An escalation situation can be thought of as a situation where decision makers have continued commitment to a specific course of action despite information suggesting that the course of action is failing , and even invest more resources . Brockner elaborates this further by arguing that an escalation situation is continued commitment in the face of negative information about prior resource allocations coupled with "uncertainty surrounding the likelihood of goal attainment." Decision-makers become locked into an escalation situation through what Staw calls a "syndrome of decision errors." Bowen criticizes this and argues that commitment to a further investment occurs because of the equivocality in the situation and not because of an over-commitment to a failed decision. He continues that one can not "technically" err in an ill-structured decision situation.

Commitment, as the central concept in escalation situations, has as such been studied from so many different theoretical perspectives that some argue that the concept should be abandoned in favor of a set of terms . For instance, commitment has been described as: the state of mind that holds individuals in a line of behavior , the binding of an individual to behavioral acts and an active counterforce to change . Commitment, in this paper, is not necessarily good nor bad, but the level of commitment of various individuals in a project will greatly influence the eventual success of the project. Without commitment you really do not have a project.

When commitment induces a person to complete a difficult or unpleasant task that benefits him and others, commitment is good. Obviously, without commitment the hard work required will not be done. However, when commitment leads to a fixation on a policy or behavior of diminishing benefit and rising cost, the situation is obviously problematic.

All escalation situations have characteristics that can be isolated and described in the following framework : (1) they entail some loss or cost–not necessarily monetary–that have resulted from an original course of action, (2) the situation involves some continuity over time–they are not once off affairs, but dilemmas involving ongoing courses of action, and (3) they are situations where a simple withdrawal is not an obvious solution. Moreover, (4) the decision-maker must have a real choice in deciding whether to persist or withdraw , (5) there must be unambiguous feedback from previous decisions made , and (6) there is uncertainty of goal attainment.

Not to lose the essence of how escalation situations emerge; attention should be shifted away from identifying the isolated antecedents of escalation situations and toward analyzing the influence of more general classes of determinants. Staw and Ross propose a model for analyzing and understanding the emergence of escalation situations in which they suggest four abstract classes of determinants for escalation situations: project determinants; psychological determinants; social determinants; and organizational determinants (figure 1). We will discuss each of these determinants below.

Figure 1: Emergence of escalation situations

The effect each of the determinants will have on different stages of the project is discussed in e.g., , but further research is needed. Also, the interrelationships among the four determinants needs to be further examined. However these two issues are beyond the scope of this paper.

Project determinants are the objective attributes of a project–mostly economic, such as the project’s benefits and costs . A project is likely to be continued with high commitment even when it is facing problems, if the project is perceived as a long-term investment, expected to have a large payoff, and/or have a long-term payoff structure . High commitment is also likely when closing costs are high and salvage value is low .

Psychological determinants cause individuals to see situations from a promising and optimistic view . They explain, to some extent, managers’ unwillingness to admit that an earlier decision was wrong . Underlying psychological theories explaining escalating situations are according to : self justification theory–when an individual desires to demonstrate rationality to himself–and prospect theory–when individuals exhibit risk averting or risk seeking behavior depending on how a problem or decision situation is framed. "Throwing good money after bad" in an attempt to turn around a failing situation, the so-called "sunk cost" effect, is another example .

Social determinants hold the individual to a course of action regardless of the individual’s own beliefs. Examples are face saving and external justification . Social comparison theory posits that people are concerned with evaluating the appropriateness of their attitudes and behavior. People regard the behavior of others as a model for their own behavior and this occurs when they are uncertain about the appropriateness of their own attitudes or behavior . Social determinants also involve a group’s relation to another group and a successful effort by a group may influence other groups to attempt the same approach, often referred to as benchmarking.

Organizational determinants are the structural, cultural and political environment of a project, for example, top management support, administrative inertia, and interorganizational interaction. According to Keil projects are more prone to escalate when there is strong political support and when projects become institutionalized. Institutionalization occurs when a project is tied integrally to the values and purposes of the organization, and when actions are taken for granted because they are so deeply imbedded in the subculture or norms of the organization. Long-standing programs and lines of business are not even considered for discontinuation because they are so closely identified with the organization.

Contemporary research on organizational behavior and systems development suggests that escalation situations or runaways projects can be avoided. These suggestions focuses on three aspects:

This section presents the two groups of students and summarizes the scenario presented to the subjects. The two groups come from Sweden and South Africa. The reason for this is partly practical, this is where the authors live, but also that the research presented in this paper is part of a larger research effort where the heterogeneity of the South African society and respectively the homogeneity of the Swedish society are studied in more depth. This is however not reported in this paper.

The first group of student subjects was 47 first year undergraduates in a four year computing program at a Swedish University. The researcher visited a lecture and asked the students to devote 30 minutes to a "decision making study." No extra credits or other incentives were offered. 10 of the subjects were female and 37 were male. 5 students decided not to participate in the survey. The average age was 24 years and they had an average of 3 years of work experience.

The second group was 70 second year undergraduates in a three year information systems program at a South African University. The researcher obtained permission to conduct a survey about the project described in the scenario. No extra credits or other incentives were offered. Of the subjects 26 were female and 43 were male, 1 student did not identify his/her sex. One student decided not to participate in the survey. The average age was 20 years and they had an average of 6 months work experience.

This section summarizes the scenario presented to the subjects. They were asked to play the role of a manager and make decisions about a systems development project in a large pharmaceutical company, MedPro. The scenario was originally written in English and later translated into Afrikaans and Swedish. This might have introduced differences in the text. As a comparison of the two groups was not initially the main focus of this study we did not see this as a major problem. However, retrospectively we believe it would have been more appropriate to use English as the only language for surveying the students as both groups of students have complete mastery of English.

In 1994, the research division of MedPro discovered a very effective new drug for rheumatism. The laboratory tests on the drug were completed late in 1994, and the tests on humans could begin. Testing of new drugs is a regulated task and the FDA has very strict rules for conducting tests on humans. Your systems development group has recently designed a prototype of a computer-based system that will both shorten the testing, and improve the quality of the test documentation. Therefore, the CIO at MedPro has recently granted $5 million for the development of the proposed system, TEST.

Shortening the test phase means that the rheumatism drug will be on the market earlier, and MedPro will increase their profits and they will get a large share of the market before any competitor will be able to enter the same market. The president of MedPro has indicated strong support for the project in the company magazine: "This is the kind of project our organization needs to maintain our position at the top of the market."

Four years ago, another systems development group designed a very effective system for the marketing and distribution division. Their system has, by all means, contributed to MedPro's position as an efficient distributor of drugs on the world market.

As MedPro is mainly a research organization, the organizational culture is very forgiving when it comes to errors and it is believed that it is necessary to take risks to learn, invent, and discover. You are confident that the system will be a real success and have spent a great deal of time discussing the benefits of TEST with departmental colleagues. You also bring up TEST as a subject to get the test division researchers’ opinions and suggestions. The test and manufacturing division will shorten the test phase by 50 per cent. The new rheumatism drug will be on the market two years earlier than with the old procedure. Introducing the drug on the market even one year earlier would mean an additional estimated $8 million in profit. MedPro is very dependent on information technology to maintain its position as world leader within drug development.

The analysis and specification were completed on schedule June 1995. The development phase started shortly afterward with a planned implementation date of November 1995. The test and manufacturing division has put a lot of faith in the new system and has decided to postpone the start of tests on humans until the computer system is implemented. The manual testing procedure is fundamentally different from the new computer-based procedure. It would be very costly to transform results either way.

During the first month of the TEST project, medical press announced that another pharmaceutical company had discovered an effective new substance for treating rheumatism. Their product has shown excellent results in laboratory tests but the competitor has not started tests with humans yet. However, they will start shortly.

Additional personnel should be hired and extensive training is needed to adequately staff the TEST project. Investment in new hardware is required for the TEST system. Furthermore, the TEST system should be compatible with other applications in MedPro, both existing and planned systems. The project team has also tried to ensure that TEST, after some modifications, probably can be used in administrating testing of future drugs, but this has not been fully investigated yet.

Minor problems have surfaced in the project group since the first week. Communication among the project staff has not been satisfactory due to interpersonal conflicts, which did result in a split of the project into two groups after the first month of the project. You believed that this problem would resolve itself over time, but instead it has become worse. In October, one month before the planned implementation, the project runs into severe difficulties: the program code is a mess according to an external auditor, and many of the assumptions made in the analysis phase are no longer valid. Large parts of the analysis must be redone, and at least three man months of coding must be repeated. The original implementation date will not be met. Instead, an additional three months is needed to complete the project.

Two additional programmers join the team to handle the problems, but the three months pass and the problems are still not solved. One of the most experienced programmers in the project group finds a ‘bug’ in the database manager, and assures you that he will fix the problem in one month. Writing the new code takes longer than first anticipated and the project needs an additional three months to be completed and an additional 50 per cent funding is needed to hire two top programmers to structure the code. Senior management are worried and as MedPro’s CIO is aware of the problems TEST has encountered he calls an additional meeting in late March 1996. You are asked to prepare a short presentation on your preferences for this project’s future.

In this section, the survey is analyzed using the escalation framework discussed in a previous section. The mode of analysis in this research was interpretive content analysis and the instrument applied in the survey focused on two issues. Firstly, subjects were asked to indicate their comprehension of the problems on a 7 point Likert scale ranging from ‘no problems’ (1) to ‘big problems’ (7). Secondly, they were asked if the project should be abandoned, this also on a 7 point Likert scale ranging from ‘absolutely’ (1) to ‘absolutely not’ (7). Each of these questions was followed by an open ended question where the subjects were asked to describe the problems as they saw them, and motivate their decision about abandonment or continuation of the project.

The following methodology was used for the categorization of problems and reasons for continuation for the open ended questions as identified by Weber . The basic unit used for the classification of the written answers of the students as phrases that could be a whole sentence or part of a sentence. The way in which a phrase was identified was a group of words describing a specific problem or reason for continuation. Each phrase was classified in one and only one problem group for the problem question or in one and only one determinant category for the abandonment question.

The categories into which the problems were categorized were identified after a preliminary investigation of the answers that were given by the students. Formal definitions were written for the problem categories by using systems development textbooks. The four determinants identified in section two were used as categories for the abandonment of the project. It was necessary to define these determinants in terms of the MedPro case study.

The responses of the students were coded by the authors and by independent researchers. The above mentioned definitions as well as the unit for coding were given to the researchers. The coding by the researchers were compared and any problems with the coding was solved by revising the definitions for each of the categories. The two South African researchers coded the South African students’ responses and two Swedish researchers coded the Swedish students’ responses. This was done because of the language used by the students in responding to the questions. By using two researchers for the coding and comparing the reliability in terms of stability and reproducibility of the result could be ensured.

Table 1: Statistics on subjects’ comprehension of problems in the project.

The following definitions were used to code the open ended question on the subjects’ comprehension of the problem, or problems, in the scenario.

1) Time: an indication that the project is taking longer in time than planned, or phrases that refer to the allocation of more time to complete the project.

2) Planning, analysis and design problems: refers to mistakes made in the planning and analysis phases of the project.

3) Economic problems: an indication that the project is over budget or that given more money the project will be successful.

4) Technical problems: refers to software development problems or hardware problems, it could also refer to problems with the programming of the system (not with people).

5) Staffing problems: refers to the experience of group members and the composition of the group members.

6) Communication problems: refers to poor communication and cooperation among group members and between group members and users.

7) Environmental: refers to competing organizations in the same industry.

This section summarizes the Swedish subjects comprehension of the problems presented in the scenario. The problems are listed in decreasing order and supplemented with anecdotal comments from the students.

This section summarize the South African subjects’ comprehension of the problems in the scenario:

Table 2: Summary statistics on subjects decision whether to abandon or continue the project.

During the classification of the comments of the respondents the following three groups were identified: respondents who gave a value of 1 (that signify that the project should definitely stop) to 3 was classified as respondents who wanted to abandon the project; respondents who specified a 4 were classified as unsure of whether the project should continue or not and respondents who specified a 5 or higher were classified as respondents who wanted the project to continue. Only respondents who where unsure or did not want to abandon the project were used in the discussion below. The table below summarize the responses.

Table 3: Summary of the preferences for the projects continuation

4.2.1 Swedish Subjects

Table 4: Summary of statistical analysis

The following hypotheses were tested:

1. Across both samples, when students were positive about the problems did they then specify that the project should continue (thus did they opt for continuation of the project)?

Performing a c ² test (5% significance level) the null hypothesis was rejected (c ² = 4.32998). There seems to be a relationship between problems and continuation and students that identified a positive value for problems will also give a positive value for continuation.

2. Is there a correspondence between the proportion of students that identified problems (Yes to problems in the table above) between the South African and Swedish samples?

Performing a c ² test (5% significance level) the null hypothesis was accepted (c ² = 0.020163). The proportion of students of Sweden and South Africa for the identification of problems are the same.

3. Is there a correspondence between the proportion of students that were positive about the continuation (Yes to continuation in the table above) of the project between the South African and Swedish population?

Performing a c ² test (5% significance level) the null hypothesis was rejected (c ² = 3.88976). The proportion of students of Sweden and South Africa for the identification of continuation is not the same. By looking at the data it can be seen that the Swedish students were more likely to continue the project than the South African students.

In this research we have investigated how students perceive a scenario of a computing project with problems.

Table 5: Summary of understanding of problem

By comparing the order of the importance of the problems identified by the subjects, large similarities could be identified as seen from the table above. Communication problems was ranked in the first and second places respectively for the South African subjects and the Swedish subjects. Time was placed first by the Swedish subject whereas the South African subjects placed it second. Planning, analysis and design, technical problems and economical problems were placed in third and fourth position by the Swedish subjects and third by the South African subjects. Staffing problems was placed in the fifth position by both groups. The environmental aspects were placed sixth by the Swedish subjects and fourth by the South African subjects.

When the motivation for continuation is compared, there is also a correspondence between the subjects’ identification for motivation. What is different though, is the number in the percentage of subjects who wanted to abandon the project. Of the Swedish subjects, 4% thought that the project should be abandoned and 19% of the South African subjects thought that the project should be abandoned. There is a bigger tendency with South African subjects to stop the project, as they were more pessimistic about the project, 4.56 vs. 5.55 in motivation to continue.

The differences as outlined above can be explained by the difference in work experience in subjects as well as the theoretical background. The differences can also be ascribed to the different cultural backgrounds of the subjects. South African students are part of a changing society with a very big possibility of conflict between differences in values and beliefs of the heterogeneous groups whereas the Swedish students are part of a homogeneous society with established values and beliefs.

However, our main finding in this research is that the subjects perceived that the project had problems, but were still convinced that the project should be continued. This result is consistent with case studies (e.g., and other experimental surveys (e.g., . Therefore we conclude that education must include escalation situations and next section is an example for how this can be done.

Reflecting on educational practice in general, we claim that pedagogical principles underlying most educational activities would not address the phenomenon of escalation efficiently. Text books used in systems development , project management and organizational behavior courses have a line or two about escalation situations. We claim that it is not enough. From this perspective, we discuss education principles and practices, however it should be clear that an extensive pedagogical discussion about educational practice is not within the scope of this paper.

Looking at higher education, the most frequently used method of teaching is the lecture. The traditional lecture implicitly embeds an objectivistic model of learning . That is, the purpose of teaching is to structure the knowledge to be learned and transfer this to passive students through effective channels. The instructor is active and in control of the learning material and the environment where learning is taking place. Information is transmitted to students with little concern for whether the students understand or assimilate the information transferred . Assessment and examinations is to determine whether content has been retained . The objectivistic model may be appropriate in some educational activities, but we claim it is insufficient in the case of escalation situations. As a contrast and alternative, the constructivistic model of learning asserts that rather than transmitted, knowledge is constructed or created. Learners are assumed to learn when they are forced to discover things themselves and not by being instructed. The role of the educator is to facilitate the learning process and support the students in the construction of their knowledge .

Whereas objectivism and constructivism is concerned with the individual learner, a collaborativistic model assumes that control of learning should rest with a group of learners. Similar to constructivism, collaborativism asserts that learners construct knowledge by making sense in terms of what they already know but also in interaction with other learners . This way, learning is the sharing of knowledge from individual views through collaboration. Whereas instructor led learning is inherently a linear and predictable process, collaborative groups ensures a more unpredictable process we also claim it to be a more rich learning experience. The importance of experience in learning is recognized by Graf and by Kolb in his learning cycle which begins with a concrete experience (see figure 2).

Figure 2: Kolbs model of the learning cycle

In experiential learning the basic underlying principle is that the best learning is by doing–learning-by-doing . A similar understanding is proposed as–learning-in-doing . Examples of experiential learning are internships, computer assisted instruction, live case, case studies, role play, games and simulations.

Clearly, new pedagogics and alternative learning models fundamentally change the roles of the students and instructors (see for instance . The role changes from the main source of information to a facilitator of the learning process. Rather than lecture information or manage behavior, teachers cultivate skills, focus effort, foster resourcefulness, and maintain an interactive climate of learning. The expected role of the student is that of active participator in the learning process.

Moving the discussion into a systems development education context, we find several examples of collaborative learning activities. Probably the most common activity in systems development education is to divide the class into smaller project teams where each team are requested to complete a systems design and development project, which they have to develop according to the methodology they were taught. Each group has access to a lecturer or instructor who act as their senior project manager. The senior project manager is there to monitor the situation and give advice if the project team have any problems. This gives the students experience in how a system is developed in a group and what possible problems might arise in this situation. Our main critique is that this project team operates in an artificial rational environment neglecting several real world problems. However, as this critique is quite common, our interest is and emphasize here is on the phenomenon of escalation situations.

In this section we articulate some ideas on how escalation situations can be integrated in system development education. Experiential learning and collaboativism serve as pedagogical foundation for our discussion. Participating in a failure can be a real eye-opener for the students and one that they are likely to remember. In line with the concept of experiential learning, knowledge that is practiced is more efficiently assimilated than knowledge passively absorbed. However, with limited resources, such as time, systems development courses must provide this experience through a simulated situation. What we suggests, is a role play activity in a development project where a group of students work together. It should be noted that role playing escalation experiments has been performed previously .

The students should be organized in teams and the instructor should provide them with the case material in a continuous fashion throughout the process. This can be documents, models, tapes, and videotapes, all depending on the instructor’s enthusiasm and time. The group plan and start the project and the instructor assures that the role play case is ‘alive’ by supplying information about problems, changes, etc., constantly, see box 1 below for a rough description of the process. Supplying additional information is a delicate task for the facilitator. This demands a lot from the educator as she must create the necessary atmosphere to make this possible. The task is initially to make the group confident and make them believe their work is professionally performed, then successively let problems surface. A central aim in developing an escalation like situation is to facilitate chained decisions, that is, the result of one set of decision influence the rest of the decision making process.

The scenario about MedPro and TEST used to survey the students can be seen as an example of a case where the instructor supplies information as the project progresses. To keep all participants and the instructor focused and concentrated on the task we believe two hours as an appropriate length for the role play.

Box 1: Outline of role play case study framework

It is very important to debrief the group after the role playing task is completed to achieve the desired effect . The awareness of the purpose and methodology of the experiential activity is underlined by Jones where the damaging effect a simulation/game can have is identified. The reason for the damaging effects is when the instructor does not recognize the methodological conflict in using gaming and simulation. Here the instructor should emphasize the purpose and the underlying principles of the role play. Otherwise there is a risk that the role play will result in the students having low self-esteem and confidence. The debriefing discussion (Step 5) with the students should be augmented by referring to the fast growing body of literature on project failures and especially project escalation (e.g., , emphasize should also be placed on escalation avoidance.

Hence, what we advocate is consistent with the underlying principle of problem based learning (PBL). The essence of PBL is the understanding and analysis of a problem situation as a basis for acquiring knowledge, skills and attitudes (see for instance http://www.imsa.edu/team/cpbl/cpbl.html, or for an introduction to PBL).

The role playing can of course be enhanced by computer technology. Innovations in technology, such as multimedia, hypertext, video, Internet and virtual reality, is now impacting experiential learning. With our role play as example it should be possible to design and develop a computerized version. Then, the instructor can follow the students’ progress through the computerized project diary, a repository where all models of analysis, design and planning are stored, as well as all documents of the decisions made by the group. The instructor can act as the client, top management, governmental organizations, competitors, etc., by communicating information to the project. Examples of these advances in technology are the use of hypertext in teaching systems analysis and design , the use of multimedia in systems analysis case studies , and the Cardiac Tutor were the students are in middle of the emergency room.

In this paper we have highlighted a neglected issue in computing education, namely the phenomenon of escalation situations. We conducted an experiment to survey how students perceive and act in a scenario of an escalating systems development project. Below is a summary of our results:

We would like to thank Andre Olivier and Erik Finnman with their help with the content analysis, as well as Rens Scheepers for his help with the statistics. We would also like to thank the reviewers for their thorough review of our paper.