The approach to both runways at San Francisco International Airport (KSFO).

The Failure of Crew Resource Management (CRM) – Part III

Part III: Asiana Airlines Flight 214

This is Part III in a series of blog posts on “The Failure of Crew Resource Management.” Click here to read Part II. 

NB: The following post is adapted from a script for a TED-style video talk that was submitted in Dr. Adam M. Grant’s Management 238: Organizational Behavior course (Fall 2014). Special thanks to Harikrishnan Joy, who was my teammate in research and execution of this project. We conducted an analysis of the Asiana Airlines Flight 214 crash in 2013 through the lens of the Hackman and Ginnett research presented in Part II. In order to paint a picture of what happened in the cockpit on that day, we listened to the tapes stored by the plane’s cockpit voice recorder (CVR). We later found the official NTSB report and CVR transcript and acknowledge several omissions in our own transcript that were picked up in the NTSB’s detailed audio analysis – so please forgive what may at times appear to be a dramatization from the original events.

July 6, 2013. Asiana Airlines Flight 214 is on final approach into San Francisco International Airport. The hot California sun beats down on the Boeing 777’s aluminum exterior. It seems like a pilot’s perfect day – light wind, no precipitation, maximum visibility, no wind shear. At the controls is Captain Lee Kang Guk, a 45-year-old with 9,793 flight hours under his belt and completing his required Initial Operating Experience training. It is Captain Kang Guk’s first time landing the Boeing 777 aircraft at San Francisco International, an airport notorious for its seaside runway requiring an approach pattern over the Bay. It is also his first time flying with the gentleman in the right seat, Captain Lee Jung Min, a 48-year-old Asiana veteran with 12,387 flight hours. As the highest ranking pilot on board, Jung Min has the title of “Pilot in Command,” assuming responsibility for the safety of the 291 passengers and 16 crew on board. Although this senior position would have generally placed him in the left seat of the cockpit, on this flight Captain Jung Min also serves as a “checkride instructor” tasked with evaluating Captain Kang Guk’s performance. Behind both Captains is their subordinate First Officer, Bong Dong-Won, 41 years old, and seated behind the cockpit in the cabin is yet another Captain, Lee Jong-Joo, 52 years old.

As the plane passes through 1,700 feet and three miles out from the airport, Captain Jung Min notes that the airspeed is too high. This leads Captain Kang Guk to disengage the plane’s autopilot and set the engine thrust to idle, which causes the plane’s onboard computers to switch the automatic throttle into a “Hold” setting. Although both pilots expect this, Captain Kang Guk appears to misunderstand this feature of the autothrottle, believing that it would hold the plane’s airspeed at a desired level, when in fact the plane was now responding purely to his controls. As a result, the plane begins to sink below its intended glide slope to the runway. About two miles out from the runway, First Officer Bong Dong-Won alerts both captains to the unusually fast descent, by calling out “Sink Rate, sir”.

No response.

“Sink Rate, sir” he repeats.

No response.

One final time: “Sink Rate.”

No response.

Finally, at around 1.4 miles from the runway, Captain Kang Guk responds to this warning and begins pulling the plane’s nose up – and with the engines still at idle, this action causes the plane to slow even more. At eleven seconds before impact, the airspeed warning in the cockpit begins to ring. Four seconds later, Captain Jung Min appears to take control of the aircraft, pushing the thrust levers up to maximum. Air Traffic Control, noticing the plane’s unusually low altitude, calls out: “Asiana 214, go around.”

It’s too late.

At 11:27am and 50 seconds, Asiana 214 slams into the seawall at the end of the runway. The plane tears apart, making a 330-degree spin before smashing into the ground and catching on fire. Three passengers, Chinese students on their way to summer camp, are killed.

Crew Resource Management, or CRM, is a set of training procedures that focuses on decision making, leadership, and communication between pilots in the cockpit. The CRM concept was developed in response to the United Airlines Flight 173 crash in 1978, where two pilots failed to work effectively together in troubleshooting a mechanical problem. In a 1990 study on pilot interpersonal behavior, Robert Ginnett found “repeated evidence of poor crew work resulting in errors, accidents and incidents…” Following the crash, United Airlines revamped its crew training program, and CRM has since become the global standard for airlines. A 2014 study by Ford, Henderson and O’Hare in the Journal of Safety Research claims that CRM increases safety by reducing communication barriers in the cockpit and decreasing the traditionally hierarchical and authoritarian relationship between senior and junior pilots. This is evidenced by several pilots’ first-hand accounts of various accidents. As Captain Al Haynes (of the UA232 Sioux City incident) states: “…we had 103 years of flying experience there in the cockpit, trying to get that airplane on the ground, not one minute of which we had actually practiced, any one of us. So why would I know more about getting that airplane on the ground under those conditions than the other three. So if I hadn’t used [CRM], if we had not let everybody put their input in, it’s a cinch we wouldn’t have made it.”

What happened on Asiana 214 is a clear example of the failure of CRM. As pilot in command, Captain Jung Min failed to take any overriding action or make any commanding statement until it was too late. Caught up in an unbiased evaluation of Kang Guk, Jung Min may have been reluctant to intervene. Even more shockingly, both captains neglected the first officer’s repeated warnings. Kang Guk later said that it was “very hard” to make the decision to abort the landing, given the deference shown to superiors in Korean culture. By misinterpreting Jung Min’s silence as a sign of approval from a senior pilot, Kang Guk failed to act decisively even when he felt uncomfortable with the landing. And perhaps there was an element of face-saving – he didn’t want to look weak in the eyes of his superior by heeding a junior’s warnings. Furthermore, First Officer Dong Won may have been reluctant to speak up in the face of two senior pilots. His first two warnings contained the respectful suffix “Sir.” His third warning did not and was more authoritative in nature. Unfortunately, this came too late.

In an interview called “Why Teams Don’t Work”, J. Richard Hackman claims that many teams consistently underperform their potential. From his research, small teams who stay together for a long time perform the best. In the context of flight crews, the NTSB found that 73% of incidents occurred on a crew’s first day of flying together, before having the chance to learn how best to operate as a team. A NASA study found that fatigued crews who had a history of working together actually made about half as many errors as crews composed of rested pilots who had not flown together before. Hackman points out that while crews are cycled in and out due to financial and efficiency constraints, this often comes at the expense of team effectiveness. Decreased communication and less support break down positive team dynamics. Hackman found that the Strategic Air Command, a Cold War nuclear bomb squad, had teams that trained together and performed better than any other flight crew ever studied.

In Hackman’s eyes, there are 5 things for building an effective team. Teams need to have:

1) Realness

2) A Compelling Direction

3) Enabling Structures

4) A Supportive Organization

5) Expert Coaching

In addition, the use of a deviant, such as junior pilots speaking up, helps propel an effective team.

Moving forward, we suggest altering the process by which CRM is implemented. The breakdown of teamwork and communication between pilots can be disastrous. Even with CRM implemented, there have been many examples of improper decisions being made due to factors such as cultural norms. It’s critical for CRM training programs to explicitly acknowledge these teamwork impediments through the lens of previous failures. By using Hackman’s principles, we hope to see safety standards greatly improved and CRM failures reduced.

On Saturday, March 8, 2014, Malaysian Airlines Flight 370 disappeared while flying from Kuala Lumpur to Beijing. Although we do not know yet what happened to the plane, the dynamics in the cockpit that day will certainly be interesting to analyze in this organizational behavior context. Teamwork takes more than simply having a group of people; in the case of airlines and especially crises, a successful team needs to train together, stay together, and deal with the situation together.

Best Friends or Worst Enemies?

The Failure of Crew Resource Management (CRM) – Part II

Part II: An Evidence-Based Approach to Analyzing Failures of CRM

This is Part II in a series of blog posts on “The Failure of Crew Resource Management.” Click here to read Part I.

Continuing on my previous post, I looked into the surprisingly vast body of academic research on Crew Resource Management (CRM), and settled on two research works.

I’d first like to examine the work of J. Richard Hackman, a leading organizational psychologist and mentor to my own professor (Dr. Adam M. Grant). Dr. Hackman “spent a decade of his career studying how to improve the effectiveness of airline crews.” Below is an Harvard Business Review interview with Dr. Hackman entitled “Why Teams Don’t Work,” along with a link to his namesake publication.

Why Teams Don’t Work – J. Richard Hackman, Harvard University

Harvard Business Review interview conducted by senior editor Diane Coutu:

Original Work:

Dr. Hackman’s research is really interesting – I remember learning about his Job Characteristics Theory in MGMT104: Human Resources Management (Fall 2013). I didn’t realize that he had spent so much time researching airline crew effectiveness. In particular, I found it really fascinating that two pilots will only work together as a team every 5.6 years. On one hand, this completely makes sense – in the race for pure efficiency and cost savings, airlines need to have their pilots be transferable to different routes and aircraft at a moment’s notice. A good example of an airline that really buys into this mentality is Emirates – I recall watching a documentary about their new hub airport in Dubai, where everything is timed to the second to ensure that unnecessary costs are not incurred. I wonder, though, if this drive for financial efficiency is implicitly exposing the airline to safety risks – and not the usual physical/engineering risks afflicting airlines, but rather risks related to the way their crews function together. Dr. Hackman would certainly say that the airline crews need to stay together to achieve the best performance as a team. At the same time, a good team requires a strong leader to ensure that group members don’t get complacent. Finding that balance between “individual autonomy and collective action” is especially crucial for pilots – and in the case of Air France 447, the individual decision of one co-pilot was at complete odds with the actions of the other co-pilot and the captain.

The Hackman paper is great context for the next reading, which is written by Robert C. Ginnett, a former Senior Fellow at the Colorado-based Center for Creative Leadership. Dr. Ginnett wrote a chapter entitled “Crews as Groups: Their Formation and Their Leadership” for the academic publication Cockpit Resource Management (Wiener, Kanki, Helmreich, 1993).

Crews as Groups: Their Formation and Their Leadership – Robert C. Ginnett (LinkedIn)

“Cockpit Resource Management (1993), Chapter 3:

One of the examples given in the reading involved United Airlines Flight 173, a DC-8 aircraft that crashed in 1978. I shared this with my grandfather, who worked on this type of aircraft in the 1970s and recalls the incident. He was able to shed some technical insight on the crash – and in particular, he noted that many of the problem with the DC-8 aircraft (including a conditionally-faulty fuel gauge) were widely known by ground engineers but largely ignored by pilots. All in all, it was the failure of the crew to stay cognizant of, prevent, and then respond to the technical fault that caused the ultimate breakdown in teamwork and resulting crash. What happened 31 years later with AF447 is almost identical.

This is Part II in a series of blog posts on “The Failure of Crew Resource Management.” Click here to read Part III. 

Recover-All: Team 110 Entprentice Exercise, 2014

Lessons in Entrepreneurship: Reflections on the Entprentice Exercise


I had the chance to catch up with Professor Wry on Wednesday to discuss a common interest of ours: microfinance. Over the last few years, I have built a microfinance portfolio on, making $25 loans to entrepreneurs across a variety of industries in emerging markets. It’s always empowering to see how far my $25 goes towards helping these businesspeople achieve their dreams – but I always remember that my investment is only a small fraction of the large pool of crowd-funded capital received by each entrepreneur. For this reason, I was rather shocked when Professor Wry gave each team $20 in convertible debt at the beginning of the Entprentice exercise. I wondered: “How could we possibly expect to turn this measly startup capital into thriving retained earnings?”  My skepticism soon disappeared, once I realized how much our team could truly achieve by applying several rigorous analytical frameworks to ideate and implement our venture.


The Team

I would be remiss not to acknowledge my team members before diving into the major aspects of the Entprentice. I had previously worked with Sean Murphy for Management 100 in our freshman year, so I knew a lot about him from both a personal and professional standpoint. And it didn’t take long for me to get close with TJ and Quincy, as they are both incredibly personable, witty and intelligent people. We had the misfortune of losing one of our team members, Larry, shortly before the end of the drop period. While this was slightly demoralizing for the team, we came together and pitched in to fill the gap left by Larry. Altogether, our team was incredibly diverse, and this provided a unique set of perspectives as well as a wide range of on-campus networks to use when implementing our project.


Idea Generation: Strategy, Challenges and Reflection

Given our team’s dynamic personalities, we approached idea generation with great enthusiasm. However, we soon found ourselves stuck in the ideation trap: wanting to create the next Venmo, but only coming up with ideas for grocery stores. It was a good idea to utilize various frameworks from the course readings in order to broaden our thinking. In particular, Nalebuff and Ayres’ “What Would Croesus Do?” theory allowed us to recognize the global trend of students become a) more health-conscious and b) more stressed about their time-constrained lives. Our strategy became centered on addressing the need of students to fuel their constantly moving (and often unsustainable) lifestyles. Thus, Recover-All was born, and our team launched the venture with boundless optimism for where it might take us.

In retrospect, however, it seems that our approach to idea generation restricted us and almost ensured that we would not end the Entprentice with the highest retained earnings. Using the “Croesus” theory – an approach that encourages idea generation without monetary constraints – led us to choose a business idea that involved significant costs compared to the other ventures. Looking back at the Entprentice, we can see that the highest-performing teams excelled because they had little to no costs. A prime example is the “Penn Portraits” team, which consisted of a skilled photographer and did not need to invest in a camera nor pay for the use of prime locations on campus. Even Selfie Straws, which had to produce physical goods, was able to manage its costs effectively by gauging customer demand before placing any manufacturing orders. By comparison, we lacked this ability to determine demand for Recover-All prior to purchasing the raw materials. As a result, we were frequently left with leftover inventory and sunk costs – often times of perishable goods, such as bananas – when sales did not match our expectations.

If we were to complete this exercise a second time, I think that we should have approached idea generation with a different mindset. Prior to using the “Croesus” framework to come up with the idea for Recover-All, we had focused on coming up with a business idea that did not necessarily address an existing problem but, if properly executed, would create a need for itself. Ironically, this “Solutions in Search of Problems” or ignorance management approach led us to an idea very similar to Penn Portraits: we wanted to use TJ’s photography skills to help Wharton students take professional LinkedIn-style photos. Recognizing that people inherently love to be photographed, we hoped to create a market where people would see us on Locust Walk every few days and want to “renew” their photo for LinkedIn or any other social media platform. However, we readily discarded this idea because we felt it lacked originality, and we feared that there would be too much competition for similar services. This is an important lesson about entrepreneurship: you are not required to have the most revolutionary business model. While venture originality definitely provides a distinct competitive advantage, there are numerous other ways to differentiate yourself in what appears to be a mature market – for instance, by being “faster, better, and cheaper” than the competition. We approached the Entprentice wanting to create a “high potential” business similar to our individual ventures, but it was unfeasible to roll out a brand new product and turn a large profit in such a short amount of time. The most successful Entprentice ventures used existing products and services, and by taking advantage of low cost barriers to entry, accessed new markets of customer demand. Perhaps if we had entertained the simpler photography venture idea, we would have saved significantly on costs and turned a much higher profit.


Venture Implementation: Strategy, Challenges and Reflection

Although Recover-All was a relatively successful product (with final retained earnings of $157.00), we faced several challenges in implementing our venture. In general, our response to these problems was appropriate, but did not align with our overall strategy of being “faster, better, and cheaper” than our competition. A prime example is the initial dissatisfaction expressed by our customers towards several items in Recover-All. In our survey for Challenge 1, many respondents suggested that we remove the water bottles from the package, citing environmental concerns along with the readily available water fountains on campus. In addition, many students felt that adding a bottle of Gatorade or other energy drink would raise the package’s intrinsic value. However, we had invested a lot of our startup capital into water, and thus we didn’t have the cash to buy Gatorade in bulk until Challenge 3. Meanwhile, we still had a surplus of water bottles, which we were forced to continue selling as part of the package. This was detrimental to our brand image, because customers didn’t feel as if we were adapting to their preferences quickly enough.

If we were to complete this exercise a second time, I think that we would need to better manage our inventory. As discussed in Idea Generation, above, the best Entprentice ventures were able to estimate demand prior to incurring any production costs. This was hard to achieve with Recover-All because we needed to show our customers a tangible product in order to convince them to buy it. However, we could have created a minimum viable product using items from our personal pantries (it’s not too hard to find a water bottle, a pack of Oreos, and a granola bar lying around). Then we could have showcased this MVP for one or two days at the start of the project, and directed potential customers to a landing page where they could express their interest in buying Recover-All, as well as any comments they may have. Based on these “pre-orders,” we would buy just the right amount of each product, minimizing both initial costs and inventory surplus. In addition, this MVP could have helped us test the core “must-have” features of our product, and we could have used the money saved to dynamically adapt to the customer’s preferences. This is something that could also be achieved by simply creating our venture’s Facebook page at the very beginning of the project, instead of during Challenge 2 – after all, being millennials, we didn’t have to wait until the “Marketing” class to know that the most basic advertising medium is Facebook. Thus, determining customer demand and forecasting our inventory needs earlier on in the project would have given us more flexibility in the start-up phase.

Addressing our marketing strategy in particular, I believe that my idea for the “dual station flyering” tactic (discussed in Challenge 2) provided additional value to the venture. However, this tactic, although unique, did not raise sufficient awareness about our venture – indeed, we only received 5 flyers in exchange for a $1 discount at the actual sales station. Although we distributed 120 flyers, we were unable to make a widespread impact because students tend to disregard and throw away flyers shortly after receiving them on Locust Walk (something I know from personal experience). Instead, we should have dedicated our resources to mass marketing, as this would have reached a wider audience and driven greater sales without the discounted price. The same goes for a sales tactic used by one of my teammates, Quincy, who regularly discounted the unit price by $1 “just to make the sale.” Although these tactics certainly drove additional sales, they were ineffective at raising sufficient awareness of our venture to justify the discounts offered.

The discounts offered as part of our marketing strategy highlight yet another challenge to our venture: keeping track of the financials. As with all retail businesses, it is incredibly important to keep track of sales as they are made. However, since each of us played the role of individual salesperson, it was difficult to tally up the sales unless we were all in one place at one time. Considering myself quite skilled with Excel, I created a spreadsheet to record the sales, and shared this with the team through GoogleDocs. However, it was difficult to ensure that people actively updated this spreadsheet as sales were made. When it came down to calculating retained earnings at the end of each period, it became a nightmare for me to reconcile the cash that Quincy had been collecting with the number of sales recorded – in part due to the fact that certain sales were also made at a discount. In the end, we had $5 in unrealized gains because recorded sales exceeded cash by one unit. This problem could have been easily avoided with a bit of task management – by putting one person in charge of holding the cash and recording sales.

Finally, there was a legal problem that we overlooked, and which a customer kindly brought to our attention during the final challenge: several of the items included in Recover-All were ineligible for re-sale, since we had bought them in bulk. For example, the packets of Oreos we bought at Walmart were originally contained in a large box marked “Not For Resale” (although we only bought individual packets, not the whole box). If we were to continue with our venture, we would either need to a) attain explicit permission from the manufacturers of these products or b) become a channel partner for these manufacturers. In the latter case, becoming a licensed distributor would most likely require us to share profits with the manufacturers, depleting our margin even further. This is a central reason for why we are closing down our venture after the Entprentice exercise.


Key Takeaways and Lessons on Entrepreneurship

The Entprentice has been a great exercise for me to learn about myself as an entrepreneur. As someone who has taken on many leadership roles in several extracurricular activities, this project really opened my eyes to the various types of entrepreneurial management. In comparison to established organizations, which already have a defined vision and merely require the execution of activities and events, startup ventures must both craft a vision and innovate on it to create a meaningful product or service. From a leadership perspective, this involves working effectively with a high skilled management team to apply ‘design thinking’ and drive core success factors.

I explored my own entrepreneurial leadership style while working with Sean, TJ and Quincy in the Entprentice. In terms of ideation, one of my core strengths as an entrepreneur is having a risk-taking attitude. When we were coming up with our venture, I offered several ideas that had a high potential for growth but also required us to spend our entire startup capital at once, with the risk of losing it all if the venture failed. I was willing to pursue these ideas, but my teammates expressed their desire to undertake a more predictable project with at least some guaranteed cash flows. I adapted to this attitude and demonstrated my ability to be cautious of risks by suggesting we keep some of our funds aside as “working capital” for any possible contingencies (although I didn’t speculate as to what these contingencies might be). After seeing how this working capital was insufficient to allow a product switch from water to Gatorade, I learned the importance of building flexibility into the venture’s business model. By tempering my entrepreneurial drive with careful financial planning, I will one day be able to run a startup that pursues its high-level vision while still remaining prepared for any pivots in its business model. Although I may have a high “tolerance for stress and discomfort” (Bhide 9), I should always remember that certain members of my team may not share this temperament and accordingly, my vision may need to reflect this balance of risk tolerances.

I am a results-driven entrepreneur: I like to see tangible actions being performed that lead to profitability and measurable product awareness and virality. While this “powerful bias for action” (Bhide 12) is certainly a positive trait to have as an entrepreneur, I have the opportunity to grow by thinking about the bigger picture. There’s more to success than having an extremely profitable and well-known enterprise, and it is important for an entrepreneur to think about the “goals, strategies, and capabilities” (Bhide 12) of his venture in order to drive its long-term growth. Entrepreneurs must constantly ask themselves about the direction they want for their venture, and whether their business decisions align with, or compromise, their vision. Where profitability is not the main motive – as with many social enterprises – there are a whole host of other concepts to consider when building a startup, many of which can ultimately lead to profitability. By asking myself tough questions about my vision and goals as an entrepreneur, I will hopefully one day create a highly successful venture.

After participating in the Entprentice, I have a much greater appreciation for the men and women whom I’ve been funding on Many of these entrepreneurs wish to start businesses in mature industries, such as clothing retail or entertainment – but I have learned that this is perfectly acceptable and equally deserving of funding as high-potential ventures. At the end of the day, what truly matters to a venture’s success is the entrepreneur’s vision: where they want the business to go, and how they want to get there. A strong, codified vision with testable hypotheses and tangible goals allows a venture to maintain a distinct competitive advantage. I also have gained a deep appreciation for the value of the team in entrepreneurial ventures. Many microfinance loans go to small groups of entrepreneurs who operate in the same space and collaborate to develop their businesses and ultimately repay their startup capital to the original lender. A quote from New Venture Creation really stuck out to me:

“The lone-wolf entrepreneur may make a living, but the team builder creates an organization and a company – a company where substantial value, and harvest opportunities, are created.” (New Venture Creation, p.277)

At its heart, entrepreneurship is all about the individuals that come together to ideate, innovate and potentially revolutionize the world.