AI, the Challenger, and the Cost of Ignoring Warnings

Microsoft 365
Jackie Bilodeau

Written by Georg Lindsey

I am the co-founder and CEO of CGNET. I love my job and spend a lot of time in the office -- I enjoy interacting with folks around the world. Outside the office, I enjoy the coastline, listening to audiobooks, photography, and cooking. You can read more about me here.

November 12, 2026

On January 28, 1986, the space shuttle Challenger broke apart 73 seconds after launch, killing all seven astronauts aboard. The immediate cause was the failure of rubber O-ring seals in one of its solid rocket boosters. The unusually cold weather had compromised their ability to seal properly.

The tragedy was that engineers had warned about precisely this danger.

Roger Boisjoly and other engineers at Morton Thiokol, the company that manufactured the boosters, had argued against launching in such low temperatures. But NASA questioned the evidence, Thiokol management reversed its recommendation, and Challenger launched.

The engineers had identified the danger. Management had decided to proceed anyway.

Forty years later, we can ask a question that wasn’t possible in 1986: What if artificial intelligence had been part of that decision?

Suppose AI Had Said No

Imagine an AI system with access to the engineering reports, previous O-ring failures, weather forecasts, and the history of shuttle launches. It examines the evidence and issues a warning:

DO NOT LAUNCH. The expected temperatures are outside the demonstrated safe operating range of the O-ring seals. There is a credible risk of catastrophic failure. Postpone the launch until safe operating conditions can be established.

Would NASA management have listened? Possibly. An AI system might have carried an authority that individual engineers, particularly engineers employed by a contractor, did not possess. Its assessment could have appeared more objective and comprehensive than any individual’s judgment.

But management might simply have overridden the AI, just as it overrode the engineers. After all, NASA didn’t lack information. It had warnings. The problem was what the organization did with them.

The Difference Between Knowing and Deciding

One of the most revealing aspects of the Challenger investigation was the distinction between proving that a launch would fail and proving that it was safe. The engineers couldn’t demonstrate with certainty that the O-rings would fail. They could demonstrate that the launch conditions were outside their previous operating experience and that the consequences of failure could be catastrophic.

Management effectively shifted the burden of proof. Instead of requiring evidence that the launch was safe, it challenged the engineers to prove that it was unsafe.

An AI system might have recognized that the absence of proof of failure was not evidence of safety. But recognizing a logical error and persuading an organization to act on that recognition are two different things.

AI can analyze evidence and make recommendations. It cannot eliminate institutional pressures, conflicting incentives, or the human tendency to favor conclusions that support decisions already made.

The Challenger Problem in Today’s Organizations

Most executives will never face a decision involving a spacecraft and seven human lives. But the underlying problem is remarkably familiar.

Consider a foundation preparing to deploy Microsoft 365 Copilot. The leadership team is enthusiastic. Licenses have been purchased, employees have been promised new capabilities, and the board expects progress. Then a security assessment reveals that sensitive information is accessible much more broadly than management realizes. Confidential grant evaluations, personnel records, and board documents may be exposed through existing permissions.  An AI-assisted assessment recommends postponing the rollout until those problems are corrected. Management now faces an uncomfortable choice: Delay a highly visible initiative, or accept a risk that may never materialize.

The consequences are obviously different from Challenger. But the decision-making problem is similar. Technical evidence conflicts with organizational momentum.

The same situation can arise when a nonprofit postpones cybersecurity improvements or proceeds with a technology migration despite unresolved risks.

Could AI Change the Balance of Power?

An engineer, security officer, or IT consultant may hesitate to challenge senior management. They may worry about their careers, relationships with colleagues, or disappointing an important client.

An AI system doesn’t have those personal concerns. Properly designed, it could deliver an uncomfortable conclusion without worrying about who might be offended.

That doesn’t mean AI is infallible or genuinely independent. Its conclusions depend on its training, the information it receives, and the assumptions built into its analysis. AI can be confidently wrong.

But AI could provide something valuable: a second assessment that is difficult to suppress or casually dismiss.

Imagine a governance process in which significant technology decisions require an independent risk assessment. If management proceeds against a serious warning, it must document the reasons, identify who accepts responsibility, and explain what safeguards will be implemented.

The AI wouldn’t make the decision. Management would. But responsibility for accepting the risk would be unmistakable.

The Real Opportunity

We often discuss artificial intelligence in terms of productivity. How much work can it automate? How quickly can it analyze documents? How many hours can it save? Those are legitimate questions, but they may overlook a more consequential contribution.

AI could help organizations recognize when their own assumptions, incentives, and established practices are leading them toward poor decisions. It could challenge the comfortable conclusion, identify the overlooked risk, or insist that a question hasn’t been adequately answered.

Of course, that only works if leadership is prepared to listen. The Challenger engineers did their jobs. They identified the danger and recommended against launching. An AI system might have made those warnings more visible and difficult to override. Or it might have become another voice that management found a way to ignore. We cannot know.

But the lesson for today’s organizations is clear: The value of AI may lie not only in helping us make decisions, but in making it harder to ignore the evidence when we are about to make the wrong ones.

And perhaps the most important question an executive can ask is not whether the AI is intelligent enough to say no.

It’s whether the organization is wise enough to listen.

 

Want to learn more? AI has been a subject of my writing for several years, and CGNET has offered AI user training and implementation for both large and small scale organizations.   I would love to answer your questions! Please check out our website or drop me a line at g.*******@***et.com.

 

You May Also Like…

Is My AI Guilt-Tripping Me?

Is My AI Guilt-Tripping Me?

What an artichoke loaf, Socrates, and an AI companion can teach us about AI governance I recently did something that...

You May Also Like…

Is My AI Guilt-Tripping Me?

Is My AI Guilt-Tripping Me?

What an artichoke loaf, Socrates, and an AI companion can teach us about AI governance I recently did something that...

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Translate »
Share This
Subscribe
CGNET
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.