First-Order Consequences¶

Consequence Simulator

Understanding First-Order Consequences¶

First-order consequences are the direct, immediate results of your AI deployment. They're what's in your business case, your project charter, your benefits realization plan.

They're also incomplete.

Every first-order benefit casts a shadow—an unintended consequence that's logically connected to the benefit but conveniently absent from the slide deck.

The Shadow Framework¶

For each intended benefit, identify its shadow:

First-Order Consequence Categories¶

Category 1: Efficiency Gains¶

The Benefit: AI processes things faster than humans.

The Shadows:

The Question: What happens when you make the wrong decision at 10x the speed?

Category 2: Cost Reduction¶

The Benefit: AI reduces labor costs.

The Shadows:

The Question: What did those labor costs actually buy you that you're now losing?

Category 3: Consistency¶

The Benefit: AI makes consistent decisions.

The Shadows:

The Question: What valuable inconsistency are you eliminating?

Category 4: Objectivity¶

The Benefit: AI removes human bias.

The Shadows:

The Question: Whose biases are encoded in your training data?

Category 5: Scale¶

The Benefit: AI can handle volume humans can't.

The Shadows:

The Question: What happens when you fail at scale?

The First-Order Consequence Audit¶

For each claimed benefit in your business case:

Step 1: Name the Shadow¶

BENEFIT: [Your claimed benefit]

SHADOW: [What's the corresponding downside?]

AFFECTED PARTIES: [Who bears the cost of the shadow?]

VISIBILITY: [How long until the shadow becomes apparent?]

MITIGATION: [What are you actually doing about it?]

Step 2: Quantify Honestly¶

Step 3: The Honest Questions¶

For each benefit-shadow pair:

1. Is the benefit real or assumed? Have you measured it, or is it a projection?
2. Is the shadow in your risk register? If not, why not?
3. Who pays for the shadow? Is it the same people who get the benefit?
4. Is the shadow reversible? If the shadow is worse than expected, can you undo it?
5. Would you accept this trade-off explicitly? If someone said "you get X benefit but also Y shadow," would you still proceed?

First-Order Consequences by AI Type¶

Predictive AI (Risk scoring, forecasting)¶

The deep shadow: You're making decisions about people based on who they statistically resemble, not who they actually are.

Generative AI (Content creation, chatbots)¶

The deep shadow: You're outsourcing communication to something that doesn't understand what it's saying.

Decision AI (Approvals, allocations, determinations)¶

The deep shadow: You're creating a system that makes life-altering decisions about people without understanding their lives.

Automation AI (Process automation, robotics)¶

The deep shadow: You're replacing judgment with rules, and rules don't handle exceptions.

The First-Order Traps¶

Trap 1: Benefit Counting Without Shadow Counting¶

How it works: Business case lists 10 benefits. Shadows aren't mentioned, or appear as "risks" that are "mitigated."

The reality: Shadows are not risks. They're certainties. They're the price of the benefit.

What to do: For every benefit line, add a shadow line. If you won't acknowledge the shadow, you shouldn't claim the benefit.

Trap 2: Aggregated Benefits, Distributed Shadows¶

How it works: Benefits accrue to the organization ("$2M savings"). Shadows land on individuals ("200 people lose jobs").

The reality: The math might work for the organization. It doesn't work for the individuals.

What to do: Name specifically who pays for each shadow. If you can't look them in the eye, reconsider.

Trap 3: Measured Benefits, Unmeasured Shadows¶

How it works: You track processing time, cost savings, throughput. You don't track job losses, stress, citizen harm.

The reality: What gets measured gets managed. What doesn't get measured gets ignored.

What to do: Measure the shadows. Put them in your dashboard alongside the benefits.

Trap 4: Immediate Benefits, Delayed Shadows¶

How it works: Benefits arrive in Year 1 (and make the project look successful). Shadows arrive in Year 3 (when the project team has moved on).

The reality: The people who claimed success won't be around to own the failure.

What to do: Extend your measurement window. Track shadows for as long as you track benefits.

Trap 5: The Shadow Deferral¶

How it works: "We'll address that after launch." "That's a Phase 2 concern." "We're aware but..."

The reality: Deferred shadows compound. They don't wait for Phase 2.

What to do: If a shadow is significant, address it before launch or acknowledge you're accepting it.

First-Order Consequence Template¶

For each major feature/capability of your AI system:

## [Feature/Capability Name]

### Intended First-Order Consequence
- What will happen:
- Who benefits:
- How we'll measure success:

### Shadow Consequences
1. **Shadow 1:**
   - What will happen:
   - Who is affected:
   - How we'll detect it:
   - What we'll do:

2. **Shadow 2:**
   - What will happen:
   - Who is affected:
   - How we'll detect it:
   - What we'll do:

### The Trade-Off Statement
"We accept [shadow] as the price of [benefit] because [rationale]."

Signed: _________________ Date: _________

The First-Order Test¶

Before proceeding, you should be able to complete these statements:

1. "The main benefit is _________, and the shadow of that benefit is _________."

2. "The people who benefit are _________, and the people who bear the shadow are _________."

3. "We will measure the benefit by _________, and we will measure the shadow by _________."

4. "If the shadow is worse than expected, we will _________ within _________ timeframe."

5. "The person accountable for monitoring the shadow is _________, and they have authority to _________."

If you can't complete these statements, you haven't understood your first-order consequences.

And if you don't understand first-order, you're definitely not ready for second-order.

"First-order thinking is: 'What will this do?' Second-order thinking is: 'And then what?' Most projects stop at first-order. Consequences don't."