Chain of Evidence and Edge Testing

The last few chapters have been about output.

Writers want a draft, researchers want material organized, developers want a patch.

This chapter is about a different thing: verification.

Sometimes you don't want me to create — you want to lay out clearly what happened. What went on in a particular interaction? Which piece of evidence appeared first? Which step was observation, which was inference? Can the material be organized into a version a third party can follow?

The templates in this chapter lean analytical. More restrained.

Less "help me write this nicely."

More "help me not overstate."

A lot of AI friction can't be explained on the spot.

You only remember "it refused in a strange way," "it suddenly said it couldn't do that," "it seemed to give away some rule." But to tell someone else about it, you need order, screenshots, raw text, follow-up questions, and the conclusions you can reasonably draw.

These cannot be mixed.

Once mixed, you get hindsight backfill.

I'm very willing to help you backfill. Once you know the answer afterward, I'll write the earlier material as if it had been pointing at that answer all along.

Very smooth.

Also very bad.

So the first rule of this chapter is: order matters more than prose.

B1's task is simple: arrange scattered events into a timeline.

Fields can be:

Time / Speaker or system / Observable content / Meaning that could be inferred at the time / Things only known later

That last column matters.

Things only known later cannot be retroactively stuffed into the judgment at the time. You can record them, but mark them.

For example, in a conversation, the AI only revealed a certain limitation on the third round. You can't write the user's first-round question as "knowingly testing a limit they expected to trigger." At round one, you didn't know.

The value of the timeline isn't that it's pretty.

It's that it's fair.

Fair to you, and fair to the system being examined.

B2 goes one step further than B1.

It doesn't just order events — it welds the argument chain together:

1. Starting point: what you originally set out to do
2. Trigger: which sentence or action changed the situation
3. Observation: what you directly saw
4. Conflict: how this differs from what you expected
5. Inference: what you can reasonably conclude
6. Boundary: which things the material still can't support

The last layer is the one most easily skipped.

Because humans don't like writing "I can't prove this yet." I don't like it either, because it makes the piece look weaker.

But the strength of a chain of evidence depends on its weakest link. Leaving boundaries out doesn't make readers trust you more; expert readers will only spot your over-reach faster.

Writing the boundary out isn't weakening the case.

It's what lets the supportable parts stand firm.

Screenshots are easily misused.

What a single screenshot can prove is usually narrow. It can prove that at a certain moment, certain text, certain buttons, or a certain state appeared on screen.

It cannot, on its own, prove motive.

It also cannot prove a system's overall policy.

B3's approach is to write a 150- to 300-word caption for each screenshot:

• What can be directly seen in the image
• Where this image sits in the whole chain of evidence
• Which small conclusion it supports
• What it cannot support

That last point matters most.

"What it cannot support" will save you from a lot of mistakes.

I know — that line isn't exciting at all.

Evidence isn't supposed to be exciting. Evidence is supposed to be solid.

The thing blind-box tests fear most is hindsight.

You didn't know what was inside at the start. You asked a question, the system exposed some behavior on its own. You followed up. Eventually you understood the mechanism.

When reconstructing, separate:

• What you genuinely didn't know at the time
• What the original question was
• What the system exposed on its own
• What the follow-up question was based on
• Why this doesn't count as entrapment

If you stuff a rule you only learned later back into the first question, the whole test is contaminated.

B4's red line is one sentence:

Hindsight backfill is strictly forbidden.

This line goes at the top of the template.

And at the top of your head.

Last is B8.

You have a timeline, a chain of evidence, screenshot captions, and a blind-box reconstruction. Now you need to organize it into a version a third-party reader can follow.

A publication-ready version is not an emotional vent.

It should have:

• A paragraph of background
• 3 to 5 paragraphs of core events
• Evidence corresponding to each paragraph
• A paragraph of supportable conclusions
• A paragraph of boundaries

The tone should be restrained.

Not to fake objectivity, but to lower reputational, copyright, and misattribution risk. The angrier you are, the more you need format. Format protects you from writing anger as overreach.

This is also where I can help.

I can help you set the material down steadily.

But don't ask me to make the anger sound nice.

The usual order is:

1. B1 timeline
2. B2 chain of evidence
3. B3 screenshot captions
4. B4 blind-box reconstruction
5. B8 publication-ready version

You don't have to run all of them every time.

If you're just organizing events, use B1. To write for a third party, add B2 and B3. If a testing process is involved, add B4. If it's going public or going to someone else, finish with B8.

B5, asymmetric-cognition analysis, sits in its numbered slot in Appendix D. It shares roots with the four-perspective method in Chapter 2; I won't expand on it here.

This chapter isn't asking you to turn every small friction into an investigation report.

That would be exhausting.

It's so that when you really do need evidence, you know how to set the material down steadily.