← Tomorrow Ready

Tomorrow Ready · Credibility and Verification

When the App and the Chatbot Disagree

Real World Protocol adaptation · Years 1–13 · Science · Field-Based STEM · Tony Jones

At the stream, two AI tools can give two different answers about the same creature, and only one of them is built to be right. Most students will never be asked which one, or why.

1. Name the criterion
2. Gather both answers
3. Compare against the criterion
4. Justify and record

The strategy: Evaluation Gate

The protocol already invites students to compare iNaturalist against a general AI chatbot. The Evaluation Gate turns that comparison into a recorded decision instead of an open question.

  1. Submit the specimen photo to iNaturalist with location enabled. Record the identification given.
  2. Describe the same specimen to a general AI chatbot, either without the photo or with location disabled. Record its identification.
  3. Name one criterion for judging which identification to trust: location awareness, expert verification, or training data source.
  4. Compare both identifications against that criterion and note where they agree or differ.
  5. Justify which identification will be recorded as the class's official evidence for the water quality claim.

In practice

Years 1–6

Teacher selects the criterion (does the app know where we are?). Students say aloud which answer they trust and why, in one sentence.

Years 7–10

Students name their own criterion, write the comparison, and justify the choice in writing before the specimen is added to the indicator table.

Years 11–13

Students run the comparison across multiple specimens, draw a conclusion about when each tool is reliable, and connect this to the conditions under which AI identification can be trusted in real conservation work.

Implementation

Decision checkpoint

No specimen identification enters the indicator table until the criterion comparison is complete.

Teacher judgement note

Location services must be genuinely enabled for the comparison to mean anything. A comparison run with location off on both tools is not a fair test.

Related

Verification Slip (Credibility and Verification core framework) · Stream Macroinvertebrates Real World Protocol

Tony Jones · Founder, Field-Based STEM · Tomorrow Ready Resources · Free to use and share