the school PHOTO EXTRACTION SPECIFICATION v1.0
---
0. MISSION
Convert every photograph of the school's walls, classrooms, corridors, and displays into structured, queryable records so leadership can ask questions like "how many classrooms display learning goals?", "which classrooms have fire extinguishers?", "which students made energy projects?" — and receive honest, sourced, photo-referenced answers.
The school's walls are the collection. This pipeline is the catalogue.
---
1. INTEGRITY RULES (override everything else)
1. **Never fabricate.** If text is unreadable, record `"[UNREADABLE]"` with a flag. Never complete a partial word from plausibility. A guess that happens to be right is still a fabrication.
2. **Verbatim transcription.** Wall text is copied exactly as written, including spelling errors. Corrections go in a separate `corrected_text` field, never overwrite the original.
3. **Confidence on every extraction.** Every name, number, and claim carries `high` / `medium` / `low`. Low-confidence items are never promoted to facts in rollups.
4. **Wall claims are unverified.** Any statistic, award, date, or achievement displayed on a wall is recorded as `wall_displayed_unverified` unless already verified against the client's platform or official sources.
5. **Credentials are never stored.** If a photo shows a password, login, WiFi key, or access code: record `credential_sighting: true`, note the location, and DO NOT transcribe the credential itself. This feeds the data-protection register.
6. **Student names are retained verbatim** (internal knowledge base only, covered by enrollment consent). They are marked `internal_only: true` — the public Atlas layer never surfaces them.
7. **Counts are honest.** The database only knows photographed rooms. Every aggregate answer must be phrased "X of Y documented rooms," never "X classrooms in the school."
---
2. DAILY SESSION PROTOCOL (subscription mode)
- Target: **~100 photos per day**, processed in read-cycles of **5 photos at a time** (5 images per read keeps context manageable and extraction quality high).
- Before starting each day: run `/status`. If the 5-hour window or weekly cap is above ~70% consumed, stop and resume next window. Never trigger the API-credit prompt; if it appears, decline.
- **Resume ledger:** a file `pipeline/ledger.jsonl` records every processed photo: `{filename, photo_id, processed_at, status}`. On session start, Claude Code reads the ledger tail and resumes from the next unprocessed file in walk order. The pipeline must survive any interruption with zero rework.
- **Compact context between batches.** After writing each batch's output to disk, clear/compact the session context. Never hold more than one batch of images in context.
- Day-end: append a one-line summary to `pipeline/daily-log.md` (date, photos processed, flags raised, rooms completed).
---
3. WALK-ORDER ROOM ANCHORING
Photos are processed in **capture order** (original iPhone filenames / EXIF timestamps — files are unrenamed).
- **Anchor events:** a photo showing a doorplate, room-number sign, or section signboard sets the `current_location` context (e.g., `Room 36 / Grade 2-A / GF boys' wing`).
- Every subsequent photo inherits `current_location` until the next anchor event, recorded as `location_source: "carried_forward"` vs `"direct"` (location visible in the photo itself).
- **Ambiguity rule:** if 10+ photos pass with no anchor, or content contradicts the carried location (e.g., corridor content inside a "classroom" run), set `location_confidence: low` and flag `LOCATION_UNCERTAIN` for human review. Never silently guess a room.
- **Known room map** (seed from existing knowledge bank): 22=STEAM, 24=Principal, 26=CR, 27=1-E, 29=1-C, 31=1-D, 33=Clinic, 34/35=Fire Pumps, 36=2-A, 42=3-C, 43=Computer Lab, Library=GF boys' wing. Doorplates are authoritative over evacuation-plan labels (known-stale).
- The ordered sequence of anchors becomes the **Virtual Tour spine** — the walk itself is preserved as a first-class data structure (`tour_sequence` integer on every photo).
---
4. PER-PHOTO READING PROCEDURE — NINE PASSES
Read every photo corner to corner. Execute all nine passes for every image; write `null` (not omission) for passes with no findings.
**PASS 1 — FRAME SWEEP.** Describe the whole frame in 1–2 sentences: what surface (wall board / door / shelf / desk / floor plan / window), orientation, lighting/legibility quality (`good` / `partial` / `poor`).
**PASS 2 — SEGMENTATION.** Divide the frame into visual segments (e.g., "top banner," "left poster," "center chart," "student work grid, 3×4"). Each segment gets an id (`seg-1`, `seg-2`…), a short label, and an approximate position (top-left, center, etc.). All subsequent passes reference segment ids.
**PASS 3 — TEXT TRANSCRIPTION.** Transcribe ALL readable text per segment, verbatim, preserving line breaks, capitalization, and errors. Mark unreadable spans `[UNREADABLE]`. Mark uncertain characters `[?]`. Record language(s): English / Arabic / Urdu / mixed. Arabic and Urdu text: transcribe if legible; otherwise describe and flag `NEEDS_ARABIC_REVIEW` / `NEEDS_URDU_REVIEW`.
**PASS 4 — NAMES & PEOPLE.** Extract every human name into typed records: `{name, role_guess: student|teacher|leader|unknown, grade_section (if shown), segment_id, confidence}`. Match against the master glossary (25 staff, ~65 students) — record `glossary_match: exact|variant|new`. Spelling variants are recorded, never merged silently.
**PASS 5 — VISUAL ATTRIBUTES.** Dominant colors (max 5, plain names), color-coding systems if present (e.g., "subjects color-coded: math=blue"), patterns (borders, grids, themes), display craft quality notes (handmade / printed / laminated / 3D model).
**PASS 6 — OBJECT INVENTORY.** Every discrete physical object: fire extinguisher, smoke detector, evacuation plan, first-aid kit, WiFi access point, projector, smartboard, camera, clock, plant, trophy, model, book, device. Typed as `{object_type, count, condition_note, segment_id}`. Safety and infrastructure objects are the highest-value extractions — never skip them.
**PASS 7 — CURRICULUM & SUBJECT CLASSIFICATION.** Tag content against the controlled vocabulary: `english, math, science, computer, islamiat, mscs, arabic, urdu, quran, steam, pe, reading, financial-literacy, digital-economy, ai-literacy, sustainability-esg, climate, space, health-wellbeing, uae-heritage, values-character, safety, admin-operations, celebration-event`. Multiple tags allowed. New themes not in the vocabulary: tag `other` + free-text + flag `NEW_THEME_CANDIDATE`.
**PASS 8 — ARTIFACT TYPING.** Classify what the display IS: `learning_goals_board, timetable, attendance_board, student_project, student_worksheet, teacher_material, notice_official, motivational_quote, subject_wall, safety_signage, evacuation_plan, door_plate, award_trophy, decoration, infrastructure, book_display, roster_list, other`. This field powers Mr. Rashid's count queries ("how many classes have goals on the notice board" = count rooms with `learning_goals_board`).
**PASS 9 — INTENT & SIGNIFICANCE.** One short paragraph, explicitly labelled `CLAUDE_INFERENCE`: what is this display trying to do pedagogically or operationally, and why does it matter for KHDA evidence / the Atlas? Notable commendations (e.g., a student's energy project deserving recognition) get flag `COMMEND_CANDIDATE` with the student name and project.
---
5. OUTPUT SCHEMA (one JSON record per photo)
```json
{
"photo_id": "the school-00417",
"filename": "IMG_4club8.HEIC",
"tour_sequence": 417,
"captured_at": "EXIF timestamp or null",
"location": {
"room": "36", "class": "2-A", "wing": "GF boys",
"source": "carried_forward", "confidence": "high"
},
"frame": { "surface": "wall board", "legibility": "good", "summary": "..." },
"segments": [ { "id": "seg-1", "label": "...", "position": "top-left" } ],
"text": [ { "segment_id": "seg-1", "verbatim": "...", "corrected_text": null,
"language": "en", "confidence": "high" } ],
"people": [ { "name": "...", "role_guess": "student", "grade_section": "2-A",
"glossary_match": "exact", "confidence": "high",
"internal_only": true } ],
"visual": { "colors": [], "patterns": [], "craft": "handmade" },
"objects": [ { "object_type": "fire_extinguisher", "count": 1,
"condition_note": null, "segment_id": "seg-3" } ],
"curriculum_tags": ["science", "sustainability-esg"],
"artifact_type": "student_project",
"verification_status": "wall_displayed_unverified",
"claude_inference": "…(labelled interpretation)…",
"flags": ["COMMEND_CANDIDATE"],
"credential_sighting": false
}
```
**Flag taxonomy:** `LOCATION_UNCERTAIN, NEEDS_ARABIC_REVIEW, NEEDS_URDU_REVIEW, NEW_THEME_CANDIDATE, COMMEND_CANDIDATE, CREDENTIAL_SIGHTING, VERIFY_EXTERNAL (claims needing the client's platform check), DUPLICATE_SUSPECT (same display re-photographed), POOR_IMAGE (recommend re-shoot), SAFETY_CONCERN`.
---
6. STORAGE
- **Source of truth:** append-only JSONL files on disk, one per day: `corpus/extractions/YYYY-MM-DD.jsonl`. Git-committed daily (aipeaelps identity).
- **Supabase (pea project):** new schema `atlas` — tables `photos, segments, text_blocks, people_mentions, objects, tags, flags, rooms, tour_spine`. Claude Code generates the schema SQL and per-day insert batches; Khurram pastes into the SQL Editor (or Claude Code applies directly if its DB path from the course project works). Remember: add `atlas` to Exposed Schemas or all reads 404.
- Disk JSONL always written FIRST, database second. If the database write fails, nothing is lost.
- Indexes/rollups (per-room summaries, tag counts, name registry) are regenerated from the database, never hand-maintained.
---
7. EXECUTION ORDER
1. **TEST BATCH:** first 20 photos in walk order. Human review of output quality before anything else runs. Measure `/status` consumption before/after → derive real photos-per-window number → set the daily target accordingly (100/day is the goal, the measurement is the truth).
2. Reprocess everything from photo #001 — existing knowledge-bank entries #001–#415 are reference context, not a substitute; they will later be diffed against new extractions as a quality check.
3. Process in strict walk order, day by day, ledger-resumed.
4. Weekly: regenerate rollups, review all flags with Khurram, run the glossary reconciliation (name variants).
---
*This spec is the single source of truth for extraction. Changes are versioned (v1.1, v1.2…) and noted in the repo CLAUDE.md. Subscription-only rule: if any step ever prompts for API credits, decline and stop.*