MIZAN RAG + Caching Architecture & Real-Time Cost Dashboard

MIZAN — RAG + Caching Architecture & Real-Time Cost Dashboard Document purpose: A complete blueprint Claude Code can implement directly.

MIZAN — RAG + Caching Architecture & Real-Time Cost Dashboard

**Document purpose:** A complete blueprint Claude Code can implement directly. Hand this entire file to Claude Code with the instruction: *"Build this for the Mizan platform, module by module, in the order listed in Part 8."*

**Author:** Claude (Anthropic) for Khurram Badar, GMALC Mizan platform
**Target volume:** 5,000 low-end + 5,000 high-end queries/month
**Target cost:** ~$200–280/month (AED 730–1,030) at full optimization
**Reusability:** This same architecture pattern works for Dr. Rashid platform, ZEROAGENCY, Audit Intelligence Suite, and any future platform. Build once, fork for each.

---

PART 1 — COST TARGETS & VOLUME ASSUMPTIONS

Volume budget

Per-query cost targets (with full RAG + caching)

**Total target: ~$209–234/month (~AED 770–860)**

Add 20% buffer for spikes, re-indexing events, and miscellaneous: **plan for $280/month (AED 1,030).**

How this is so cheap

---

PART 2 — SYSTEM ARCHITECTURE

High-level flow

Tech stack

---

PART 3 — RAG IMPLEMENTATION

Step 1: Chunk the seeded UAE law corpus

Step 2: Generate embeddings

Step 3: Embed firm knowledge bank the same way

Step 4: Retrieval at query time

Step 5: Hybrid search for legal precision

Supabase schema for RAG

CREATE TABLE law_chunks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
statute_name TEXT NOT NULL,
article_number TEXT,
jurisdiction TEXT NOT NULL, -- 'federal', 'difc', 'adgm', 'free_zone'
chunk_text TEXT NOT NULL,
embedding VECTOR(1536),
language TEXT DEFAULT 'en', -- 'en' or 'ar'
last_updated DATE,
source_url TEXT
);

CREATE INDEX ON law_chunks USING ivfflat (embedding vector_cosine_ops);
CREATE INDEX ON law_chunks (jurisdiction);
CREATE INDEX ON law_chunks (article_number);

CREATE TABLE firm_bank (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
content TEXT NOT NULL,
embedding VECTOR(1536),
matter_id TEXT,
practice_area TEXT,
partner TEXT,
confidentiality_level TEXT DEFAULT 'firm', -- 'public', 'firm', 'partner_only'
created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX ON firm_bank USING ivfflat (embedding vector_cosine_ops);
```

---

PART 4 — PROMPT CACHING STRATEGY

What to cache (Anthropic native caching)

1. **System prompt** (~1,500–2,500 tokens) — Cache for the entire conversation
2. **Common firm context** (firm name, jurisdictions covered, practice areas) — Cache
3. **The "rules of engagement" section** of any legal prompt — Cache
4. **Frequently retrieved statute chunks** (top 100 most-referenced) — Cache for 1-hour TTL

What NOT to cache

Cache pricing reminder

Implementation pattern

---

PART 5 — ROUTER MODULE

Layer 1: Response cache check (before any LLM call)

Layer 2: Hard-coded rules (deterministic, free, instant)

// Auto-Opus triggers (high-stakes keywords)
const opusKeywords = ['draft', 'opinion', 'memo', 'advise the client',
'sharia', 'criminal', 'constitutional', 'merger', 'acquisition',
'arbitration', 'litigation strategy', 'court filing'];
if (opusKeywords.some(k => q.includes(k))) {
return { route: 'opus', tier: 3, reason: 'high_stakes_keyword' };
}

// Auto-Haiku triggers (simple lookups)
const haikuKeywords = ['what is', 'define', 'list the documents',
'what is the deadline', 'what is the fee', 'how do i register'];
if (haikuKeywords.some(k => q.startsWith(k)) && query.length < 100) {
return { route: 'haiku', tier: 1, reason: 'simple_lookup' };
}

// Partner role gets Sonnet minimum
if (userRole === 'partner' && query.length > 50) {
return { route: 'sonnet', tier: 2, reason: 'partner_minimum_tier' };
}

return null; // No rule matched, escalate to Haiku classifier
}
```

Layer 3: Haiku classifier (only if rules don't match)

TIER 1 — SIMPLE (route: "haiku")
- Single statutory lookup, definition, procedural question
- Document checklist, fee/deadline question
Examples: "What's the VAT registration threshold?", "Define wakala"

TIER 2 — STANDARD (route: "sonnet")
- Application of law to a scenario
- Multi-statute synthesis, comparative UAE jurisdictions
- Client-specific context, moderate stakes
Examples: "Can my client terminate this employee for poor performance?"

TIER 3 — COMPLEX (route: "opus")
- Drafting opinions/memos for client delivery
- Novel questions, multi-jurisdictional analysis
- High-stakes: M&A, litigation, regulatory investigations
- Sharia interpretation, criminal exposure, constitutional questions
Examples: "Draft an opinion on VARA licensing for our token offering"

Output JSON only:
{
"route": "haiku" | "sonnet" | "opus",
"tier": 1 | 2 | 3,
"topic": string,
"stakes": "low" | "medium" | "high",
"needs_human_review": boolean,
"confidence": "high" | "medium" | "low"
}

If confidence is "low", default UP one tier.
`;

async function classifyQuery(query: string) {
const response = await anthropic.messages.create({
model: "claude-haiku-4-5-20251001",
max_tokens: 200,
system: [{
type: "text",
text: ROUTER_SYSTEM_PROMPT,
cache_control: { type: "ephemeral" }
}],
messages: [{ role: "user", content: query }]
});

return JSON.parse(extractText(response));
}
```

Layer 4: Self-correction escalation

The router intercepts this, re-routes to Opus, user just sees a longer wait.

---

PART 6 — COST TRACKING MODULE

Database schema

-- Routing
route_decision TEXT, -- 'cache_hit', 'rule_based', 'classifier'
selected_model TEXT, -- 'haiku-4-5', 'sonnet-4-6', 'opus-4-7'
tier INTEGER,
topic TEXT,
stakes TEXT,

-- Token usage (from Anthropic API response)
input_tokens INTEGER,
cache_creation_tokens INTEGER,
cache_read_tokens INTEGER,
output_tokens INTEGER,
web_searches INTEGER DEFAULT 0,

-- Cost calculation
cost_usd DECIMAL(10, 6),
cost_aed DECIMAL(10, 6),

-- Performance
latency_ms INTEGER,
rag_chunks_retrieved INTEGER,

-- Outcome
user_rating INTEGER, -- 1-5, optional thumbs up/down feedback
escalated BOOLEAN DEFAULT FALSE
);

CREATE INDEX ON api_call_log (timestamp);
CREATE INDEX ON api_call_log (user_id);
CREATE INDEX ON api_call_log (platform);
CREATE INDEX ON api_call_log (selected_model);
```

Logger module

const USD_TO_AED = 3.6725;

function calculateCost(usage: TokenUsage, model: string): number {
const p = PRICING[model];
const cost = (
(usage.input_tokens * p.input) +
(usage.cache_creation_tokens * p.cache_write) +
(usage.cache_read_tokens * p.cache_read) +
(usage.output_tokens * p.output)
) / 1_000_000;
return cost;
}

async function logApiCall(data: ApiCallLogEntry) {
const cost_usd = calculateCost(data.usage, data.model) + (data.web_searches * 0.01);
await supabase.from('api_call_log').insert({
...data,
cost_usd,
cost_aed: cost_usd * USD_TO_AED
});
}
```

Wrap every Anthropic call

await logApiCall({
user_id: params.userId,
platform: 'mizan',
query_text: params.query,
query_hash: hashQuery(params.query),
selected_model: params.apiParams.model,
tier: params.tier,
topic: params.topic,
stakes: params.stakes,
usage: {
input_tokens: response.usage.input_tokens,
cache_creation_tokens: response.usage.cache_creation_input_tokens || 0,
cache_read_tokens: response.usage.cache_read_input_tokens || 0,
output_tokens: response.usage.output_tokens
},
web_searches: params.webSearches || 0,
latency_ms: latency,
rag_chunks_retrieved: params.ragChunksUsed,
});

return response;
}
```

---

PART 7 — REAL-TIME COST DASHBOARD

Page route

Top-line widgets

Charts (use Recharts)

Alerts (write to Supabase + send email via Resend)

Re-tuning recommendations widget

React component skeleton

return (
<div className="grid grid-cols-12 gap-4 p-6">
<KpiCard title="Today" value={today.aed} unit="AED" trend={today.trend} />
<KpiCard title="Month-to-date" value={mtd.aed} unit="AED"
subtitle={`Projected: AED ${mtd.projected}`} />
<KpiCard title="Cost per query" value={today.cpq} unit="USD" />
<KpiCard title="Cache hit rate" value={today.cacheRate} unit="%" />

<RecommendationsCard />
</div>
);
}
```

---

PART 8 — IMPLEMENTATION ORDER FOR CLAUDE CODE

Build in this exact order. Each step is independently testable.

**Phase 1: Foundation (1–2 days of Claude Code work)**
1. Set up Supabase tables: `law_chunks`, `firm_bank`, `api_call_log`, `response_cache`
2. Build the `callClaude()` wrapper with cost logging
3. Test: Make 10 manual API calls, verify all are logged with correct cost calculation

**Phase 2: RAG (2–3 days)**
4. Build the chunking script for seeded UAE law (run once)
5. Generate and store embeddings (one-time job)
6. Build the retrieval function (vector + keyword hybrid)
7. Test: Query "What is the VAT threshold?" should retrieve the correct article

**Phase 3: Caching (1 day)**
8. Add `cache_control` to system prompts in all calls
9. Build the response cache: hash queries, store responses, TTL 24 hours
10. Test: Repeat the same query 3 times, verify hits 2 and 3 cost $0

**Phase 4: Router (2 days)**
11. Build the rule-based router (Layer 2)
12. Build the Haiku classifier with the ROUTER_SYSTEM_PROMPT
13. Wire all queries through: cache → rules → classifier → answer model
14. Test: Send 20 sample queries of varying complexity, verify routing decisions

**Phase 5: Dashboard (2–3 days)**
15. Build the API endpoints that read from `api_call_log` and aggregate
16. Build the dashboard UI components
17. Add the alerting cron job
18. Test: Generate fake data for a week, verify all charts render and alerts fire

**Phase 6: Production hardening (1 day)**
19. Add the user feedback mechanism (thumbs up/down on every answer)
20. Add the weekly auto-tuning recommendations
21. Document the system for the GMALC team

**Total estimated build time: 9–12 days of Claude Code work.**

---

PART 9 — REUSE FOR OTHER PLATFORMS

This same architecture, with different system prompts and seeded knowledge bases, powers:

**Dr. Rashid platform** — replace law corpus with Dr. Rashid's IP, books, consulting frameworks. Same router, same dashboard, same cost profile.
**ZEROAGENCY** — replace law corpus with marketing playbooks, brand guidelines, ad copy patterns. Same router pattern, swap the system prompts.
**Audit Intelligence Suite** — replace law corpus with audit standards (IFRS, ISA, GCC-specific). The 21-agent architecture sits on top of this same RAG layer.
**createagent.ai** — meta-use: each agent built on createagent uses this architecture as its template.

**Build the Mizan version cleanly. Then forking takes 1–2 days per platform, not 9–12.**

---

APPENDIX — FAILURE MODES & MITIGATIONS

| Failure mode | Mitigation |
|--------------|------------|
| Vector search returns irrelevant chunks | Add re-ranker (Haiku call to score retrieved chunks before passing to answer model) |
| User asks question outside seeded scope | Detect zero high-similarity hits, fall back to UAE-domain web search |
| Hallucinated article numbers | Post-answer validation: regex extract any "Article X" references, verify against law_chunks table |
| Cost spike from infinite loops | Hard daily cap per user (e.g., 200 queries/user/day) at the API gateway level |
| Cache poisoning (wrong cached answer) | TTL of 24 hours + manual flush button on dashboard + invalidate on law_chunks update |
| Anthropic API outage | Graceful degradation message + queue queries for retry |

---

**End of spec.** Hand this entire file to Claude Code in the Mizan project directory and start with Phase 1.