Metals trading desk technical platform guide

Production whitepaper on COMEX gold and silver trading platform with 15 autonomous agents and Dubai trader workflow integration.

Metals Trading Desk — Technical Whitepaper

**Platform:** metalstradingdesk.com
**Codebase:** titan_trader
**Target user:** COMEX gold/silver swing trader (Dubai, UTC+4)
**Author:** Engineering log, build sessions through 2026-05-22
**Status:** Production on Railway (backend) + Vercel (frontend)

---

1. What this platform is

A session-aware, multi-agent intelligence layer for COMEX gold and silver.
15 autonomous agents running on a 5-min to weekly cadence.
Real data only — no mocks, no AI-generated prices, no placeholders.
Built around one trader's actual workflow (Rashid, Dubai, COMEX swing trader).
Anchor times: Shanghai 2:00 AM, LME 12:00 PM, COMEX 4:20 PM, COMEX close 9:30 PM — all Dubai local.

---

2. Problems solved

**Daily-only signals on an intraday market.** Original engine used daily closes; COMEX gold moves in hourly bars during the 4:20 PM – 9:30 PM Dubai window. Replaced with session-aware hourly engine.
**Silent data failures.** Watchlist showed `—` for SPX, DXY, palladium, agriculturals because of a sparse-cache bug. Caught by the Health Monitor agent and auto-fixed via live yfinance fallback.
**Fake conviction.** Regime classifier returned 100% confidence on thin macro data. Replaced with honest 17–33% confidence bands tied to data source count.
**Arbitrary stops.** Entry/Stop/Target used round numbers. Replaced with ATR(14)-based stops at 1.5× ATR and resistance/support-derived targets with min 1:2 R:R.
**Missing intermarket inputs.** Intermarket signal had 2/4 inputs voting. Fixed: now 4/4 (DXY, SPX, US10Y, BTC) with full breakdown.
**No autonomous execution.** Agents only fired on HTTP hit. Built Agent Orchestrator with APScheduler — agents now fire on session opens + cadences without manual triggers.
**No workflow visibility.** Clicking an agent showed raw JSON. Now: pipeline view with fetch → process → classify → publish steps.
**No memory of decisions.** Signals weren't recorded. Built Signal Recorder (SQLite) + Learning Pipeline that tracks outcomes at 1h/4h/24h horizons against 9 patterns.
**English-only narrative layer.** Shanghai session sentiment was invisible. Built Social Media Intelligence Agent that ingests Chinese sources (Jin10, EastMoney, Wallstreetcn) and translates via Haiku.
**Trump posts were a basic keyword matcher.** Upgraded to Trump PhD agent — Truth Social + executive orders + White House briefings + frequency + language patterns + history.

---

3. Architecture — one liner per layer

**Data layer:** yfinance (prices, OHLC, volume), FRED (macro), CFTC SODA (COT), BLS API (CPI/PPI/NFP), AlphaQuery (options), LME Excel, Chinese site scrapes via Playwright fallback.
**Storage:** SQLite on Railway `/tmp` for signal history; in-memory caches for hot data; TimescaleDB planned for Phase 4.
**Message bus:** Redis pub/sub channels (`master_signals`, `agent_status`, `signals`) — currently in-process, Redis container deferred.
**Compute:** Railway Hobby ($5/mo, no cold starts) for backend; Vercel for Next.js frontend.
**Orchestration:** APScheduler-driven Agent Orchestrator firing on session opens + cadences.
**LLM layer:** Haiku for classification (Chinese translation, post sentiment, Master Agent Q&A); Opus only for engineering sessions.
**API:** FastAPI REST + WebSocket. Endpoints: `/api/ohlc`, `/api/signals`, `/api/intermarket-signal`, `/api/session-signal`, `/api/agents`, `/api/diagnostics`, `/api/health`.
**Frontend:** Next.js 14, TradingView Lightweight Charts v4 + KlineChart fallback, dark theme, hamburger nav, /agents dashboard, Master Agent chatbot with voice input.

---

4. The signal engine — algorithm

4.1 Inputs (per signal generation cycle)

Price layer: real-time gold (GC=F), silver (SI=F), copper, platinum, palladium — yfinance.
Intermarket layer: DXY (DX-Y.NYB), SPX (^GSPC), US10Y (^TNX), BTC (BTC-USD).
Session layer: which of {Shanghai, LME, COMEX, Globex} is active in Dubai time; assigned weight 0.4 / 0.6 / 1.0 / 0.3 respectively.
Macro layer: real yield (10Y – 10Y breakeven from FRED), Fed Funds, CPI YoY, PPI YoY.
Positioning layer: CFTC managed money net position (z-scored vs 5-year history) + week-over-week delta.
Sentiment layer: GLD put/call ratio, Trump posts count, social media sentiment score.
Pattern layer: 9 classified patterns (e.g., "DXY down + real yields down" = high-prob gold long).

4.2 Algorithm — step by step

1. **Fetch.** Pull all inputs in parallel; reject any older than the staleness threshold (5 min for prices, 24h for macro).
2. **Validate.** Run data integrity check: gap detection, spike detection, cross-source agreement, null fallback. Health Monitor records failures.
3. **Score per layer.** Each layer outputs a directional score in [-1, +1] with a conviction in [0, 1]:
- Intermarket: weighted vote of 4 inputs (DXY inverse, SPX, US10Y inverse, BTC).
- Macro: real yield direction + inflation surprise.
- Positioning: COT z-score + WoW delta.
- Sentiment: GLD P/C deviation from 0.85 baseline.
- Pattern: closest-match pattern from learning DB with historical win rate.
4. **Session-weight.** Multiply each layer's contribution by the active session weight. COMEX hour weights 1.0; Shanghai 0.4. This means signals generated during Shanghai are softer than COMEX-hour signals.
5. **Aggregate.** Weighted sum → composite conviction in [-100, +100]. Threshold: `|conviction| > 20` to emit BUY/SELL, else FLAT.
6. **Risk levels.** Compute ATR(14) on hourly bars. Entry = current mid. Stop = entry ± 1.5 × ATR. Target = nearest swing high/low with min 1:2 R:R; if no swing, use 2.5 × ATR.
7. **Record.** Write to SQLite signal history with timestamp, all layer scores, ATR, entry/stop/target.
8. **Publish.** Push to Redis `master_signals`; WebSocket to frontend; available via `/api/signals/GOLD`.
9. **Track outcome.** Learning pipeline checks price at +1h, +4h, +24h vs entry/stop/target. Updates pattern win rates.

4.3 Honest constraints baked into the algorithm

Conviction caps at the count of healthy inputs. If only 2/5 layers report, max conviction is `2/5 × 100 = 40`.
Regime confidence reflects source count, not classifier softmax.
"FLAT" is a real output; the engine refuses to force a signal.

---

5. The 15 agents — what each does

| # | Agent | Cadence | Source | What it produces |
|---|---|---|---|---|
| 1 | Master Agent | On demand + every session open | All other agents | Synthesized buy/sell narrative; Haiku-powered chatbot |
| 2 | Session Agent | Real-time | Time-zone math | Which exchange is active, weight, next critical moment |
| 3 | COMEX Hourly Engine | Hourly during COMEX | yfinance GC=F/SI=F 1h | Opening range breakout signals, hourly H/L tracking |
| 4 | Intermarket Signal | Every 30 min | yfinance DXY/SPX/US10Y/BTC | 4/4 voting composite STRONG SELL → STRONG BUY |
| 5 | COT Positioning | Friday + ad hoc | CFTC SODA API | Managed money net, z-score, WoW delta |
| 6 | Options Sentiment | Daily | AlphaQuery GLD P/C | Put/call ratio + interpretation |
| 7 | BLS Economic | Daily + release days | BLS API | CPI, PPI, NFP, Unemployment + release calendar |
| 8 | Fed / Warsh | Daily | FRED API | Real yield, Fed Funds, FOMC calendar, Chair-watch |
| 9 | Trump PhD | Every 4 hours | CNN archive + Federal Register + WH briefings | Posts, executive orders, frequency, language patterns |
| 10 | LME Warehouse | Daily | LME Excel + yfinance fallback | Warehouse stock levels for gold/silver |
| 11 | Shanghai Premium | Shanghai open | 518880.SS ETF + USD/CNY | Physical demand proxy via Huaan Gold ETF vs COMEX |
| 12 | Health Monitor | Every 5 min | Internal checks | Null detection, auto-fix, staleness alerts |
| 13 | Diagnostics | Continuous | Health Monitor reports | 24h rolling history, uptime %, worst feeds ranked |
| 14 | Signal Recorder + Learning | Every signal | SQLite | Records signals, tracks 1h/4h/24h outcomes, 9 patterns |
| 15 | Social Media Intelligence | Each session open + every 4h | X RSS, YouTube RSS, Chinese sites | Multilingual narrative layer with bullish/bearish tally |

---

6. Agent workflow — what they actually do

6.1 A normal Dubai day

**2:00 AM Dubai — Shanghai opens.** Session Agent flips to `shanghai_main` (weight 0.7). Social Media agent fires Chinese sources (Jin10, EastMoney). Shanghai Premium agent computes ETF vs COMEX spread.
**12:00 PM Dubai — LME opens.** Session Agent flips to `lme_active`. Social Media agent fires Western analysts. Master Agent re-synthesizes.
**4:20 PM Dubai — COMEX opens.** Session weight goes to 1.0. COMEX Hourly Engine starts capturing opening range. Health Monitor tightens checks. All signals are gold-graded during this window.
**5:20 PM Dubai — first COMEX hour close.** Opening range breakout signal published if range exceeded by ATR threshold.
**Every hour through 9:30 PM Dubai.** New hourly bar → re-score → publish if conviction crosses threshold.
**9:30 PM Dubai — COMEX settles.** Settlement price recorded. Signal Recorder writes the day's outcomes. Learning pipeline updates pattern win rates.
**Weekly Friday.** COT agent pulls new CFTC report. Z-scores recomputed.

6.2 Per-agent micro-workflow

**Fetch** → raw source (HTTP, RSS, API, scrape).
**Validate** → schema check, staleness check, null check.
**Transform** → normalize units, compute derived fields (ratios, z-scores, deltas).
**Classify** → Haiku call where text is involved (sentiment, language patterns, translation).
**Cache** → in-memory with TTL; SQLite for historical.
**Publish** → REST endpoint + WebSocket + Master Agent context.
**Report** → Health Monitor logs success/failure + latency.

---

7. The signal pipeline — end to end

```
[ yfinance ] [ FRED ] [ CFTC ] [ BLS ] [ AlphaQuery ] [ LME ] [ Chinese sites ] [ X/YouTube RSS ] [ CNN ] [ Federal Register ]
│ │ │ │ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Per-agent fetch + validate │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Health Monitor (5 min) ─── catches NULLs, staleness, cross-source disagreement ─── auto-fixes when possible │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Layer scoring: Intermarket / Macro / Positioning / Sentiment / Pattern ─── each outputs [-1, +1] + conviction │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Session weighting (Shanghai 0.4 / LME 0.6 / COMEX 1.0 / Globex 0.3) │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Composite conviction → BUY / SELL / FLAT + ATR-based Entry / Stop / Target │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Signal Recorder (SQLite) → Learning pipeline tracks 1h / 4h / 24h outcomes vs 9 patterns │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ REST + WebSocket → /agents dashboard, Master Agent context, signal cards │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

---

8. Constraints we worked under

**Data cost.** $5/mo Railway Hobby + free data sources only. No Polygon, no Refinitiv, no Bloomberg.
**Latency.** yfinance is ~15 min delayed on futures. Acceptable for swing trading; not for HFT.
**Hosting.** Railway `/tmp` is ephemeral — SQLite may reset on container restart. Postgres migration deferred.
**Python version.** System Python 3.9.6 — no MCP SDK; raw JSON-RPC for the MCP server.
**No Playwright on Railway.** Forced ETF-proxy approach for Shanghai (518880.SS instead of direct SGE scraping).
**No paid data feeds.** SGE physical premium, real-time COMEX options, LME live stocks all require paid APIs we didn't buy.
**Single-user product.** No multi-tenancy, no Supabase Auth yet, no notification fan-out.
**One trader's workflow.** The whole signal weighting is calibrated to Rashid's Dubai sessions. Other users would need re-weighting.
**LLM budget.** Opus used only for engineering sessions; Haiku for all in-product classification.

---

9. Platform vs alternatives

9.1 vs TradingView

| Dimension | TradingView | Metals Trading Desk |
|---|---|---|
| Charting | Industry leader, full Pine Script ecosystem | Lightweight Charts v4 + KlineChart fallback — basic |
| Asset coverage | Every market on earth | Gold/silver + 18 watchlist instruments |
| Data feeds | Real-time on paid tiers | ~15 min delayed (yfinance free) |
| Custom signals | User-built indicators only | 15 autonomous agents pre-built + Master Agent synthesis |
| Macro context | Manual; user reads news | Automated BLS, Fed, COT, options, Trump, Chinese sentiment integrated into signal |
| Session awareness | Manual time-zone math | Built in (Dubai-anchored, COMEX-priority) |
| Narrative layer | User reads news themselves | Social Media Intelligence agent in 3 languages |
| Trade journal | Yes, paid | Signal Recorder + Learning Pipeline, free |
| Mobile | Best-in-class app | Web only, mobile-responsive |
| Community | Millions of users | Single-user product |
| **Honest verdict** | **Better for charting, breadth, real-time data, community.** | **Better for COMEX-specific autonomous synthesis and decision support.** |

9.2 vs other platforms

**Bloomberg Terminal:** Vastly better data, $24K/year. We are ~$60/year.
**Refinitiv Eikon:** Same story — institutional pricing.
**GoldChartsRus / Kitco:** Read-only news + charts. No signal engine, no agents.
**Goldhub (WGC):** Excellent for central bank data. No live signals.
**CME Direct:** Order entry + DOM. No analytical layer.
**TradeStation / Interactive Brokers:** Execution + basic analytics. No multi-source agent synthesis.
**What's unique here:** Multi-agent autonomous synthesis specifically for COMEX gold/silver, calibrated to a Dubai trader's session calendar, with a multilingual narrative layer including Chinese sources. We haven't found another product doing all of this in one place at this price point.

---

10. What's running every day

**Continuous:** Session Agent (real-time time-zone math), Diagnostics Agent (always on).
**Every 5 min:** Health Monitor.
**Every 30 min:** Intermarket Signal recompute.
**Every hour during COMEX:** COMEX Hourly Engine (4:20 PM – 9:30 PM Dubai).
**Every 4 hours:** Trump PhD, Social Media Intelligence.
**Daily:** Options Sentiment, BLS, Fed/Warsh, LME Warehouse, Shanghai Premium.
**Weekly (Friday):** COT Positioning.
**On every signal:** Signal Recorder + Learning Pipeline.
**On every user query to /agents:** Master Agent synthesizes all of the above via Haiku.

---

11. Daily debugging — what we monitor and why

**NULL watchlist prices.** Auto-detected by Health Monitor; auto-fixed by live yfinance fallback. Currently 18/18 healthy.
**Sparse cache.** Some tickers occasionally return < 20 historical rows. Patched with on-demand fetch fallback.
**yfinance cloud reliability.** Railway IPs occasionally throttled. Retries with exponential backoff; Render mirror set up as standby.
**Sticky regimes.** When macro source count is low, regime classifier was over-confident. Patched: confidence now scales with `n_sources / 5`.
**Trump agent zero-post days.** CNN archive layout shifts break parsing. Logged; we re-verify selectors weekly.
**Shanghai ETF lag.** 518880.SS NAV trails real-time COMEX. We report directional spread but warn on magnitude.
**COT 10-day lag.** Inherent — CFTC publishes weekly with Tuesday cutoff, released Friday. Surfaced in UI with "data as of" tag.
**GLD as options proxy.** GLD is a proxy for COMEX gold options. Surfaced as a caveat in the UI.
**SQLite ephemerality.** Railway `/tmp` may wipe on restart. Monitored; Postgres migration is the next infra task.
**Railway build cost.** Each deploy is ~$0.05; budget tracked per session.

---

12. Engines running on this platform

**Signal Engine** — composite buy/sell/flat with conviction and ATR-based risk levels.
**Session Engine** — Shanghai / LME / COMEX / Globex detection with weights.
**COMEX Hourly Engine** — opening range breakout, hourly H/L tracking, volume confirmation.
**Intermarket Engine** — 4-input weighted vote (DXY / SPX / US10Y / BTC).
**Regime Engine** — Dalio 4-quadrant (growth × inflation) with honest confidence.
**Learning Engine** — pattern matching against signal history with 1h/4h/24h outcome scoring.
**Health Engine** — silent failure detection + auto-fix.
**Diagnostics Engine** — 24h rolling uptime + worst-feed ranking.
**Translation Engine** — Haiku-driven Mandarin → English for Chinese sources.
**Narrative Engine** — Master Agent synthesis across all agent outputs.

---

13. What enhances value next

**TimescaleDB + Redis containers** — proper time-series storage + cross-agent pub/sub. Replaces SQLite-on-tmpfs and in-process pub/sub.
**Postgres migration on Railway** — survive container restarts.
**Twelve Data subscription ($29/mo)** — 1-min intraday bars + WebSocket. Removes the 15-min delay on COMEX.
**Polygon.io ($199/mo)** — true real-time COMEX with depth-of-book. Phase 4 only.
**TradingView Advanced Charts license** — replaces Lightweight Charts limitations.
**Notification agent** — Resend email + web push on signal threshold crossings.
**Pattern Agent / Daily Brief** — once Learning Pipeline has 30+ days of recorded outcomes.
**Multi-user auth** — Supabase Auth for paid access.
**Paper trading mode** — record hypothetical fills against signals; compute realized P&L.
**Backtest harness** — replay agent outputs against historical OHLC to validate weight calibration.
**Top analysts engagement-verified list** — once X/YouTube API keys are wired in, replace the curated list with actual top-engagement accounts per session.
**MCP server hardening** — currently 12 tools, raw JSON-RPC; upgrade to Python 3.12 + official MCP SDK.
**Mobile app** — React Native wrapping the same API.
**Compliance layer** — KYC / risk disclosures if we monetize.

---

14. Verification artifacts (latest live numbers)

Gold: $4,530.70 — yfinance GC=F
Silver: $77.17 — yfinance SI=F
DXY: 99.227 — yfinance DX-Y.NYB
US10Y: 4.586% — yfinance ^TNX
SPX: 7,445.72 — yfinance ^GSPC
VIX: 16.76 — yfinance ^VIX
BTC: $78,131 — yfinance BTC-USD
COT MM net: +98,015 (88th percentile, +3,761 WoW) — CFTC SODA, report dated 2026-05-12
GLD put/call: 0.791 — AlphaQuery
CPI YoY: 3.78% — BLS API
PPI YoY: 5.99% — BLS API
Real yield (10Y): 2.13% — FRED
Fed Funds: 3.75% — FRED
Active session at 10:30 AM Dubai: Shanghai main, weight 0.7
Intermarket signal: 0/4 bullish → STRONG SELL composite
Trump PhD: 241 posts / 7d, 34 executive orders YTD, 1 language pattern alert
Watchlist health: 18/18 instruments returning real prices
Health Monitor: every 5 min, auto-fix on NULL
Signal Recorder: SQLite, 9 patterns tracked

---

15. Closing — honest self-assessment

**Where we beat TradingView:** Autonomous multi-source synthesis, COMEX session-priority weighting, multilingual narrative including Chinese sources, integrated Trump / Fed / BLS event tracking, ATR-based honest risk levels, learning pipeline that records outcomes.
**Where TradingView beats us:** Charting depth, real-time data, asset breadth, community, mobile experience, indicator ecosystem, execution integration.
**Where we are honest about limitations:** ~15-min delayed data, ephemeral SQLite, ETF proxies for SGE, GLD proxy for COMEX options, single-user scope, free-tier source reliability.
**What we will not do:** Invent numbers, fake confidence, ship signals without data integrity checks, or claim parity with paid feeds we don't have.
**The moat:** Data integrity discipline + session-aware synthesis tuned to one real trader's workflow, end-to-end transparent (every number traceable to a source).