The problem
Stock Copilot does five things every day: refresh price data, run AI-powered technical analysis on every tracked stock, generate personalized audio briefings, and push notifications to users — all before they wake up the next morning.
The naive approach is to build one giant function that does everything. The moment that function fails at step 3, steps 4 and 5 die with it. Users get no briefing, no notification, no explanation. You’ve coupled five concerns into one blast radius.
What the system actually needed
Three things became clear early:
Sequencing with isolation. Each stage depends on the previous one’s output — analysis needs fresh prices, briefings need fresh analysis. But a failure in briefing generation should never corrupt the price data that was already written successfully.
Controlled parallelism. Analyzing 500 stocks sequentially takes hours. Analyzing all 500 in parallel overwhelms the API rate limits and the function memory. The system needed a middle ground.
Predictable timing. Users check their briefing in the morning. The system had to finish all work by then, every day, without manual intervention.
How it became an orchestration layer
The orchestration is a time-driven sequential pipeline running on Firebase Cloud Scheduler:
5:15 PM EST — OHLC cache refresh. Pull latest candlestick data for every tracked symbol. Write to Firestore. Done.
5:30 PM EST — Daily analysis. Read the freshly cached prices. Run technical analysis (RSI, MACD, moving averages) and AI-powered pattern recognition through OpenAI. Write results to stockAnalyses/{symbol}/swing_{date}. Process 5 stocks in parallel, queue the rest via Pub/Sub workers.
7:00 PM EST — Briefing generation. Read each user’s watchlist. Pull their stocks’ analysis. Generate a personalized script. Convert to audio via OpenAI TTS. Upload to Firebase Storage. Write the briefing document to Firestore.
7:30 AM EST — Push notifications. Check which users have new briefings. Send via FCM.
Each stage is a separate Cloud Function. Each writes its output to Firestore. The next stage reads from Firestore. The database is the contract between stages — not function calls, not shared memory.
Three product decisions and the reasoning
1. Time-driven, not event-driven. The obvious “modern” choice is event-driven: price data lands, triggers analysis, triggers briefing. I chose scheduled execution instead. Why? Because the stock market closes at 4 PM. There’s a natural daily rhythm. Event-driven would add complexity (deduplication, ordering guarantees, partial-trigger handling) for zero user benefit. The market doesn’t produce surprise data at 2 AM.
2. Pub/Sub workers for the analysis fan-out, not parallel function calls. Firebase Cloud Functions have a 9-minute timeout. Analyzing 500 stocks sequentially exceeds that. Spawning 500 parallel functions would blow through API rate limits. The solution: an orchestrator function publishes batches of symbols to a Pub/Sub topic. Worker functions subscribe and process 5 stocks at a time. Backpressure is built in. If a worker fails, only its batch is lost — the rest complete.
3. Firestore as the inter-stage contract, not in-memory handoff. Each pipeline stage writes to Firestore and the next stage reads from it. This means every stage is independently retryable. If briefing generation fails at 7:00 PM, I can re-run it at 7:15 PM — the analysis data is already sitting in Firestore, intact. It also means the iOS app and webapp can read intermediate results. Users see fresh analysis even if the briefing stage hasn’t run yet. The database isn’t just storage — it’s the orchestration boundary.