Designing Data Platforms That Scale With the Business

I'm Tom Lang — I've built and scaled engineering organizations for three decades, from startups to 250-plus engineers and through multiple acquisitions, and I work as a fractional and interim CTO for growth-stage companies, with particular depth in regulated and integration-heavy environments.

Most data platforms are built backwards. Startups optimize them for the dashboard they need to show investors tomorrow, rather than the machine-learning models they'll need to train in two years. Then the pivot comes, or the AI initiative starts, and the platform that looked fine turns out to be a prison.

This is the data-architecture-specific companion to modularity and scalability — the same "build so you can change your mind" instinct, applied to the platform your analytics and AI actually run on. Here's how I build data architectures that don't become prisons when the business scales.

The one rule: immutable raw data, decoupled compute

The single rule that separates a scalable data platform from a technical-debt trap: never overwrite raw data, and strictly decouple storage from compute.

When you ingest data, treat it as an immutable, append-only log. The tempting mistake is to aggregate an event the moment it arrives — to save space — and discard the raw event. Do that and you've permanently destroyed the fidelity of your history. When the business pivots next year and wants to analyze that data a different way, you simply can't; the detail is gone.

So store everything raw, first, in cheap object storage like S3. You can always re-derive a new aggregated view from raw data. You can never reverse-engineer raw data back out of an aggregation. Cheap raw storage today is what buys you optionality forever.

The three breakpoints (and what breaks at each)

Data architecture scales through three distinct, painful breakpoints — the data-platform version of the scaling walls I wrote about for engineering orgs.

Stage 1 — the Postgres monolith. You run your app and your analytics on the same production database. What breaks: the business team runs a massive GROUP BY on a Friday afternoon, locks the tables, and takes down the live customer-facing site. That's your signal to physically separate the transactional store (OLTP) from the analytical one (OLAP) — heavy analytics never belong on the database serving your customers.

Stage 2 — the modern data warehouse. You pipe everything into a warehouse like Snowflake or BigQuery. What breaks: cost and rigidity. You start paying a real premium to store semi-structured JSON in a proprietary warehouse, your ETL pipelines congeal into a tangle of spaghetti SQL, and every new data source triggers a three-week schema-design project before anyone can use it.

Stage 3 — the event-driven lakehouse. You decouple everything. Data streams (via something like Kafka) into a data lake on cheap object storage, and you spin up independent, ephemeral compute clusters only when you actually need to query — you pay for the compute you use, when you use it, without it ever threatening ingestion or the live app.

The AI delusion: it's not a new database

Founders often assume that "building for AI" just means buying a vector database and dumping their PDFs into it. That's the delusion. AI does not demand a new database; it demands a militant data-quality pipeline.

Feed a retrieval-augmented (RAG) system raw, unversioned, undocumented data and you don't get AI — you get a highly confident, hallucinating chatbot. What AI actually demands is the Bronze/Silver/Gold discipline I described in right-sized governance: raw data (Bronze) is programmatically cleaned and standardized (Silver), and only highly validated, business-ready data (Gold) is ever embedded into the vector store for the model to read. The vector database is the easy part; the pipeline feeding it is the whole game.

The 4-terabyte scar

In 2012, running Videology's platform at three billion transactions and four terabytes of data a day, I learned the hardest lesson of my career about schemas.

We started the way most teams do: heavy, traditional ETL pipelines that cleaned and structured the data before inserting it into a big, rigid relational data warehouse. The problem was our upstream data providers, who changed their XML and early-JSON formats constantly, without telling us. Our strict database schemas rejected the malformed payloads, and our ingestion pipelines broke hourly. The entire engineering team turned into full-time plumbers, manually fixing brittle ETL jobs just to keep the nightly batch loads from crashing.

We survived by flipping the architecture. We abandoned the relational bottleneck and moved to schema-on-read: we spun up a large Hadoop cluster and blindly dumped the raw, messy log files straight into HDFS as fast as they arrived, then applied schema and structure only at the very last step — with Hive, when an analyst actually ran a query. That single change moved the failure point off the mission-critical ingestion pipeline, and it taught me the rule I still design around: raw storage is cheap, but brittle schemas are fatal. Never let a rigid schema stand between you and getting your data safely stored.

The honest goal

So build the platform forward, not backward: store raw and immutable, decouple storage from compute, cross each breakpoint deliberately instead of being surprised by it, and feed your AI clean, validated data rather than a vector database full of hope. Do that and your data platform scales with the business — ready for the analytics you need today and the models you'll want in two years. Get it wrong and every future ambition runs headfirst into an architecture that can't bend.

If you're designing a data platform now — or your analytics are taking down production, or your "AI initiative" is really a data-quality problem in disguise — that's exactly the kind of architecture I've built at scale. Book a call and we'll design yours to bend instead of break.