November 2025 · 4 min read
Data Is Just Data. The Hard Part Starts After the Pipeline Works.
The tooling is commoditized. A competent team can stand up a modern stack in a quarter. The hard part is what happens the day after the pipeline works. And it has nothing to do with the data.
The POS data lands clean. Schemas conform. Freshness is green. Every pillar in the business is looking at the same numbers, or could be.
Then a pillar lead tells me they’re going to write their workload in Python. Not because the job needs Python. Because they want to work outside the platform. The transformations could be SQL. They should be SQL. SQL is what the rest of the business runs on, what the rest of the business can monitor, what the rest of the business can reuse. But SQL means conforming, and conforming means they can’t make unilateral calls on the baseline data anymore.
So they pick Python. Not for the problem. For the autonomy.
This is the scar I keep coming back to, because it has nothing to do with the data and everything to do with the job of leading in this space.
The claim
Data is a solved problem. Snowflake, Databricks, dbt, Fivetran, the LLM layer on top of it. The tooling is commoditized. A competent team can stand up a modern stack in a quarter. Anyone who tells you the technology is the hard part is selling you the technology.
The hard part is what happens the day after the pipeline works. Who gets to decide what “correct” means. Whose roadmap survives contact with the shared infrastructure. Whose budget funds the cleanup when a pillar lead’s Python sprawl breaks a downstream report that finance needed yesterday.
People. Process. Politics. That is the job.
The blackjack table
Picture your enterprise as a blackjack table. Product managers are the players. Analytics resources are the chips. The goal is to beat the dealer, which is the market.
Now picture a coach standing behind each player. The coach is your functional analytics lead. The CX lead. The DTC lead. The merchandising lead. Each coach is wearing high-tech blinders. They can only see their own player’s two cards. Not the dealer’s up-card. Not the other players’ hands. Not the shoe.
The player is holding a hard 16. The coach, by the book, says don’t hit. Technically correct in a vacuum.
But the player is not in a vacuum. They can see the dealer is showing an Ace. They can see the player next to them, representing another pillar, holding a hand that perfectly complements their own. Pooling chips and taking the risk is the optimal play for the table.
The coach forbids it. “That’s not on our roadmap. Use your chips on the CX plan. Protect our budget.”
The person with the most information is disempowered by the person with the least. The optimal play for the company is sacrificed for the suboptimal play of a single pillar.
Every day, in every business still clinging to the pillar model.
What the Python thing costs
When a pillar lead writes everything in Python to stay outside the platform, the costs don’t show up on their dashboard. They show up on everyone else’s.
Scalability. Their Python job runs on their laptop, then on a VM, then on three VMs nobody owns, then on a Kubernetes cluster the data platform team is now on the hook to babysit. The same transformation in SQL would scale on the warehouse the business already pays for.
Monitoring. The platform has lineage, freshness checks, alerting, a runbook. Their Python script has none of that. When it breaks at 2 a.m. on a Saturday, nobody knows it broke until Monday morning when a VP sends the “why are the numbers wrong” email.
Shareability. Something written on the platform is something another pillar can reuse, fork, or verify. Something written in a pillar lead’s private Python project is something only they understand and only they can change. The same transformation gets rebuilt three more times by three more pillars who didn’t know it existed.
None of this is a data problem. Every one of these is a leadership problem wearing a data costume.
The job, restated
The hardest decisions you will make are about who gets to say no, who gets to go around the platform, and what you do on the day a senior pillar lead tells you they are doing it their way because they can.
Saying yes is the path of least resistance. It buys a week of peace and a year of sprawl.
Saying no, specifically and publicly, with a reason the business understands, is the work. Nobody outside our function calls this leadership. They call it being difficult, being slow, being a blocker. Until the day the numbers break and someone has to put them back together. Then they call it judgment.
That is the hard part.