I've been using AI coding assistants, mainly GitHub Copilot, for over a year now, and they're genuinely impressive overall.
GitHub Copilot can autocomplete my functions with decent accuracy. And it can even predict (somehow) the very next update that I was thinking of making.
And ChatGPT can debug my React components faster than I can say "useEffect dependency array." These tools have made me more productive, and I'm not here to diminish their value.
But there's one thing that AI is terrible at doing by default: understanding your companies broken business logic.
The AI hype cycle wants you to believe that software development is just about writing code. Feed the machine some requirements, get some functions back, profit, repeat.
It's a compelling narrative, especially when you see AI effortlessly generating sorting algorithms or building todo apps that look suspiciously similar to every other todo app tutorial on the internet.
Here's what they're not telling you: real software isn't built from requirements, it's built from compromises and often poorly thought out assumptions made by many people.
Every meaningful piece of software I've ever worked on has been shaped more by what we couldn't do than what we could do.
The legacy database that can't be touched because it powers the CEO's favorite dashboard.
The "temporary" workaround from 2019 that became load-bearing architecture and needs to be maintained.
The customer who pays 40% of your revenue and has very specific ideas about how the UI should work. And you account for each and every one of those ideas.
And the 4 different payment gateways that had to be implemented to make each of the customers happy.
Is this an ideal architecture in the grand scheme of things? Probably not. But it works and it keeps the monthly revenue steadily coming in.
AI, for now, excels in the clean room of isolated problems. "Build me this new thing that doesn't require any previous context please".
But software development is fundamentally about navigating the messy, contradictory, politically-charged reality of human organizations.
The context problem is harder than you think
Consider this seemingly simple requirement: "When a user cancels their subscription, revoke their access immediately."
Sounds straightforward, right? Any AI could write that function. But here's what the algorithm doesn't know:
- The VP of Sales promised Enterprise customers a 30-day grace period that was never documented formally, because it was suppose to be temporary.
- Marketing runs campaigns that depend on cancelled users still receiving certain emails.
- The accounting team needs cancelled subscriptions to remain "active" until the billing cycle ends for revenue recognition.
- Legal requires certain data to be retained for compliance, but other data must be deleted immediately.
- The support team has been manually extending access for "VIP customers" who call and complain.
- Some new privacy law was just passed that might affect 10% of your customers, maybe, kind of, sort of.
Now that simple cancellation becomes a political negotiation between five different departments, each with legitimate but conflicting needs.
The "correct" implementation isn't a technical decision in this scenario. It's a business decision that requires understanding organizational dynamics, regulatory constraints, customer relationships, and revenue implications.
AI can't attend your cross-functional meetings (yet). It can't read the room when legal and marketing are having a turf war. It can't understand that the CTO's "suggestion" is actually a mandate.
Frankly, the algorithm doesn't know what questions to ask you so that it can actually generate something useful.
Where it breaks down
Here's where AI tools really start to break down: they optimize for confident answers, but the most valuable developers are the ones who ask uncomfortable questions.
When someone hands you a feature request, the AI says "sure, I can build that." The experienced developer says:
- "Who asked for this and why?"
- "What happens when this scales to 10x our current usage?"
- "How does this interact with the deprecation plan for the legacy API?"
- "What's the fallback if this third-party service goes down?"
- "Have we considered the accessibility implications?"
- "What happens when users inevitably try to game this system?"
These aren't technical questions, they're business questions. And answering them requires understanding not just the codebase, but the company, the market, the users, and the dozens of unwritten rules that govern how things really work.
AI afficionados love to argue that we just need better specifications. If we could perfectly describe what we want, they claim, AI could perfectly implement it. This reveals a fundamental misunderstanding of how software development actually works.
Software specifications aren't discovered, they're negotiated.
Every feature starts as a vague idea ("we need better analytics") and evolves through countless conversations, prototypes, user feedback sessions, and reality checks. The final implementation looks nothing like the initial "requirement" because the act of building software teaches you what you actually need.
I've never worked on a project where the final product matched the original specification, and that's not a failure, it's a feature. Good software development is an iterative learning process, not a translation exercise.
AI can translate requirements into code, but it can't participate in the messy human process of figuring out what those requirements should be in the first place.
And that's because AI models still lack temporality. What do I mean by that?
As I said above, most features that need to be coded out are carefully molded day by day and meeting by meeting by various people. And this takes time. We humans are surprisingly effective at working within the constraints of time.
We remember that the choices made on Monday, will have a strong influence on the data that we see on Wednesday. And for that reason, we take that chain of events into account when building something new.
AI models by default have no sense of time. They live in the moment and respond as such. And you can imagine that feeding a model your entire 8 hour work day every single day until the end of time, is eventually going to hit some performance bottleneck.
Until AI can somehow chunk your life into compacted packets of sequential data, it probably wont be able to make business decisions for you.
At the end of the day, code isn't just about making computers do things, it's about making organizations work. And organizations are fundamentally human systems with human politics, conflicts, and compromises.
Embrace the Mess
AI can write your boilerplate. It can generate your test cases. It can even refactor your legacy code (sometimes). But it can't sit in a meeting with Sales, Marketing, and Legal and figure out how to make everyone happy while still shipping something that users will actually want to use.
That's your job. That's why you're here. And that's why, despite all the hype about AI replacing developers, the demand for people who can do that job keeps growing.
The algorithm will never understand your business logic because business logic isn't logical. And humans, messy and contradictory as we are, still need other humans to build software that works in the real world.