AI & Industry · 2 Aug 2026 · 9 min read

AI Made It Harder to Spot a Good Engineer

Two developers show you a demo. Both look great. One of them will cost you a rebuild in eighteen months, and there is nothing on the screen that tells you which one. Here is what a business owner can still check.

Heartbyte Team

Engineering & Strategy

A while back we wrote that engineer quality runs on a scale from -1 to +1, and that AI multiplies whatever number you already are. Plenty of engineers read that and asked themselves what their number was. Fair enough. But almost nobody asked the harder question, and it's the one that costs real money.

If you're the one paying, how do you tell what number you're hiring?

You're a business owner. You need a system built. Two developers sit in front of you, or two vendors send proposals. Both show you something that works. Both have a portfolio full of clean screens. Both talk confidently and quote a similar price. One of them is going to give you something you'll still be running happily in five years. The other is going to give you eighteen good months and then a bill for a rebuild.

Two years ago you had a few rough ways to tell them apart. Most of those are dead now.

"AI didn't make bad engineers good. It made them look good — and looking good was the only thing most buyers ever had to go on."

The signals you used to trust

None of the old checks were great. They were just correlated with skill, and that was enough. Every one of those correlations has now broken, because the thing that used to take skill takes a prompt.

What stopped working

▸ The demo. A working app used to mean weeks of real effort. Now a weekend gets you a polished one, and polish was the whole signal.
▸ The portfolio. Beautiful screens are the cheapest thing in software now. They tell you nothing about what's behind them.
▸ The take-home test. Every candidate now submits clean, commented, tested code. Everybody scores well, so the test sorts nobody.
▸ The code sample. You couldn't read it before and you can't read it now. The difference is that it used to be theirs.
▸ Speed. "They delivered in three days" was once a sign of a strong developer. Now it might mean they didn't read what they sent you.
▸ The vocabulary. Anyone can sound senior for forty minutes. The words are free and they're in every model's training data.

Notice what all of those had in common. Every one was a measure of output, and output is exactly the part AI got cheap. We've said before that code got cheap and software didn't. This is the hiring version of that. The thing you were measuring stopped being scarce, so measuring it stopped telling you anything.

Why picking wrong costs more than it used to

Here's the part that makes this urgent instead of just annoying. The old signals broke at the same time the penalty for a bad hire went up.

A weak developer used to be slow. That was a kind of accidental protection. They could only do so much damage in a quarter, and you'd usually notice the mess before it got big, because the mess arrived slowly. That brake is gone. A weak developer with AI ships at the same speed as a strong one. Same volume, same green ticks, same confident status updates in your Monday meeting. The problems just go in faster and deeper.

So the gap between a +1 and a -1 didn't stay the same size. It grew. Both of them now come with a 10x multiplier attached, pointing in opposite directions.

Same six months, same budget, same demo

▸ The +1 hands you a system where the next feature is cheaper than the last one, because the base is clean and somebody understood every decision in it.
▸ The -1 hands you a system where the next feature costs three times the last one, and nobody alive can explain why the thing works.

On day one, from the outside, these look identical. That's the whole problem.

The demo was never the product

A demo shows you the happy path. One user, clean data, good wifi, nothing weird. That was always true, and it was always a bit misleading. What changed is how much easier it got to build something that only survives the happy path.

Everything you actually pay for sits outside the demo. What happens when two people edit the same record at once. What happens at month nine when the table has four million rows in it. What happens when your accounts person types a date in the wrong format, or the payment gateway times out halfway. What happens when someone leaves the company and you need their access gone by Friday. What happens when you want to change one rule that turns out to be copied into eleven places.

None of that shows up in a twenty minute call. All of it is where the money goes. We've written about the shipping version of this before: it looks like it works, so they commit. You're now on the receiving end of that same mistake, except you're paying for it.

"You're not buying the screens. You're buying the next three years of changes to them — and that price is set on day one, by decisions you'll never see."

What still works: stop looking at the work, look at the thinking

Good news. There is still a signal, and you don't need to be technical to read it. AI can produce the artefact. It can't sit in the room and defend the reasoning behind it under follow-up questions from someone who's genuinely curious.

So stop trying to judge what they built. Judge whether they understand what they built. These questions work in a hiring interview and they work in a vendor pitch, and you can ask every one of them without knowing how to code.

Six questions that still sort people:

✓ "Walk me through a decision you made here and why you rejected the other option." Then ask why again. And again. A +1 gets more specific each time and eventually mentions a real trade-off they weren't happy about. A -1 gets vaguer and starts repeating adjectives.
✓ "What did you decide not to build, and why?" Everything good has stuff left out on purpose. Someone who only ever added things has never had to choose, which means they've never really owned a system.
✓ "Tell me about something you shipped that broke. What broke, and what did you do?" Anyone who's built real things has an outage story and tells it in detail, usually with a bit of embarrassment. A polished "nothing major, we test thoroughly" is the answer of someone who hasn't been on the hook for anything yet.
✓ "Where would this get slow or fall over if my business grew ten times?" A strong answer is immediate and specific, because they already thought about it while building. A weak answer is a reassurance. Reassurance means they never asked themselves the question.
✓ "If I asked for this one change, what would it touch?" Pick something small and awkward. Someone who understands their own system answers in about ten seconds and names the parts. Someone who doesn't will say they need to check.
✓ "Explain this bit to me like I'm not technical." The best engineers can. It's the strongest proof of understanding there is, and it's the hardest thing to fake, because you can't simplify something you only half-follow.

You're not grading the answers for correctness. You couldn't, and you don't need to. You're listening for whether the answers get sharper as you dig or thinner as you dig. That direction is the whole test, and it's very hard to fake for more than about five minutes.

Red flags that got more useful, not less

A few things always meant trouble. In the AI era they mean more trouble, because there's no longer a slow pace holding back the consequences.

Walk away signals

▸ Nothing is ever a trade-off. Every option you raise is "no problem, easy". Real engineering is full of costs. Someone who never mentions one is either hiding them or hasn't found them yet.
▸ They can't say what they don't know. Strong people say "I'd have to check how your payment provider handles that" without flinching. Total certainty across every topic is a tell.
▸ Estimates that don't move. If a two-week job and a two-month job both come back as "about a month", the number isn't based on anything.
▸ They talk about tools, not your business. Half an hour on their framework and no questions about how your orders actually flow. They're going to build what they know instead of what you need.
▸ Nobody owns the code once it's handed over. Ask who answers the phone at 2am in year two. If the answer is fuzzy, the answer is nobody.

That last one deserves its own paragraph, because it's where most of the pain in this industry actually lives. We've argued before that your vendor isn't your partner, and AI hasn't changed that. It's just made the gap between the two easier to hide during the sales process.

The one check that beats every interview

If you only do one thing from this article, do this one. Don't hand over the whole project on the strength of a pitch. Buy a small piece first, pay for it properly, and watch how they work.

Give them something real but contained. One report, one integration, one screen that talks to a system you already have. Two weeks of work, priced fairly, no discount asked for. What you learn in those two weeks is worth more than any interview, because you get to see the parts that don't fit in a demo.

What the trial job shows you

▸ Do they ask questions before they start? A good one comes back with three awkward questions about your process. A bad one starts building immediately, because they never noticed the gaps.
▸ Do they tell you bad news early? Or does everything look fine until the deadline, then slip?
▸ Does it survive contact with your staff? Let the person who'll use it every day try to break it. They will, and how the developer reacts tells you plenty.
▸ Then change your mind on purpose. Ask for a small change in week two. How they handle a change request is how they'll handle three years of them. If a tiny tweak triggers a big quote, you've learned something valuable for the price of two weeks.

A two-week trial that goes badly costs you two weeks. Picking the wrong partner for a year-long build costs you the year, the budget, and usually the rebuild after it. That's not a close call.

Don't ask whether they use AI. Ask how

Some buyers have started asking vendors to promise they won't use AI. Don't do that. You'd be filtering out the best people and keeping the slowest ones, and the honest answer is that everyone worth hiring uses it. We use it heavily.

The useful question is about ownership. Ask what happens between the AI producing something and it reaching your system. If the answer involves a human reading every line and being able to defend it, you're talking to a +1 who got faster. If the answer is a shrug and "the tests pass", you're talking to someone about to multiply themselves in a direction you won't like. We wrote about what that output looks like in AI slop, and about the bill it eventually generates in technical debt.

"The question was never whether a human typed it. It's whether a human will answer for it when it breaks."

The bottom line

The scale still runs from -1 to +1, and AI still multiplies whatever's there. Nothing about that changed. What changed is your side of the table. The cheap ways you used to guess where someone sat on that scale have all stopped working at once, and the cost of guessing wrong went up at the same time.

So the job moved. You can't evaluate the output anymore, because the output is free and it all looks the same. You evaluate the person: how they reason out loud, what they admit they don't know, what they chose not to build, and how they behave on a small paid job before you bet the year on them.

That's slower and more awkward than looking at a nice demo. It's also the only thing left that works.

"Everyone can show you working software now. Very few can tell you why it works — and that gap is the only thing you're really buying."

Trying to pick between two proposals?

Ask us the six questions above. We'll answer all of them, and we're happy to start with a small paid piece so you can see how we work before you commit to anything bigger.

Talk to Us About Your Project