I Asked ChatGPT to Explain Its Own Blind Spots. It Was More Honest Than Most Quotes I've Seen.
Completed concrete pool build by Cronulla Pools, Sydney pool builders.
We've been talking a lot lately about how people compare pool quotes. Most of our clients aren't just getting three quotes anymore, they're pasting them into ChatGPT and asking it to tell them which one's the best deal.
So I decided to ask ChatGPT directly what it's actually bad at when it comes to comparing pool quotes. Here's what it told me:
IT ANCHORS ON THE NUMBER, NOT THE SCOPE
"If I'm handed three quotes, my first instinct is to compare totals or line items that share a label. But 'electrical, $5,000' and 'electrical, included' aren't comparable unless I know what's actually being wired. I'll often just line them up as if they're equivalent because they occupy the same row."
Two quotes can use the exact same word for wildly different scopes of work, and on a spreadsheet they look identical.
IT DOESN'T KNOW WHAT'S MISSING UNLESS SOMEONE TELLS IT
"I can only reason about what's on the page. If a quote leaves out engineering detail, a heater upgrade, or a maintenance period entirely, I have no way of flagging that it's conspicuously absent unless I already know that category needs to exist in every legitimate quote. A first time pool buyer doesn't know that either, so we'd both be missing the same blind spot at the same time."
Most people building their first pool don't know what should be in a quote, so they can't spot what's been quietly left out of it, and neither can the tool they're using to check their work.
IT DEFAULTS TO CHEAPEST EQUALS BEST
"Without extra context, ranking by lowest total is the easy, obvious answer. I won't naturally interrogate why one number is so much lower, I'd need to be explicitly prompted to ask why before I'd treat that as suspicious rather than a selling point."
We see this constantly. The lowest number wins the comparison every time, right up until the extras start rolling in a few weeks later.
IT CAN'T JUDGE THE SITE, THE TRADES, OR THE RISK
"A quote might look complete on paper but assume a straightforward dig on flat, easy access land. I have no way to know that a specific block has rock, tight access, or a stormwater line that will blow the budget out later, because that's not information contained in the quote itself, it lives in a site inspection."
No spreadsheet has ever met your backyard. We have.
IT TAKES "INCLUDED" AT FACE VALUE
"If a line item says included, I read that as a settled fact. I can't tell if included means done properly, done at minimum spec, or quietly downgraded to hit a number. That word carries no information about quality unless someone unpacks it."
One word, doing an enormous amount of work, and nothing forcing it to be honest.
IT HAS NO MEMORY OF BUILDS GONE WRONG
"A person who's built or sold pools for years has a gut reaction to a suspiciously low number, because they've seen what happens six weeks later when it wasn't real. I don't have that instinct. I only have the document in front of me."
That instinct is the entire reason Scott and Matt still walk every quote comparison by hand, even with all this technology sitting right there.
THE PART THAT SURPRISED ME MOST
I didn't expect a chatbot to sum up its own limitations this cleanly, but it landed on something true. It's genuinely good at comparing what's written down, and genuinely bad at knowing what should have been written down but wasn't.
So bring us your quotes. All of them, side by side, extras and all. We'll go through them properly, no hard sell, just an honest read on what's actually being compared, because that's the one part of this whole process no AI is ever going to do for you.
