We spent 3 weeks testing whether AI could do our design work
Where it genuinely earns its place in Amazon creative, where it quietly costs you trust, and the 13 tests this year where shoppers called an image fake without being asked.
Last month, I went full nerd and did something I’d been wanting to do for years. I used Claude to analyze 900+ PickFu split testing polls (now 991) from 34,000 respondents across the 850+ eCommerce brands we’ve worked with. The result was a database that lets me look up trends, spot patterns, and answer almost any question about a client’s creative before we even start designing. This has reframed how my team and I make every creative decision and I’m sharing all of it here in Design Proof. Twice a week: data-backed creative insights to help you convert better on Amazon and eCommerce.
A few weeks ago we ran an experiment inside the agency. We wanted to know if we could use AI to generate the first concepts for a client’s A+ content (the detailed image sections further down an Amazon listing) the same way we already use it for main images. Main images have been a real win for us with AI. So we assumed A+ would follow.
It didn’t. And I want to walk through why, because our honest experience contradicts almost everything you’re seeing about AI design on LinkedIn and socials right now.
Here’s what we ran into. To get an A+ concept out of AI, a skilled human with design training still has to sit there and prompt it, round after round, guiding every version.
This means the entire time that person is prompting, they could just be designing the thing instead. The AI didn’t save the work, it just turned designing into prompting.
And the burn is that the versions that came out of AI in “1-click” weren’t even close to the quality of what our team ships. Interesting to look at, occasionally useful for sparking an idea, but never something I’d put in front of a client as a finished direction.
When you show a 7-figure brand an AI concept and they’re expecting the quality they hired you for, you’ve set the wrong first impression before the project even starts. We’d spend the call explaining that the AI versions were for concepting, which is just not the best use of anyone’s time.
This is where I want to talk about the thing everyone’s posting. You’ve seen it:
“Claude built this in five minutes and it’s as good as [famous brand].”
And sometimes the image genuinely looks great BUT here’s the part that they left out. That famous brand already spent tens of thousands, sometimes millions, building their creative over years, and all of that work is online. When AI makes something that looks on-brand for them, it’s remixing an asset library someone else already paid a fortune to build.
Now hand that same AI image to that brand’s actual design team. They’ll come back with twenty edits before it’s allowed anywhere near a live listing. And here’s what the demo videos never show you: you go to make those twenty edits by prompting, and every time you fix one thing, the AI changes something else you didn’t ask it to touch. You end up re-checking the entire image after every round to make sure it didn’t quietly break. It becomes a longer process than building the design properly in the first place.
Here’s a story that made this real for me.
I had two clients around the same time. One in the kids category. One in home goods. Completely different products, completely different customers, nothing in common except that they’re both Amazon sellers.
At some point in the process, both of them went off and made their own images in ChatGPT. Neither one knew the other existed. And when they came back to me with what they’d made, the images looked almost identical to each other. Same lighting. Same composition instincts. Same overall feel and colors, even though a five-year-old and a fifty-year-old were supposed to be looking at these two listings.
Think of it like this: if two strangers walked into two different stores, in two different cities, shopping for two different things, and came out wearing the same outfit, you’d assume they shopped at the same place. That’s what’s happening with AI-generated creative right now. Different brands, different categories, same store.
Since then I’ve started noticing it everywhere, I can’t unsee it. I’ll scroll LinkedIn and see creative from brands I’ve never worked with, in categories I don’t touch, and I can tell within a second that it came out of the same tool with the same defaults and even the same fake AI models. The aesthetic is becoming a tell.
My prediction: this gets worse before it gets better. More brands are going to lean on AI to move fast, and more of them are going to end up looking like slightly different flavors of the same brand. On Amazon, where you’ve got half a second to earn a click, that’s the opposite of what you want. You’re not trying to look like everyone else’s AI output. You’re trying to look like the one option a shopper actually stops for.
In our own testing, these one-click tools get it right maybe 70 to 80 percent of the time. That sounds high until you remember the last 20 percent is exactly where brand consistency lives, the part a shopper actually notices.
And it’s not just us seeing this. Adobe surveyed more than 16,000 creators in 2026: 57 percent said their AI outputs need moderate or extensive editing before they’re ready to share, and 81 percent said human judgment is still what decides whether something is good enough to publish.
But the cost I care about most isn’t the wasted time. It’s trust. And on Amazon, trust is the entire game. A shopper is deciding in about two seconds whether your product is real, whether it’ll show up looking like the picture, whether they believe you. And AI fatigue is real.
We tag every poll we run for the reasons shoppers give us, and one of those tags fires whenever someone, completely unprompted, says an image looks fake or AI-made.
This year alone, that happened in 13 separate tests, across beauty, supplements, pet, home, and baby. Nobody asked these shoppers about AI.
They brought it up on their own. It isn’t new either, though the vocabulary is. Here’s a sample of what real people have typed into these tests, in their own words:
“B doesn’t even look like a real product. It looks like someone said ‘AI generate me a sheet set with a box.’” (Female, 35-44, bed sheets)
“All the other ones look AI. I would want legitimate items.” (Female, baby product)
“B looks really fake. All the other candles are actual pictures of the candle, and I trust that a lot more.” (Female, 45-54, candles)
“Please no more of the fake hands. They just make products look cheap and poorly made.” (Female, 25-34, nail care)
“Looks fake, like an infomercial. It makes me feel weird to look at her photo as part of a product meant for my sensitive skin areas.” (Female, 35-44, skincare)
One of those lines, the baby-product one, came from an A+ content test. The same kind of content I just told you we couldn’t get AI to do well on our own.
Look at the candle shopper for a second. She spelled out exactly why she landed where she did: the real photos earned her trust, so that’s where she went. That’s the point to pay attention to.
When a shopper types this kind of thing into a test, they’re telling you something simple and expensive: they don’t quite believe the product is real. On Amazon, that doubt is the gap between a click and a scroll. It’s the same gap that many brands are competing for.
Here’s where this gets expensive in a way most brands never see coming. Plenty of full-service Amazon agencies charge a $5,000 to $10,000 monthly retainer and promise they handle all of it: your ads, your logistics, your account, and your creative.
In my experience, creative is usually the weakest thing most agencies do. For years it was one freelance designer taking odd jobs with no real strategy behind it.
Now I’m watching those same agencies quietly hand that freelancer a one-click AI tool and keep charging the same premium. So you’re paying top dollar and getting AI slop on your listings, and the trust cost lands on you without anyone flagging it.
Whether that actually hurts you depends on what kind of brand you are. If your creative was already weak, or you’re selling a pure commodity, the AI version might even test a little better.
But if you’re building a real brand, with a catalog people are starting to know and trust, this is the exact blind spot that quietly bleeds you. The shopper who calls an image fake in a test is the same shopper deciding whether to trust your whole brand.
To be fair, one-click AI genuinely makes sense at the very start or even for prototyping and testing. If you’re brand new with almost no budget, testing whether a product even has legs, AI images are probably fine, and you can likely make them yourself.
It’s once you’re an established brand with a design system and years of data behind your decisions that good enough starts costing real money. At that point your team needs to open a working file, make a precise change, and move on, not negotiate with a tool that reinvents the layout every time they nudge it.
Quick thing you can do right now: pull up your main image and your most recent A+ section and look at them the way a stranger would in two seconds. Does anything read as too perfect, too plasticky, too generated? If it does, that’s a trust leak, and the good news is it’s testable.
IMPORTANT NOTE:
One more thing that recently caught me off-gaurd and turned all of this from an opinion into a risk decision for our clients. As of June 9, a new New York law requires a visible disclosure on any ad that uses an AI-generated human built to look like a real person.
Your Amazon listing reaches shoppers everywhere, New York included, so “that’s just one state” doesn’t get you out of it. And Amazon now asks, on every image you upload, whether it contains AI or AI-generated people.
The Risk: Nobody knows yet what they’ll do with that answer: a disclaimer stamped on your creative, reduced visibility for the listing, or nothing at all. Human-led design with a few AI-augmented pieces, rather than fully AI-generated people, keeps you on the safe side of all of it while the rules settle.
🔒 Now Let’s Get To The Good Stuff
Upgrading to paid is like getting the version of my brain that isn’t filtered for a general audience. Less than one Starbucks order per month, and you get the strategy I charge clients real money for — broken down so you can use it yourself.
Below the paywall:
The four specific ways we do use AI successfully every week (including the one I think is genuinely the future for large catalogs)
The exact workflow we use to turn a winning design into a fresh split test in 15 minutes instead of three hours
Bonus: Claude prompt you can run on your own listing to catch the AI tells before your shoppers do
Subscribe to unlock.
