🎨 THE CREATIVE TESTING FRAMEWORK
What to Test
✏️ Copy & CTAs
Headlines, body text, button text. Small words, big impact.
“Advertise” → “Create an Ad” = 40% lift🖼️ Images & Video
Product shots vs lifestyle, static vs motion, with or without people.
👤 Personalization
Name in subject line, contextual logos, behavioral targeting.
📐 Ad Format
Single image, carousel, video, stories format. Each reaches differently.
📍 Placement & Prominence
Feed vs sidebar, above vs below fold, mobile vs desktop.
Link bottom → top = 30% lift💬 Social Proof
Friend endorsements, ratings, testimonials, user counts.
🔁 Persistence & Frequency
How many times to show before sequencing to new creative (impression discounting).
🔗 Landing Page
What happens after the click matters as much as the click itself.
How to Test
- Isolate each test — Multivariate tests are fine, but don’t cross-contaminate experiments. Keep each test cleanly separated so you know what caused the difference.
- Pre-register test length and sustain significance — Decide the test duration upfront. One day of statistical significance is not enough — you need sustained significance over the pre-registered period. Don’t peek and kill early.
- Business significance, not just statistical — A statistically significant 0.01% lift doesn’t matter if nobody notices.
- Run long enough — Account for day-of-week effects, time zones, seasonal patterns.
- Document everything — Build institutional knowledge. Today’s loser insight is tomorrow’s winning hypothesis.
Reading Results
✅ This Result Matters
- Double-digit percentage change
- Visible impact on business metrics
- Finance would care if it went away
- Replicable across regions/audiences
🚫 This Result Doesn’t Matter
- Needs a microscope to detect
- Statistically significant but tiny
- Only affects a niche segment
- Can’t be explained simply
Impression Discounting
The Results > Awards Rule
🏆 Results Over Awards
“My number one sign to look for in a creative agency is that they focus on their results over their awards. Most effective digital direct response campaigns don’t even submit for industry awards. Marketing awards don’t tend to correlate closely with the most effective work.”