A recent study by CMO Huddles, Marketing Insider Group, and Renegade.com, put generative AI tools like ChatGPT to the test against human writers. The goal was to see how machine-generated content compared to human-authored and collaborative human/AI hybrid content.
The study compared 60 blog posts: 20 written by humans, 20 generated by AI (ChatGPT), and 20 AI-written but refined by humans. Articles were published over a 10-week period on Renegade.com. Performance was tracked across search visibility, rankings, and site traffic. Time spent writing and editing was also compared.
The results? Humans still come out on top for quality.
Here are the key takeaways from the research:
Human Content Drives More Traffic and Improves Rankings
The core finding is that human-written content simply performs better. Posts written exclusively by people generated 3x more website traffic and improved search rankings substantially more than AI-generated content.
This was evident in the study, where human-authored articles caused the site’s average Google ranking to jump from #96 to #84. AI-written posts actually caused rankings to decline slightly.
One standout AI article did briefly boost traffic and rankings. However, it was an outlier – overall human content clearly won on performance.
AI is Faster at Writing
What AI indisputably offers is speed. In the study, AI-generated posts took only 1/3 the time to produce compared to human authors. With the ability to instantly generate outlines and drafts, AI can significantly accelerate content creation. Even if the output needs refinement by humans, the time savings are substantial. For now, marketers should see AI as a way to write faster, but not necessarily better.
AI Alone Falls Short on Quality
The research found that AI-only content lacked consistency and contained factual errors. As a result, every single AI-generated article required human editing before it was suitable for publishing.
Additionally, all headlines written by AI had to be discarded and replaced by humans. This suggests that, at our current stage of AI advancement, generative tools still require oversight by human authors and editors to catch mistakes.
Human-AI Hybrid Content Shows Promise
AI-generated posts that were rewritten by humans to be on par with fully human-authored articles did demonstrate potential. While human-only content still performed best overall, collaborative human/AI hybrid content had bursts of strong performance. It generated more visibility than human-written articles during certain periods. This indicates that, with practice, content teams may get better at optimizing AI to enhance rather than replace human writing. More testing is needed.
The Takeaway for Marketers
For B2B content marketers, the implications are clear: AI can accelerate writing, but human creators are still indispensable. Brands eager to cut costs should proceed cautiously with over-reliance on AI, as quality issues persist. Instead, savvy marketers should focus on using generative tools collaboratively to aid human writers, not replace them.
With thoughtful integration of AI into their content processes, marketers can drive efficiency while maintaining the superior performance of human-crafted content. But pure AI-generated content alone cannot yet produce the results needed to justify eliminating human authors.
At least not yet. With rapid advancements in natural language AI, the trajectory seems clear. But for now, the message for B2B brands is simple – don’t hand over content creation to the machines just yet. When used strategically, AI can enhance human writing, but it takes skillful humans to develop B2B content that cuts through.