The new #1 writing model comes with an asterisk
by Lisa Peyton, full-time faculty at the University of Oregon School of Journalism and Communication, AI marketing practitioner, and Forbes contributor
My morning started like any other: quick meditation, up and out to walk the pup, grab my standard breakfast, and hop online to check my daily AI news and models-for-marketers feeds in Slack. After a few weeks of Fable 5 and ChatGPT Sol 5.6 dominating the model leaderboards, I was SHOCKED to see a new contender at the top of the creative writing leaderboard: Kimi K3! Shock quickly led to frustration as I now needed to spend the hours required testing yet another model. With a fair dash of skepticism, I began setting the stage for my three-way bake-off between Fable 5, ChatGPT Sol, and Kimi K3. The result is the blog post you see before you.
The BIG takeaway for me is that the model at the top of the writing leaderboard may be the ‘best’ at a specific writing task; however, that is only half of the story. Maybe even more important than its skill at writing is its ability to take actions like search the web, fetch and analyze web data, connect to other platforms, and remember projects it has written for me in the past. It’s like if you just hired an award-winning writer but didn’t train them on your industry, brand, and desired output format. Their output would be unusable despite their creative writing prowess. This is the new frontier of model selection and it’s not making our jobs any easier. Faster than we can say ‘prompt engineering,’ the skills we have honed over decades are being replaced by skills historically aligned with systems engineers and developers.
Three leaderboards, three different winners
Before we get to the bake-off, let’s look at what the leaderboards say, because they don’t agree with each other. That disagreement is the whole story.
On EQ-Bench’s Creative Writing v3 leaderboard, Kimi K3 sits at #1 with an Elo of 2377, a full 286 points ahead of Claude Fable 5 (2091) and well clear of GPT-5.6 Sol (2143). That’s a decisive win, and it’s the ranking that stopped my scroll that morning. One caveat worth knowing before you take it as gospel: EQ-Bench’s Elo scores are judged by a Claude model (Sonnet 4.6), and the benchmark’s own documentation openly says it doesn’t control for judge self-preference.
Flip to EQ-Bench’s Longform Writing leaderboard and the podium reshuffles. Fable 5 leads at 83.0, Claude Opus 4.7 and GPT-5.6 Sol sit right behind, and Kimi K3 drops to sixth at 79.6. Multi-chapter consistency is a different sport than short-form punch.
Now check Arena’s creative writing leaderboard, which ranks models by blind human votes instead of an AI judge. Fable 5 is #1 with a score of 1512. Kimi K3? Sixth, at 1479, on a preliminary vote count with a wide confidence interval.
You’ll also see aggregators like evy.so crowning K3 the best writing model. Read the fine print: that ranking sorts primarily by the EQ-Bench score, so it’s the same data wearing a different hat, not independent confirmation.
Three credible sources. Three different answers. And we haven’t even asked what kind of writing you do, what it costs, or whether you can get your hands on the model. Which brings me to the asterisks.
The first asterisk: who else is reading your prompts?
Kimi K3 comes from Moonshot AI, a Beijing-based lab founded in 2023, with Alibaba disclosed as holding roughly a 36% stake. The model is impressive and the price is aggressive ($3 per million input tokens and $15 output, against Fable 5’s $10 and $50). But before a single word of your brand strategy goes into that chat box, you need to know what you’re agreeing to.
I hit this question personally before I typed my first prompt. Signing up to try Kimi required linking my Google account, and I’ll be honest, I hesitated. I did not love handing account access to the damned thing before I’d even tested it. That gut check turned out to be the right instinct to interrogate, because the data-governance picture behind it is bigger than one sign-up screen.
The U.S. National Counterintelligence and Security Center has published a plain-language bulletin on the PRC legal framework that applies to China-domiciled companies. The short version: China’s 2017 National Intelligence Law creates affirmative obligations for PRC entities to cooperate with state intelligence services, the 2021 Data Security Law gives Beijing broad authority over data transfers, and the PIPL authorizes government collection of personal data for purposes the state deems in the public interest. Stack those together and here’s what it means for us as marketers: drafts, customer data, and proprietary strategy submitted to a China-hosted API are potentially subject to PRC government access, whatever the provider’s own privacy policy promises.
Washington has noticed. A House investigation is probing U.S. corporate adoption of Chinese AI models, some government agencies have already banned models like DeepSeek internally, and a State Department spokesperson called the growing corporate use of Chinese models a serious concern.
Then there’s the provenance fight. In February, Anthropic publicly accused three Chinese labs, Moonshot among them, of industrial-scale “distillation attacks”: more than 24,000 fraudulent accounts generating 16 million exchanges with Claude to extract its capabilities, with 3.4 million of those exchanges attributed to Moonshot. Reuters and CNN independently confirmed the accusation came directly from Anthropic. On July 22, the dispute jumped to government level when White House OSTP Director Michael Kratsios stated the U.S. has evidence Moonshot distilled Fable specifically. China’s embassy called the claims “entirely groundless.”
Let me be crystal clear about what’s fact and what isn’t. It is a fact that these accusations were made, at company and now government level. It is NOT an adjudicated fact that K3 was built on distilled Claude outputs. No independent audit or legal finding has been published, and Moonshot itself hasn’t issued an on-record rebuttal that I could find. You should treat it as an open dispute, and you should also know it exists before you bet your content operation on the model at the center of it.
One more wrinkle: Moonshot has promised to release K3’s full weights by July 27 under a “Modified MIT license.” Self-hosting those weights (or using a U.S.-hosted provider) would solve the data-transit problem, since your prompts would never touch a Beijing server. It would not resolve the provenance dispute, and the exact license terms won’t be knowable until the weights actually drop.
A note for the API-savvy: routing K3 through a U.S. aggregator like OpenRouter fixes less than you’d hope right now. It does keep your account and identity out of Moonshot’s hands (I’ve since deleted my Kimi account entirely). But OpenRouter currently lists exactly one provider for K3, and it’s Moonshot, so your prompts still land on their servers. That math changes once the weights are public and U.S. hosts can serve the model. Until then, a middleman is not a moat.
The second asterisk: good luck getting in the door
Here’s the part of the story no leaderboard will tell you. Testing Fable 5 and ChatGPT Sol took me zero setup, because I already live in both. Testing Kimi K3 took me an afternoon and twenty bucks.
Moonshot’s consumer app was at capacity and pushed me to upgrade. The upgrade? Waitlist only. To get my hands on the model at all, I had to go through a third-party developer router, pay $20, and run K3 in a sandbox. I toggled on web search. It didn’t work. The model that topped the creative writing leaderboard could not fetch a single one of the articles my newsletter test required.
For those of us building real content operations, this is not a footnote. The product layer around a model IS part of the model. A brilliant writer you can’t reach, can’t connect to your tools, and can’t trust with your accounts is a brilliant writer you can’t hire.
Bake-off round one: one newsletter, three very different results
My test was my own weekly newsletter production prompt, the same one I run every week: fetch and verify eight source URLs, draft the full issue to my template, preserve my verbatim sections, and flag anything that needs my review. I ran it through all three platforms. Full disclosure on methodology: because K3’s sandbox couldn’t fetch, its run tests raw writing from the prompt alone, not the full workflow. Not apples to apples, and I’m telling you that upfront.
The drafts came back with a built-in accuracy test I didn’t plan: the EU AI Act fine number. My newsletter covered the Article 50 transparency deadline, and the source articles contained two different penalty figures. The correct one for Article 50 violations is €15 million or 3% of global turnover, straight from Article 99(4) of the Act. The €35 million / 7% tier applies to prohibited practices, a different and more severe category.
Claude Fable 5 ran the entire workflow: fetched and verified every URL, picked the correct €15M figure, and, this is the part that impressed me, explained in a production note WHY it rejected the bigger number that was sitting right there in a source article. It flagged open editorial decisions instead of making them silently. The one draft that was ready to ship.
ChatGPT Sol produced the most polished-looking draft and the single best headline instinct of the day. It also grabbed the €35M / 7% figure for my stats section. Its framing was technically honest (it labeled the number as the maximum for the most serious violations), but it’s the wrong tier for an Article 50 story, and it made that call without flagging it. When it couldn’t access one of my articles, it correctly flagged the gap instead of inventing a description. Credit where due.
Kimi K3, writing blind with no tools, opened its draft with four upfront flags declaring exactly what it couldn’t verify, refused to fabricate titles for links it couldn’t open, and pulled the correct €15M / 3% figure from its own knowledge while still bracketing it for verification. The prose had genuine snap. My favorite line of the whole test came from K3: it described one source as “the thing you forward to the teammate who still thinks this is a 2027 problem.” But the draft had holes only I could fill and claims only I could verify. Strong writer. Unusable issue.
Round one verdict: the best prose and the best production system were not the same model. Fable won the workflow. K3’s writing punched above its handicap. Sol shipped the prettiest draft with the least reliable number in it.
Bake-off round two: the headline test
Round one wasn’t a fair fight for K3, so I designed one that was: a fully self-contained headline prompt, no tools required, identical across all three platforms. Ten headline options for this very article, mixed styles, tight constraints, and each model had to pick its own winner and defend it.
Fable’s set followed every rule but circled the same thesis too many times. Sol delivered nine options instead of ten and then misnumbered its own pick (it said #5 and quoted #4), though its best line, “Kimi K3 won the benchmark. Claude won the writing test,” had real tension in it. K3 turned in the cleanest execution of the three: ten distinct options, labeled styles, self-audited character counts, and four genuinely different angles on the piece.
And the winner is at the top of this page. The headline on this post, “The new #1 writing model comes with an asterisk,” was written by Kimi K3. The model that lost the workflow test won the writing test, fair and square, and I think that’s the perfect summary of this entire experiment.
While I’m disclosing: this post itself was drafted by Claude Fable 5, one of the three models under review, working from my research direction, my test results, and my editing. Every source in this piece was verified against the live page before it went in, every opinion is mine, and yes, I clocked the irony of the contest winner’s rival writing up the results. That’s exactly why the benchmarks, the receipts, and the final judgment stayed in my hands.
Routing is the skill. Here’s how I’d route.
So which model should you use? After this experiment, I’m convinced that’s the wrong shape for the question. The right question is which model gets which job. Here’s where I landed:
- Short-form punch, headlines, and high-volume variants: Kimi K3’s writing chops are real, and at a third of Claude’s API price, it earns a look for non-sensitive, high-volume work. IF the access and governance questions clear for your organization. For my money, that’s a big if until the weights drop and the provenance dispute shakes out.
- Production workflows, research, and anything requiring verification: the work engine matters more than the wordsmith. Fable’s ability to fetch, verify, flag, and remember my project context is what turned a prompt into a shippable issue.
- Anything touching customer data or proprietary strategy: route around China-hosted inference entirely, aggregator or not, until your legal and security teams have read the NCSC bulletin and made a real call. Your gut check at the sign-up screen is data. Listen to it.
We’ve spent two years mastering prompts. The next skill is routing: knowing your models’ strengths, their access realities, and their governance risks, and assigning work accordingly. You already run this play every time you match a freelancer to a brief. Same muscle, new roster.
Want to see how your own brand shows up across these models? That’s exactly what we’ll test together in my AI Visibility for Marketing and Comms workshop on August 5th. And if you want experiments like this one in your inbox every week, subscribe to my Weekly AI Marketing Brief. Tried Kimi K3 yourself? Hit me up on LinkedIn and tell me what you found. I read every message, and your tests make these bake-offs better.
Made with my team of AI superheroes and my own skills, including AI-generated imagery. Every opinion, edit, and fact-check is mine. I am an AI practitioner, professor, and pioneer helping marketers put AI to work with purpose. Find more resources at lisapeyton.com/ai-marketing-resources or connect with me on LinkedIn.
