Google Is Freaking Out About ChatGPT (The Verge, Jan 2023)
The ChatGPT AI Hype Cycle Is Peaking, but Even Tech Skeptics Don’t Expect a Bust (CNBC, Feb 2023)
The Inside Story of How ChatGPT Was Built From the People Who Made It (MIT Review, March 2023)
All these are headlines reflecting upon the fact that ChatGPT has had the best Public Beta of all time.
Every nook and corner of search engines and social media has been flooded with narratives – some hyping it even more, others talking about the possible improvements, and a few informing how it has taken human jobs already.
But while a lot of ChatGPT text-generation use cases and applications have surfaced over the past few months (thanks to excellent prompt engineering), there are big question marks around AI’s viability to create long-form content – especially for a B2B audience.
Is AI Any Good for Long-form B2B Tech Content? (Our Research Attempted to Answer This)
The argument for writing articles and blogs using AI goes back to Q1 2021 when GPT-3 AI tools were being rolled out one after another.
In October 2022 (before the launch of ChatGPT), we started testing the top GPT-3 tools to understand how well they do for B2B articles. We tested them across major technology themes (like cloud, data analytics, cybersecurity, etc.), and here’s what we observed:
- AI writers contributed (in terms of text used for the final article) less than 50% for more than 55% of the articles.
- Less than 30% of the text was contributed by AI for 20% of the articles that were highly technical.
- Surprisingly, listicles weren’t easy to create with AI – with 600 words articles taking more than 2 hours of continuous work on average. In fact, How-to guides and journalistic articles took more than 3 hours.
Here’s the full research report.
Of course, these results can be a bit biased in a sense that:
- We focused on creating high-quality consumer-facing content based on our decade-long experience writing for the B2B audience. And so, we scrutinized and edited the AI outputs significantly before considering them.
- We tested both technical and highly technical articles, which weren’t presumably in the purview of AI with respect to factual and conceptual accuracy.
But that’s the thing. When we’re talking about consumer-facing content, there’s no room or excuse for compromising the quality. And so, the question pops up – Why exactly are we using AI writers?
Is it because human writers aren’t adept? (Surely, that isn’t the case)
Is it to increase the frequency of publishing content?
Is it to lower the costs of content creation?
Is it to increase productivity?
The reasons could be manifold. But they often get obscure when riding the hype train. So, almost every answer will include the umbrella terms “productivity” and “cost.” And that’s understandable since these are integral to a business’s bottom line.
But then, what exactly does productivity entail here? How is it being defined? And how are the cost advantages being realized?
One explanation could be amping up a writer’s workflow – equipping them with an AI tool that reduces their cognitive load. So, with relatively lesser cognitive input, they would be able to create a similar-sounding output.
In fact, AI writers accrue several benefits for inspiring ideas, removing writer’s block, and helping with lo-fi content outlines. We’ve compiled the good things here.
But a lot of the presumable capabilities fall flat when it comes to creating long-form B2B tech content. Why’s that the case? Let’s discuss.
1. Original Content
Before getting into the granularities of B2B-esque content creation, let’s address the elephant in the room, shall we?
Although duplicate content (or plagiarism as it’s called in academic circles) isn’t always penalized by Google, it’s still an aspect of content creation that must be avoided. At the end of the day, search engines (especially Google) want original, helpful content to come up.
However, often, AI is very standardized in pushing out answers to your queries. Here’s an example. I asked ChatGPT – what is low-code development? But I did that three times – at different times. The answers, albeit quite relevant, overlap both structurally and linguistically.
All these explanations start with a definition, transition into the goals, approach the benefits, and outline the applications. From the looks of it, that’s an ideal, all-encompassing explanation – but only for the purpose of learning and not creating original content.
Notice the repeated keyphrases across definitions and the astonishingly similar sentence structure accommodating them. Even if you change the prompt to, let’s say – “explain” or “define” low-code development, the language used is similar.
These are just three queries from a single account. Imagine how many people will be asking the same questions. So, even if you structurally refine the content, you’re still leaving much to be desired when it comes to originality.
A possible and reasonable refutation to the same can be put forth in the name of prompt engineering. What if you give highly-specific, example-laden prompts and probe the AI until it gives you a quality output? Well, that’s surely opportune. However, that will also involve structural overlap in how the prompts are created. Why? Because the entire field is based on formulaic frameworks – much like you keep seeing on LinkedIn these days. So, it’s highly likely that the most relevant prompts are exceptionally similar.
2. Service/Capability-Oriented Writing
A good B2B content strategy supports a mix of informational and promotional content. The latter is concerned with reflecting upon a business’s capabilities to potential leads. This can be realized via:
- Showing how the products or services can benefit the potential businesses. Case studies, for instance, help build awareness around how you can help and encourage businesses to move forward.
- Showing how your product or service fits into a business’s current strategy. Most businesses, especially smaller ones, aren’t often looking to overhaul their entire strategies — they’re looking for incremental improvement and performance by integrating your product or service into what they’ve already put in place.
- Elaborating on the differences between your product/service and the competitors’ — to help businesses identify the value of your offering over theirs. Comparison landing pages, for example, are a great way to show how your product or service stacks up against the competition in terms of the quality of your product, the value of your service, and how the business would benefit from a switch.
- Putting out thought leadership content that drives business owners to think about their current practices, the industry as a whole, and your offering in relation to both.
Besides, a host of B2B businesses help implement products from global vendors. For example, IBM, ServiceNow, Salesforce, Microsoft, Amazon, SAP, and Google have product and service implementation partners (the B2B businesses) helping spread the word about their products and lowering the entry barrier to their adoption. Such businesses center their content around the capabilities of these vendors, and understandably so.
Why isn’t AI a good fit here?
For one, GPT-3 powered tools cannot scrape data from the web as per a business’s specific preferences. In general, they can control the subject-related output from a technical standpoint using certain parameters like temperature, top p, frequency penalty, presence penalty, and more. These parameters work to increase the creativity, originality, and recency of the outputs. As such, a lot of AI writing tools allow users to tweak the creativity levels of the text.
However, stringently controlling the output to talk accurately in terms of business offerings is a whole different ball game and certainly something that hasn’t been accomplished – at least until now. It’s even more challenging when we consider unique offerings from small businesses – a capability that the B2B SaaS landscape thrives on.
And this is also where the problem of emulating a brand’s voice comes to the fore. But can’t you have an editor refine that? Certainly! Humans working through AI outputs would only do good to the final outcome.
However, we’re talking about substantial edits in the case of promotional writing. And we’re giving more than the required control to the robot since the rough draft would be created by it. So, eventually, the output would still be inferior. This brings us to the question asked earlier – Why exactly are we using AI writers? Because this workflow certainly doesn’t sound productive.
A refutation to the brand’s voice facet could be AI tools that can help maintain consistency in the tone and voice – much like Grammarly Business which features a Tone Detector and Company Style Guide. With the help of these features, businesses can fixate their terminology and tone and align the content according to it. But then again, this needs substantial due diligence on the part of humans to set everything up. And the writing is manual.
3. Time-Bound Content
“Recency” is the hallmark of the B2B SaaS landscape. And why go far to understand that? The topic we’re addressing in this article serves as a great example.
Back in May 2022, Rob Toews wrote on Forbes that “a wave of billion-dollar language AI startups is coming.” Safe to say, it’s already here.
VC investments in Generative AI increased by 425% between 2020 and 2022. In 2023, this is the number one funding area. Out of 183 startups in Y Combinator’s first 2023 batch, 51 are AI startups, and 32 are explicitly into Generative AI.
As a result, AI and specifically GPT-3 is now powering apps for multiple use cases. These include, but aren’t limited to:
- Workflow productivity
- Content and copywriting
- Image and video generation
- Text to speech
- Health monitoring
- Virtual assistance
Such quick evolution of the space demands that the content that’s being pushed out:
- Concretely outlines the “evolving” consumer pain points
- Presents the product as a solution to those problems
- Differentiates the brand from competitors
- Evolves as the product landscape evolves
So, a good portion of such content is time-bound — i.e., it draws upon the recent assessment of the market, the experiences of CxOs in navigating the complexities of such market, the evaluation of the product itself, and the ongoing developments in the space. This differs from the B2C space, where products have a set description. This description remains independent from the next version of the product. For example, you wouldn’t have to tweak iPhone 14’s product description when iPhone 15 comes out.
In the B2B world, however, even the products or services that don’t serve a market as aggressive as AI entail a more rigorous process of iteration and refinement. As a result, the content that’s being put out needs to be time-bound.
Does that mean that every write-up should completely build on recency? Definitely Not! For example, HubSpot’s compilation of templates for sales, resumes, copywriting frameworks, etc., would work well for a long time.
However, it’s highly likely that a good portion of your content strategy is centered on weaving in recency to accommodate “relevance.” Even the most basic of articles – like XYZ trends about a [technology] in the [industry] space would warrant a B2B-centric focus on recency.
Why isn’t AI a good fit here?
One explanation is the dated data set that the AI is trained on. For example, ChatGPT inspires answers from its knowledge base, which is limited to 2021.
Even if the data corpus is updated in real-time, factual and conceptual discrepancies are likely to pop up. That might be due to the limited amount of data these tools have at their disposal regarding recent developments. Further, it’s just not possible to source statistics or facts. There’s a very high probability that the stats laid out by AI are fabricated and fictional.
Cuing back to our four-month-long research; throughout the process of testing GPT-3 AI writing tools, we observed that the articles involving the time element were almost impossible to create without successive interventions. These interventions were associated with:
- Tweaking the direction that AI was taking
- Manually including research and statistics to provide context
- Ensuring that the text generated was factually on-point.
These articles ranged from exploring technological developments to how-to guides for explaining the best practices going forward.
On average, we could only let the AI write the next 9-10 words before we intervened.
“Computers have never been instruments of reason that can solve matters of human concern; they’re just apparatuses that structure human experience through a very particular, extremely powerful method of symbol manipulation.” — Ian Bogost, The Atlantic.
Structuring “human experience through a very particular, extremely powerful method of symbol manipulation” — this is perhaps the aptest and most concise way to sum up the personality argument.
Going back to the example shared earlier about low-code development, did you notice how standardized the response was? Now, such standardization still works well for definitions. But what if it encompasses every nook and corner of your narrative?
For one, it wouldn’t engage people; it wouldn’t draw them in. Second, it wouldn’t support the brand voice a business wants to realize. Most importantly, it wouldn’t do justice to the solution under consideration.
For example, I asked ChatGPT to create four introductions for a topic – each following a particular tone descriptor (Professional, Upbeat, Conversational, and Authoritative). While there are subtle distinctions in the sentence structure, they all read the same from an emotional standpoint. Conversational sounds a bit different, but it’s more casual than it’s conversational.
It’s still possible to sell a bad product through good marketing. But when the marketing’s bad, it doesn’t matter how good the product or service is.
In the B2B space, personality is even more critical. Your audience is most likely educated on the service/product they’re seeking. If not as well informed, they’re at least quite aware of what it takes to deliver on the promise and understand the level of expertise they’re seeking to navigate complex problems. In that light, the personality of your narrative and self-expression matters a lot. And this is reflected in the long-form content that builds the platform for a product or service to succeed.
The Bottom Line
While aesthetically pleasing, it’s one thing to integrate AI into your workflow and completely another to be persuaded by the guise of its grammatically sound articulation. No doubt, the lexical aptness gives AI writers remarkable fluency in their arguments. However, their storytelling capabilities are not to be mistaken as excellent.
Use them to increase the quality of your narrative, not to define it.
More to come in this series around AI content detection and whether AI writers are good for listicles and how-to guides.
Besides the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.