There’s no denying it: LLMs (Large Language Models) are the hot topic in marketing right now.
Almost every week, a new model hits the scene, each with its own claims of being faster, smarter, or more capable than the last. If you’re feeling like you need a map to keep up, you’re not alone. But here’s the thing—it’s not about chasing the newest shiny tool. What really matters is how these models fit into your organization’s needs and how you can stay ahead in a landscape that’s evolving faster than ever.
Let’s break this down into three key areas.
- The Constant Evolution of LLMs
By now, you’ve likely heard of the big players—ChatGPT, Gemini, and Claude—but the truth is, there are dozens of LLMs being introduced all the time, each with different capabilities. The pace of innovation in this space is staggering. New models claim everything from better creative writing skills to faster data analysis, and it can feel impossible to know which one is the “best.” The truth is, there is no best model across the board.
The speed of these developments is both exciting and overwhelming. But as a marketing leader, you don’t need to keep up with every new release—what you need is a system to evaluate them as they come along. Understanding how a new model performs against the tasks that matter to your team is what will keep you ahead.
Omnicom evaluation of leading models from Q3 2024 against a collection of prompts that strategists use for research.
- There’s No One-Size-Fits-All for Marketing Needs
Like software, when it comes to LLMs, we tend to want to stick with the latest release from companies we trust. But different teams across your organization have different needs and will benefit from models that were trained in different ways.
For example, if your strategy and insights teams are spending hours manually combing through data to find key trends, you’ll want an LLM that excels in insight extraction and analysis. On the other hand, your creative team might benefit more from an LLM that’s tuned for generating copy variations and understands the nuances of both the industry vertical and the channel—tasks that require a more nuanced understanding of language and tone.
This means it’s unlikely there’s a one-size-fits-all LLM—you’ll likely need multiple models that align with your specific use cases.
Omnicom evaluation of leading models from Q3 2024 against a collection of prompts that strategists use for creative writing.
- Focusing on Core Needs and an Evaluation Framework
With so many LLMs hitting the market, the real challenge isn’t just keeping up—it’s figuring out which model will truly serve your marketing team’s specific needs. This is where having a clear evaluation framework becomes critical to align the AI with your business objectives.
At Omnicom, we looked at our top four tasks (insight extraction, creative writing, research, and data analysis) and used the R.A.C.C.C.A. framework (Relevance, Accuracy, Coherence, Completeness, Creativity, Appropriateness) to evaluate the responses across a variety of models using the same input prompt. The evaluation and scoring were then performed by the people in those roles.
By anchoring your decisions to core marketing needs and evaluating each model systematically, you’re not just picking the right tool for today—you’re setting your team up for long-term success.
Wrapping It Up
Here’s the bottom line: the LLM world is moving fast, and you will fall behind if you’re not keeping up. This means you need new strategies for working smarter, not harder, to keep pace. Focus on understanding your unique needs, select the right tools for the right tasks, and ensure you’re staying adaptable as the technology evolves. By doing this, you’ll not only keep up—you’ll stay ahead.
So, next time a new model comes across your radar, don’t ask, “Is this the best?” Ask, “Is this the best for us?”
Authored by
Tony Bailey
Head of Technology Innovation, Omnicom DAS
