New Framework: How to Evaluate AI Tools

Earlier this month, RSG released a major update to our Marketing AI vendor evaluations. There’s been a lot of movement in this space — new players, improved capabilities, collapsing price tiers — but also some constants.  We’ve adjusted our evaluations accordingly.

This update brings in a revised scenario framework, updated vendor ratings, and changes in how we categorize different AI tools. If you’re trying to make sense of how AI fits into your MarTech stack, you can download a free sample of our tough evaluations, and don’t hesitate to reach out to us for further guidance to accelerate your strategy.

A Shifting Vendor Landscape

Several vendors have evolved significantly in just the past few months — some improving their offerings, others simply shifting their positioning. Some key themes have emerged:

Foundation models adding services

OpenAI (ChatGPT), Google (Gemini), and Anthropic (Claude) have all added layers of enterprise controls, multimodal access, or productivity-focused interfaces like projects. But these tools remain very much generic assistants — not specialized marketing platforms.

Creative GenAI tools are getting more advanced

Vendors like Midjourney, Runway, and Firefly continue to improve their capabilities in visual and video generation. That said, these tools remain isolated — often disconnected from broader workflows and data layers – which reduces their business value.

Decisioning vendors hold steady

Platforms like Pegasystems, Adobe Sensei, and AWS Personalize remain focused on areas like Next Best Action and Recommendations. However, the learning curve (and implementation effort) remains relatively high.

Updated Landscape Categories

Finally, we’ve changed how RSG visualizes the AI vendor landscape, again leveraging the use case categories. MarTech leaders face an AI landscape divided into five categories:

  1. General-Purpose Foundation Models
  2. Insights Platforms
  3. Decisioning Engines
  4. Generative AI (narrative and creative)

Some tools sit at the edge of two or more categories — which is fine. But we’re also careful to flag vendors whose claims don’t quite match their actual capabilities.

 

RSG Marketing AI vendor evaluations
Market spans four categories.  Source: RSG Marketing AI vendor evaluations.
 

New Scenario Framework

At RSG we’ve responded to these changes by modifying how we evaluate AI tools — based on what marketers are actually trying to do. Our revised framework includes three main categories of marketing-oriented AI services, each with specific use cases.

Generative AI

  • Writing and editing
  • Research and analysis
  • Graphics
  • Audio
  • Video
  • Productivity

Insights AI

  • Audience and customer insights
  • Marketing intelligence
  • Creative analytics
  • CLV prediction
  • Attribution

Decisioning AI

  • Next Best Action
  • Ad targeting
  • Behavioral Recommendations
  • Dynamic Creative Optimisation (DCO)
  • Customer Journey Testing and Optimisation
  • Conversational AI

This makes it easier to map vendors to your real needs — instead of being swayed by vague “AI-powered” claims.

 

Where IBM’s watsonx excels and lags, by use-case category
Where IBM’s watsonx excels and lags, by use-case category.  Source: RSG Marketing AI vendor evaluations.

What Should You Do?

If you’re looking to evaluate or re-evaluate AI tooling in your MarTech stack, this update should give you a more grounded view of where different platforms actually stand — and which ones are likely to keep up with your needs over time.

If you’d like help navigating this space, or want access to detailed vendor profiles and scenario fit assessments, just get in touch. You can also download a sample evaluation to see how we evaluate vendors.

Other AI for Marketing posts