Social media measurement should track outcomes, not outputs. Professor Jim Macnamara learned this when his biggest client rejected a long activity report and asked for results instead. Impressions, reach and share of voice show what you put out. Awareness, enquiries and sales show whether anyone noticed.
Professor Jim Macnamara built a whole career on one painful meeting. He ran a communication agency with some very large clients, then spent years as a professor of communication evaluation, and he joined me and Will Ockenden on Embracing Marketing Mistakes to explain why so much marketing and PR still measures the wrong thing.
His story starts with a report, and I suspect many of us have written a version of it.
What did the client who slid the report back actually want?
He wanted results, not a list of activity. Jim’s biggest client asked for a detailed report on the value the agency had brought. Jim and his team listed every press conference, event and piece of publicity, and he took it back feeling “pretty chuffed”.
The CEO looked at it for about two minutes and slid it back across the desk. As Jim remembers the words: “Jim, I don’t care how much work you did. I just want to know the results.” The uncomfortable truth, he told me, was that he did not know how to calculate that.
That bruising meeting sent him into a master’s degree in research, then a PhD, then a professorship. I told him he had used the worst thing that ever happened to his work to craft his entire career, and he did not argue.
What is the difference between outputs and outcomes?
Outputs are what you put out, and outcomes are what changes in your audience. Jim says media volume, impressions, reach and share of voice are all outputs, because they describe your activity rather than anything in your audience’s heads. In his words, “they are not outcomes or impact.”
Outcomes are the questions that matter. Has your audience’s awareness changed? Has their attitude towards you changed? Have people made an enquiry, subscribed, registered, visited a showroom or bought something? Jim borrows a line from Alex Aiken, the former head of the UK Government Communication Service, who says “we’ve got to do more than sending out stuff”. Aiken calls it SOS, which is a rather good acronym for a problem this common.
Advertising has the same disease, Jim says. You can reach a market and they can ignore you, and you can recall thousands of adverts without buying any of the products.
Is visibility in AI search a result or just another output?
It is another output, and a useful one. I raised the tools that measure visibility in large language models by running a thousand prompts and handing you a share-of-voice percentage. To me that looks a lot like the old question of whether you are on page one of Google.
Jim agreed with the principle. Tracking outputs is legitimate, he said, because without visibility you will probably not succeed. But the industry must recognise that “getting your brand or your organization name into an LLM is only visibility.” The next step is what happens in the target audience: clicks, conversions, enquiries and behaviour.
He also separates three layers that people blur together. Measurement is taking the numbers, evaluation is judging them against your objectives, and insight is using them to shape future strategy. Most teams stop at the first layer, and as Jim puts it, “you can end up drowning in data”.
Can you prove that PR caused the sale?
You can get close by asking the buyers. Jim repeated an old saying: “success has many fathers and mothers and failure is an orphan”. When sales rise, everyone claims the credit, and advertising usually claims it loudest.
For Volkswagen in Asia Pacific, his team interviewed people who had recently bought a car and asked why. Advertising was the main trigger for awareness of the model. But price was the number one reason for buying, followed closely in third place by independent editorial reviews. That showed PR was a primary driver of sales, and he believes the marketing team raised the budget for it from 5 per cent to 15 per cent.
Jim is careful about the limits. Results are multi-causal, so pricing, distribution and brand reputation all play a part. His method was simple: “I’m a believer in the horse’s mouth.”
What did the World Health Organization do with all that data?
It changed the content, not just the report. Jim led evaluation for the World Health Organization through the pandemic, analysing website statistics, surveys, trust studies and more. The long technical videos were barely being watched, and research showed people did not understand the science behind vaccines.
So the team commissioned a short series called Science in 5, where scientists explained things in under five minutes. Jim says it reached 400 million views in a very short time. That is evaluation feeding directly into strategy, which is the only reason to measure anything in the first place.
The mistake: Reporting activity and visibility to a client or board that has asked about results.
Why it happens: Outputs are easy to count and quick to collect, whereas audience change needs research, baselines and a clear objective.
The fix: Set the outcome first, record a baseline, then track outputs only as steps towards it.
Why should you stop selling your work by the hour?
Because a flat rate hides the value of the work. Jim called charging by the hour “a bit of a 19th century approach” and says he prices on the value he can bring. He earns $10,000 for one day and $2,000 for another, depending on the results he expects to contribute.
His warning to agencies is blunt: “I can hire students who can get stuff in the media.” What clients should pay for is tracking progress through to awareness and behaviour change. I pointed out that plenty of proposals still ask for your day rate before they consider anything else, and he did not pretend that was easy to fix.
The practical answer is better training for the people doing the reporting. That is what our social media measurement training for your team is built around, because a dashboard only helps if somebody knows which numbers to trust.
My view on this: a page one ranking is not a result
I run an agency in the north of the UK, and I have always wanted us to measure actual business impact, not just numbers. We built our own framework around the Barcelona Principles for that reason.
When I talk to clients about their plans, I often end up asking why they are doing something, and then why again. Some clients do not know. They are doing it because a competitor is, or because their boss said so. That is where measurement falls apart, because you cannot measure against an objective nobody wrote down.
I also remain sceptical about visibility scores in AI tools. Being on page one never meant people were clicking or buying, and being mentioned in an AI answer does not either. Jim’s view that this is a step on the way, not the end result, is the one I will be repeating to clients.
He was also sensible about the hype. Asked whether AI is bringing a golden age for PR, he said, “I don’t think it’s going to be a permanent golden age.” I liked that, because honesty is always cheaper than a bad forecast.
Listen to the full episode
Watch the full episode with Professor Jim Macnamara on YouTube, find every episode at embracingmarketingmistakes.co.uk, and read more about his research on his University of Technology Sydney profile.
Embracing Marketing Mistakes is the weekly podcast I host with my Prohibition PR co-director Will Ockenden, where marketing and PR leaders share the career mistakes that taught them the most.
Chris Norton is the founder of Prohibition and an award winning communications consultant with more than twenty years’ experience. He was a lecturer at Leeds Beckett University and has had a varied PR career having worked both in-house and in a number of large consultancies. He is an Integrated PR and social media blogger and writes on a wide variety of blogs across a huge amount of topics from digital marketing, social media marketing right through to technology and crisis management.
