Posts Tagged measuring

Do we have a better way of measuring engagement online?

There are loads of tools that help you measure social media/community engagement online right now. Clearly this is something that people are becoming more and more concerned about; who is looking at, reading about, writing about our product? Who is sharing our news stories? Why?

I just can’t help but notice that so much of it is about numbers.

Klout scores

Klout assesses your engagement (supposedly) and gives you a number, ‘ranking’ you in how you influence people in your circle, ie how much you can get them to engage with things you write about or point out online. I’ll be honest; I just don’t understand the reasoning behind of pinning a number to your ‘value’ as an engaging user. What does the number 46 tell me about you? Bugger-all. Are Klout scores merely another vanity tool, a way to tell you you’re great… or a way to tell other people that you’re great?*


I use Hootsuite a lot (though I am going to move to for various reasons) to schedule, since Tweetdeck was bought by Twitter and fell off a very high cliff. I like Hootsuite – I’m a free user, and I get detailed analytics about which articles do well, where people click from, etc – but again my issue is that these are all numerical values. They don’t tell me much about the intention behind the action.

Did someone click my link by accident? Did they click it because they saw someone else retweet it? How many degrees is this person from me? Are they glad they clicked it or are they annoyed because it wasn’t what they expected or wanted? The answer to those questions would perhaps be more interesting. It would be useful to know if people click directly from my page or from other people’s retweets, because if it’s the case that people click through retweets, then that makes a retweet – through people you already know – arguably more valuable than the amount of followers you have. Similarly, it’d be much easier to ‘do social media right’ if I knew how successful I was at pointing people to what they wanted to read. What’s the point in having followers who don’t click links or engage with you?

Why use quantitative data to measure actions?

It’s easy. I get it. But really, what does having 50,000 followers tell me about someone? That they’ve been on the site for long enough, that they are a nice person, that they are useful? Perhaps. But that doesn’t mean that someone with 500 followers isn’t also all of those things.

The key thing is: engagement has intention and meaning beyond numbers. I wonder how many people click on Daily Mail links because they don’t like it and they want to hate-read. How many people tweet along with the #XFactor hashtag but love to hate it? Numbers just don’t cut it in these instances.

What about Facebook? Facebook Page Insights gave us stats about users ‘talking about’ a page, users ‘engaging with’ a page (both in terms of who has seen it, and who has liked, commented, shared). I have no idea if people are talking about my page in a positive way. In the case that they are talking negatively, there’s not a lot I can do to resolve that person’s negative perception of my page. (In fact, I have yet to find a way of actually seeing where people are talking about my page, but that’s a different issue.)

My answer to all of these questions would be that analysing engagement in a way that takes into account intention or meaning would be brilliantly useful for social media/communities editors.

Of course, this analysis is already done – by people who are employed by brands or PR companies, to monitor social media for mentions of the brand. Is there a brand with so much action on social media that they cannot cope? I don’t know. But it would be a lot easier to automate it.

I’m not arguing that current analytical tools are not useful or should not be used. Not by a long shot. Some of them are really sophisticated (and I’m aware as a free/non-corporate user, I don’t have full access to them) and great for measuring your success on social media. I would just be wary of drawing too many absolute conclusions about what those figures really mean when we have so little information about what’s behind them.

Automating qualitative analysis of engagement

People are currently trying to fix this very problem with comments on articles. In fact, recently, there was a Hack Day on ‘re-imagining comments’ which a team from The Times won. The ideas to come out of that are definitely interesting. These take into account either an extra layer of ‘moderation’ whereby communities or staff label people as useful/experts, or an analysis of the words used in the comment.

So when you comment “this is great, I agree”, it’s flagged up as a positive comment; “terrible article” is clearly a negative one. It’s definitely a move closer to the kind of thing I’m talking about, but there are some obvious issues especially when it comes to exact words that could mean something very different out of context. Instead of searching for words or phrases in isolation, the whole sentence/paragraph needs to be contextually analysed. Now it gets difficult… right?

I like coding now and again, but I’ve never touched social media APIs and I wouldn’t have a clue about where to start. Maybe the next Hack Day could be about re-imagining analytical tools?

If I’ve missed an important analytical tool which does take into account semantics, feel free to tweet or leave a comment. I’m very interested in trying it out!

*If you’ve found a use for Klout, again…tweet me, because I cannot for the life of me understand how it’s useful


, , , , ,

1 Comment