Social Media Dashboard: Dec. 2011 version and Analysis

A few months ago, towards the end of 2011, as I delivered a pitch on social media ROI/ROE at the iStrategy conference in Amsterdam, I asked my team at Orange to put a social media dashboard together, one that would be monthly, more appealing to managers than the weekly slapdash XL fires were produce for ourselves, and yet would be a useful tool for decision-making. Here is – below – its December 2011 version.

As I pointed out in my presentation in Amsterdam, this dashboard is in work-in-progress mode, and will always be. It is not meant to be used to show that we are stronger, better or unduly proud of ourselves but on the contrary, that we aim at doing well and getting better.

to Klout or not to Klout?

A lot of the metrics in this dashboard are here, for want of a better purpose, to underline that new metrics are needed. Sometimes, these metrics merely require fine tuning. Klout for instance, has been heavily criticised, including by myself. We aim to use Edelman’s Tweetlevel as a replacement very soon, because of the level of noise and fuzziness around the Klout measure, and because nonetheless we need some kind of measurement. Edelman provides this, without the hype.

engagement rates … what are we measuring here?

As to engagement rates, this is yet another kettle of fish. We realised, in the course of our comparisons, that we were being penalised by social bakers for the wrong reasons. For instance, our account was filed as a British account (vs. global) which was penalising us. Secondly, we use polls a lot and they are supposedly not considered as part of the engagement rates by the system. Yet, we have found repeatedly that polls are a very valid way of engaging with our community and we have no intention to suppress them. Last but not least, our account is multilingual and multinational. This is perfectly normal for a company which is operating globally, that is to say in 35 countries from the B2C perspective, and 220 countries and territories from the B2B point of view. Whereas many of our competitors are not that global and therefore enjoy a much better engagement rate which is not justified, we actually choose to speak Polish to the Poles, Spanish to the Spaniards and French to the French-speaking audiences of Africa and other countries as well as English to everybody else, and to us this is perfectly normal ; yet, we are being penalised for this by social bakers.

note: no hard feelings against social bakers I think this is a very good tool, and a great company. I’m a happy customer. I am only aiming at improving the system/measure.

We are trying to work with them in order to correct this measurement, but might also introduce our own calculation in which we will introduce a new way of measuring engagement rates in order to correct this bias.

What we’re trying to prove here is not that measuring return on engagement and social media isn’t worth doing, on the contrary, but that there is a lot in our plates in order to correct all these numbers and systems, before we can rely on these numbers and move forward.

here is our Orange december 2011 dashboard:

Klout Scores: Moveable Goalposts!

today’s selection is …

Today’s annoying announcement by Social Media influence analysis tool Klout. I suddenly discovered that my score had been significantly downgraded to 52 and the “specialist” category, all down from 65 or 67 and the “broadcaster” status. I then realised that all users had been downgraded too. To top it all, I also realised that all the history of my Klout score had been changed, in George Orwell fashion, like rewriting History. How do you want a serious Exec to believe in such a farce? Making things more accurate is one thing, rewriting historical data is another. I believe that one day – soon – I will stop visiting the Klout portal. “A more accurate score”: this must be a joke!

A More Accurate, Transparent Klout Score « Measuring Online Influence: The Official Klout Blog

A More Accurate, Transparent Klout Score

October 26th, 2011 by Ash Rust

Today we’re releasing a new scoring model with insights to help you understand changes in your influence. This project represents the biggest step forward in accuracy, transparency and our technology in Klout’s history. Joe shared the full vision behind these changes in his post last week.

Influence is the ability to drive action and is based on quality, not quantity. When someone engages with your content, we assess that action in the context of the person’s own activity. These principles form the basis of our PeopleRank algorithm which determines your Score based on:

how many people you influence,

how much you influence them and

how influential they are.

We analyze 2.7 billion pieces of content and connections daily. Reaching this scale, we’ve introduced significant upgrades to our platform, allowing us to handle this explosive growth. Now, we can add more networks and other sources of your influence much, much faster.

Insights help you understand why your Score changed. Each day, you can see which subscore and people in your network caused that change. You can also view insights on your friends’ profiles.

via A More Accurate, Transparent Klout Score « Measuring Online Influence: The Official Klout Blog.