Wanna' Know How Your Klout Score is Really Calculated?
Klout is one of the more divisive social media metrics. For some, it's a great indicator of influence and presence, while for others, it's a vanity metric - and like all vanity metrics, it can be gamed, it can be cheated, and thus it's rendered largely useless. However you look at it, Klout is still an 'interesting' measure, and can be taken as another indicative note - it's not the measure, definitely, it's not the only thing that you should be using to judge a person or brand's social media presence, but it can be helpful when taken into account as part of the wider picture, and it can provide some level of comparative benchmarking.
But how is your Klout score actually calculated? Much like the algorithms of Facebook or Google, the actual metrics are largely kept quiet - we know, in general terms, how they work, but there's no concrete evidence linking certain actions to certain performance. But now, in an effort to improve the transparency of the service, and boost its relevance in the field, Klout's released a research paper which details how their scoring algorithm works, including what measures are taken into account, how they're calculated and how Klout's tested their theories in the real world in order to validate their findings. Here are the details.
In the paper, titled "Klout Score: Measuring Influence Across Multiple Social Networks", developers from Lithium Technologies (which owns Klout) and Klout itself outline the work they've undertaken to establish Klout's scoring system.
From the paper:
"In this work, we present the Klout Score, an influence scoring system that assigns scores to 750 million users across 9 different social networks on a daily basis. We propose a hierarchical framework for generating an influence score for each user, by incorporating information for the user from multiple networks and communities. Over 3600 features that capture signals of influential interactions are aggregated across multiple dimensions for each user. The features are scalably generated by processing over 45 billion interactions from social networks every day, as well as by incorporating factors that indicate real world influence."
That's a lot of factors to take into account - the paper then explores each of these elements in more detail, examining how they're measured and, ultimately, how they contribute to your final Klout score.
First off, the researchers lay down the key factors required, in their view, to establish an accurate influence scoring system. These elements include:
User Scalability: The ability to process information from the complete network graph, which may include hundreds of millions of users.
Network Scalability: The ability to measure influence across multiple social and professional networks, including the distinct sets of interactions and user behavior patterns associated with each network.
Interaction Graph: The ability to identify and capture variables that indicate dynamic information flows, including the manner of interaction (i.e. some interactions may indicate a greater degree of influence than others).
Temporal factors: The ability to measure factors that contribute to long-lasting influence versus dynamic and changing variables - established presence versus day-to-day interactions.
Offline factors: The ability to measure a user's real world influence - interestingly, Klout uses "Wikipedia and news articles to extract signals that may indicate the user's offline influence" in their measurement.
Reach and Strength of Influence: The ability to determine the importance of each user's reach with respect to the number of reactions, as well as the manner and frequency of reactions - "thus a user who induces a total of 100 reactions among 10 other users may or may not have the same influence score as a user who induces 100 reactions among 50 users"
These core measurements form the basis of the Klout scoring algorithm, but the means with which they've then established their system, and then proven it's applicability, are pretty fascinating.
The paper then examines the actual working of the Klout system, based on the above measures.
"Klout scores are computed for 750 million users from 9 major networks including Twitter (TW), Facebook (FB), LinkedIn (LI), Google+ (GP), Foursquare (FS), Instagram (IG), YouTube (YT) and Lithium Communities (LT), in addition to Wikipedia (WK). When a user registers on Klout.com he associates his identities on different social networks with his Klout profile. For Twitter, public data is collected via the Mention Stream1 ; anonymized data for opted-in Lithium Communities2 comes from in-house datastores; and data for other social networks is collected via REST APIs on the user's behalf, based on the granted permissions. All collected data is parsed and normalized to protocol buffers that encode user interactions, graph, and profile information. Data is collected continuously from interactions in a trailing window of 90 days using the Play Framework."
In other words, when you sign up to Klout, you grant the system permission to access your data via the available resources - for Twitter, this is the public data associated with your mention stream (interactions, including Tweets, re-tweets, favorites, etc.) while for others it's based on the information accessible via their APIs. This can be restrictive for some networks - LinkedIn, for example, doesn't make a heap of information available via their data resources - though the data's generally indicative enough to establish a baseline understanding.
To construct an accurate measure of your online influence, Klout uses both 'Long-Lasting' and 'Dynamic' data points in it's calculations. 'Long-lasting' features are those that change gradually or infrequently.
While dynamic features are those that happen day-to-day and change based on interactions within your social graph
Klout then measures each of those dynamic interactions based on five factors, giving each interaction a weighted measurement to determine their overall impact.
- Who: The characteristics of the audience who reacted to the original post from the user
- When: The difference between the current time and the time at which the reaction occurred
- Where: The social network on which the reaction was performed
- What: The unit of original content or action on which the reaction was performed
- How: The type of reaction
This is where the specifics of each interaction come into play - for example, the weighting assigned to a re-share will likely carry a higher weight than a 'Like', but a re-share action from a user who's more influential than the user himself will have a higher weight than the same action from one of the user's peers - if Barack Obama, for example, re-shared your post, that would indicate to the system that you're highly influential among the influential, and that interaction would carry significantly more weight as a result.
Based on these measures, each person's score is then normalized through Klout's hierarchical scoring architecture to finally establish your Klout score, based upon your influence.
Possibly the most interesting element of the paper, however, is the testing phase. In order to confirm that their scoring model works, Klout has conducted a range of real-world experiments to measure their system against actual, known influence.
Did you ever take part in any Klout perks? If so, you were part of the testing pool.
"To validate the effectiveness of the Klout Score, we ran a year long experiment to measure the spread of information with respect to the user scores. Users with Klout Scores varying in the range of 10 − 80 were targeted with perks, which could be claimed by the users. The users were then encouraged, but not mandated, to post messages about their experience with the claimed perk. The users' audiences then reacted to these messages, with a higher number of reactions indicating a greater spread of information."
87, 675 users posted messages after claiming a 'perk', of which 18, 308 posts received a total of 394, 083 reactions. The curve on the above graph, says Klout, shows that users with higher scores garnered a higher number of reactions, thus validating their methodology - those with higher Klout scores had greater levels of influence and were able to spread information more effectively through a network.
Klout also highlights comparisons against real-world measures - in this example, they match people's Klout scores against the ATP Tennis Player Rankings and the Forbes Most Powerful Women list.
This comparison seems less conclusive, given a player's tennis ranking is based on skill and not on his/her media persona, but you can understand the principle - those who rank higher in the ATP rankings are more likely to garner more media attention, thereby increasing their influence as a result.
A more interesting example is Klout's comparison of Klout scores versus Google Trends data - in this, comparison, you can see how fluctuations in Klout score correlate with Google search activity around those same entities.
"For both Starbucks and Airbnb, the Klout scores show similar fluctuations compared to their Google Trends, indicating a strong correlation between online influence and search popularity. For Fitbit the Klout score catches a few spikes that are not seen in Google Trends. This also reveals that the Klout score is sensitive to short-term variations, and tracks such changes very closely. For the music artist Psy, we see that the Google Trend drops significantly after 2013-06-01 while his Klout score decreases more gradually. This is because while the Google Trend reflects only the immediate short-term popularity, the Klout score incorporates long-lasting features as well."
And the final validation experiment looks at influencers by topic - using Klout's topic specification methodology, researchers looked at the top ranked users, by Klout score, in different arenas.
Klout suggests that these examples clearly show that Klout's methodology is correct, as it's able to correctly identify relevant influencers in a variety of domains.
A Measure of Relevance?
Whether you agree with it or not, whether you like it or not, Klout score is used by many as a measure to determine a level of social influence. And increasingly, that's becoming important - as social media matures and grows as a legitimate business platform, companies are looking beyond vanity stats and numbers and are seeking to determine real influence, the people who can generate real reach and response within social communities. This is especially true of influencer marketing - as the practice of utilizing influencers grows, so too will the means of faking reach numbers and making money based on those metrics. As such, the importance of systems like Klout will become more relevant - the more accurate and trusted they can become, the more utility they'll provide.
At present, there's a lot of skepticism about what Klout is and what your Klout score actually means. By making the process more transparent, through data and research like this, Klout may be on the path to establishing the importance of their metrics as a key part of social marketing strategies moving forward.
The full report: "Klout Score: Measuring Influence Across Multiple Social Networks" is available here.
Follow Andrew Hutchinson on Twitter