The Truth About Algorithms (And Why You Should Expect to See More of Them)
Twitter's got one, Facebook's had one in place for years, and now Instagram's the latest social media network to take the leap. I'm referring, of course, to algorithms, those machine learning systems designed and developed to 'learn' what users are interested in based on their on-platform activities, then serve them more relevant content as a result. And of course, users aren't happy about it - no one likes being told what they really want, what they're going to like, and people really hate being told that a machine's going to determine it for them. Machines don't know what you want - how could they?
It's human nature to resist any change that takes control out of your hands, a survival instinct, of sorts. We're in control of our lives, and especially our recreational activities and how we spend our free time. Who are these networks to tell us what we want?
But the truth is that algorithms know you better than you think. More than that, the best algorithms may actually know your likes and interests better than you do.
Not possible? It's actually statistically proven - there's good reason why more social networks are hopping aboard the algorithm train.
Learning Your Likes
While Facebook was the first social network to implement an algorithm, the roots of computer learning systems actually go far deeper than that. There's Google, of course - Google's been tweaking their search algorithm since 2002, routinely upgrading and evolving their back-end processes to stay one step ahead of SEO scammers and continue delivering the best, most relevant search results. Amazon too has been using personalized algorithms since the late 90s with a system that recommends products for you based on your search and purchasing history.
That algorithm, as simple as it is, has lead to significant increases in click-through and conversion rates - and really, Amazon's revenue results speak for themselves, the company posted $35.75 billion in sales in the last quarter of 2015 alone.
Netflix too is an interesting example of an algorithm in action - did you know that 'House of Cards' was actually conceptualized on the back of Netflix user data, essentially the same information that fuels their recommendation algorithm?
From the New York Post:
"Netflix, which has 27 million subscribers in the nation, and 33 million worldwide, ran the numbers. It already knew that a healthy share had streamed the work of [David] Fincher, the director of "The Social Network," from beginning to end. And films featuring [Kevin] Spacey had always done well, as had the British version of "House of Cards". With those three circles of interest, Netflix was able to find a Venn diagram intersection that suggested that buying the series would be a very good bet on original programming."
Algorithms are in effect more than most people realize, but those versions of machine learning systems feel less personal than changes to your social networks. Google and Amazon aren't telling you what you want, they're suggesting things you might like based on what they know, and that feels less obtrusive, less about control and more about choice, which is what people want. Give people a choice, even if you know the option they'll choose, and people will feel happier with the outcome. But take that element of choice away, regardless of the end result, and people will instinctively resist.
Yet, despite this, and the backlash against every switch to an algorithm-based system, algorithms, as highlighted in these examples, actually deliver better outcomes. And despite the difference in emphasis, the data is no different when it comes to social platforms.
Here's the data on algorithms as per the networks themselves. Back in 2010, when Facebook introduced the first iteration of their News Feed algorithm (then called 'EdgeRank'), the average session time, per user, on Facebook was around 13.5 minutes a day. At the time that was significant, but in 2015, when Facebook released their second quarter earnings results, CEO Mark Zuckerberg noted that on-platform engagement was now up to 46 minutes, per user, per day, across The Social Network.
Of course, you can't attribute all of that to the implementation of an algorithm - wider adoption of, and reliance upon, social media within that time frame would suggest that user engagement would inevitably increase. But at the same time, there's no been no decrease in engagement. For all the user gripes and complaints, people are actually using Facebook more and more, every single day.
When Twitter announced they were introducing an algorithm-defined feed recently, the company noted that:
"...people who use this new feature tend to Retweet and Tweet more, creating more live commentary and conversations..."
And you'd assume Twitter would have done the research to back up such a move, they wouldn't be looking to implement it if they had any concerns that it might negatively impact the user experience (the last thing Twitter needs right now is to see their active user numbers decline). Twitter's since switched their algorithm timeline to the default option and have rolled it out to all users, and the results thus far have supported those initial findings - a Twitter spokesman told Marketing Land last week that less than 10% of users have opted out of the new feature.
Instagram, too, have run their own numbers - from the Instagram blog:
"...people miss on average 70 percent of their feeds. As Instagram has grown, it's become harder to keep up with all the photos and videos people share."
As the 'noise' on each platform gets louder due to the ever-increasing amount of users, it's basically inevitable that, at some stage, it'll no longer be possible for people to see everything that they could potentially be shown, based on their own selections. On Facebook, for example, each user could be served up to 1,500 posts per day based on average Like activity, way more than anyone has time to consume.
Given that people are only seeing a fraction of the content they've indicated an interest in, it's crucially important that the networks themselves do what they can to ensure that they're providing the best user experience possible. Like Google, if spammers are allowed to run riot, the search results end up meaningless and people will go elsewhere as a result. On Facebook, Instagram and Twitter, the concept is essentially the same - Facebook's actually had to confront that exact issue several times, with Pages sending out Like-bait content; re-purposed images of kids with cancer asking for Likes, inspirational quotes of questionable origin and funny cat pictures. If 'Likes' were the only indicator of preference, Facebook would be a never-ending stream of these types of posts, and while such material has its place on the network, not everyone wants a constant feed of such content. Eventually, without some level of filtering, user experience is diminished and engagement rates drop.
But it's the 'how' of an algorithm that really puts people on edge.
You Don't Know Me
So how can a machine know you? How could a Matrix-style flow of '1's and '0's accurately learn your preferences and personal tastes?
It can't, right?
If computers could do that then we'd already be reaching the next level of artificial intelligence, beyond the capacity to beat people at chess or Go.
In actual fact, we are at that next level - while the applications available are not necessarily advanced enough to self-learn, independent of human support, there are several applications of machine learning that would already put it into that next echelon of understanding, in regards to our preferences and intended meanings.
At Google, this next level of understanding user intent is represented by RankBrain, their machine learning system which already processes a 'very large fraction' of search queries conducted via the search giant every day. RankBrain utilizes machine learning to take into account tens of thousands of possible factors in order to deliver the best possible search results in response to each query, and via RankBrain, Google's able to process infinitely more potential relevance factors in order to deliver more refined, customized results. The impact of RankBrain is significant - at some stage, the expectation is that Google's algorithm will be so reliant on machine learning systems that no one, not even the highest SEO experts within Google itself, will be able to tell you all the factors that correlate to the results shown to each user.
And as noted, RankBrain is already in use, this is happening right now - machine learning is already providing you with increasingly relevant search results, whether you realize it or not.
In regards to social networks more specifically, researchers from Stanford University and the University of Cambridge conducted a study in 2014 which found that your Facebook Like profile - a mapping of all the things you've Liked in your time using The Social Network - can provide a more accurate psychological profile of who you are than your friends, your family, even your partner.
But more than that, one of the head researchers from the project, Dr. Michal Kosinski, noted that actually, Facebook data was sometimes more accurate in profiling peoples' leanings that the individuals themselves.
As noted in this post about the study on Wired:
"At times, the Facebook model could even beat the subjects' own answers. As part of the survey, the researchers also asked respondents to answer concrete questions, such as how many drinks they have a week or what type of career path they've chosen. Then, they tried to see if they could predict how many drinks someone was likely to have in a week based on their answers to the personality test. Once again, they found that Facebook Likes were a better indicator of people's substance use than even their own questionnaires were. "When people take the questionnaire, they present themselves in a slightly more positive way than they really are," Kosinski says. "This tendency to self-enhance makes computers slightly more objective."
Again, this is based on real data, the information you've already uploaded to Facebook.
The key element behind such powerful insight is volume - one person Liking a single Page is meaningless in isolation, but when you consider the dataset that Facebook has to work with, with more than 1.59 billion users worldwide, undeniable patterns and correlations emerge, behavioral links that become indicative. Given this, it's actually little surprise Facebook has been able to build an algorithm that's delivering a better user experience - the data pool they have available is simply beyond anything that's come before it.
In fact, I asked Dr. Kosinski about this specifically, in regards to how Facebook is using their data to fuel their News Feed algorithm:
"Facebook's doing quite an amazing job in terms of improving user experience. The News Feed feature is, in essence, an ingenuous information recommendation mechanism, selecting the stories that users are most likely to be interested in."
That's a significant endorsement from someone who knows Facebook data, having conducted the largest published report on its correlations with individual preferences.
And again, this level of data is already being utilized, these correlations and tracking processes are already mapping your preferences and behaviors to get a better understanding of what you like and what you're interested in. Whether you like it or not, machine-learning systems probably already know more about your personal preferences, based on your combined social and search activities, than anyone you know.
How could machines know you and what you like? Because you've told them, and you've been telling them for years. And the more data you enter, the smarter these systems get.
The truth about algorithms is they work.
They deliver better user experiences, they sort the signals from the ever-increasing noise of social networks and they help connect users to the content of most relevance to them. And, because of this, we're only going to see more of them.
At some stage of growth, whether you like it or not, it makes sense for all social networks to implement an algorithm in some form. The ideal time to implement an algorithm is right at that point when engagement is starting to plateau, when the noise is increasing and people are seeing fewer posts, and are starting to spend less time on platform as a result. That would be indicative that the content that they are seeing is becoming less engaging, so rather than just let things slide, the networks are putting measures in place to maximize their on-platform material and enhance the user experience by showing them the content they'll most want to see. The logic makes perfect sense - why let your audience lose interest when you have the tools and posts available to keep them engaged, whether they understand the benefits of such a system or not?
You may not like the idea of an algorithm, you may not agree that they deliver better results. But the data speaks for itself.
Unfortunately, in this instance at least, we may be better off ceding control to the robots and letting them help us create better experiences.
And really, isn't that fundamentally what machines are built to do?
Main image via Shutterstock
Follow Andrew Hutchinson on Twitter