Why did BBC iPlayer go social?

Simple. People love TV and Radio. But they also love telling people about it. People want others to enjoy just as much as they did. Often, they might evangelise 'round the watercooler' or down the pub. But with iPlayer and other media-on-demand services, suddenly you can take links to these programmes and email them round, or post them to social networking sites.

Now that's great, but when you're on Facebook or reading your email, you're not always in the mood to watch a 60 minute TV show or listen to a 3 hour Zane Lowe show.

Many people's watching and listening habits are driven by recommendations from friends, so what if you could see what your friends have recommended right at the moment you want to watch or listen to something? Wouldn't that be cool? It'd mean the next time you say "I've got an hour spare. I wonder what's good on iPlayer", you don't just have to rely on the schedule to pick programmes from - you have a ready made selection of awesome programmes, recommended by your friends, right there.

Clearly it was something we should try and make happen - the question was, How?

The Challenge

Friends. More and more people are using social networks to communicate with their friends. Some networks are different to others. For example, Facebook tends to be about the friends you've actually met. People on Twitter often follow people they know, but also celebrities or notable people within a field of interest. I follow lots of web geeks for example.

So how do we get your friends into iPlayer? I mean right there on the homepage of iPlayer? We had a few options.

1: build our own social network

One way of doing this was to build a totally standalone social network - one where you can find people on the BBC, add them as friends, and voila! Well, why do that? People have got friends already - they're on Facebook, Twitter, and many other social networking sites. Building our own would have made it hard work to find your friends again, and hard work to keep it up to date. Not cool. Anyway, there are already too many social networks - the world doesn't need one more. In fact, the BBC in it's recent strategy review said we "should not create stand-alone social networking sites, with any social propositions on the BBC site only there to aid engagement with BBC content... [we] will also ensure that [our] social activity works with external social networks". Clearly, this option was a no-go.

2: just do Facebook

Facebook, and its Facebook Connect system for third-parties has been phenomenally successful - used by hundreds of thousands of sites and millions of users. It allows a site to not bother building its own login system or social network, and in effect, to outsource it all to Facebook. Users get a nearly-one-click sign in, and a ready made circle of friends. Simple. But for the BBC, it had one big drawback. What about people who don't use Facebook? Believe it or not, plenty of people don't. As a public service organisation, should we limit any social functionality in iPlayer to only those licence fee payers who use Facebook? No, we shouldn't. We'd really prefer something which lets people pick from a range of social networks, and something which is flexible enough to change and adapt as the popularity of social networks changes.

3: do Facebook AND other social networks

Another, simpler solution would be to let you pick just one social network from a range of social networks, and see just your friends from that social network in iPlayer. Sounds alright, but you'd only see recommendations from your friends who'd also chosen the same social network as you. Imagine if your mate Dave had picked Twitter, and you'd picked Facebook. You wouldn't see anything he'd recommended. Rubbish.

4: a hybrid solution

Looking at all the previous options, it was clear we needed to do something different. We wanted users to be able to choose from a range of social networks - and allow people to connect to multiple networks, not just one. We also wanted it to just work - set it up and off it goes. We didn't want people to have to go back to select new people to add every few days, or have to worry about where they came from. We just wanted all the recommendations, from all your friends, in one place.

What we eventually came up with allows you to get exactly that. You can connect your BBC iD to a range of social networks (currently two: Facebook and Twitter, but expandable over time) and our system will do all the clever stuff to bring your friends from all those social networks right into iPlayer.

For example, I'm pretty active on both Facebook and Twitter. Facebook is where most of my real friends are, but on Twitter I follow loads of really interesting people. With the technology we've built for BBC iPlayer, I get to see friends from both those sites right there in the Friends drawer, no segregation, no duplication. Superb.

Automagically, our system will find all your friends from all your networks who also use iPlayer. And as more of your friends connect their BBC iDs to Facebook and Twitter, they'll just start appearing on the iPlayer homepage as soon as they recommend some content. "But what if one of my friends keeps giving me rubbish recommendations?"... I hear you ask. Luckily, you've still got the option to remove people in your BBC iD Social Settings. Removing people just means you won't see them any more in iPlayer. Don't worry, they're still your friend on Facebook or Twitter, but you no longer have to be bombarded with their 'dodgy' recommendations.

The clever technology layer that makes this all work is something we call SNeS (officially 'Social Networking Services', but named in homage to the classic console). SNeS is actually our own implementation of OpenSocial, the Google-backed project which standardises social applications. Furthermore, SNeS is built around Shindig, the reference implementation of OpenSocial. It's SNeS that keeps track of who your friends are, and what they've been recommending - and serves it up to you, in real time right there in iPlayer. It's the social brain of the BBC website. SNeS is a proper heavyweight piece of engineering. It can deal with millions of users making millions of recommendations to millions of friends - something you need when you're working with one of the biggest video sites on the web. You'll be seeing more in-depth blog posts about how it all works over the coming months.

But why did you put all the social stuff behind your own BBC iD?

The plan for the next year is to roll out the full, social, personalised BBC iPlayer experience across nearly all the other platforms which currently support iPlayer. That includes mobiles, games consoles, set-top boxes and IP-connected TVs. We're going to bring Favourites and For You to those platforms too - and you're going to need BBC iD if you want your personal Favourites and suggestions to follow you across platforms. Given this, it would have been silly to make you connect to Twitter and Facebook directly on each of these platforms. Besides, iPlayer is available on many more diverse platforms that Facebook Connect is.

By putting our social connectivity behind BBC iD, it means all the great social features will be rolled out to all our other platforms over time. It means you only have to connect once to Facebook and Twitter on the web, and manage all your settings there. Then, when you're on your IP-connected TV or your mobile device - your Friends will be right there, recommending great content to you.

Broadcasting your Recommendations

Everything I've talked about so far is about getting recommendations from Friends - but for this to be of any use, we had to get people to make recommendations in the first place.

Again, we could have just added a few links on each programme page saying 'post to Facebook' or 'post to Twitter'. But as we discussed earlier, we wanted to show what people were recommending in iPlayer, not just in Facebook and Twitter. Only posting out to Facebook or Twitter would have meant we'd have to constantly monitor the whole of Facebook and Twitter to see what your Friends were recommending, and pull that into iPlayer. Not an easy task, and something which gets much harder the more people join. Even so, scouring Facebook and Twitter for direct recommendations is something we're actively looking at for the future.

In the meantime, our solution was to allow you to make public recommendations on the BBC website.

When you turn on the social features in BBC iPlayer, you're opting in to getting a public profile page on the BBC website. Here's mine for example. Any recommendations you make in iPlayer (or indeed elsewhere as we roll out the ability to Recommend across the rest of the BBC website) will appear on your public profile for others to see. This isn't new for the BBC at all - we've had public profiles and public spaces on the BBC website for years now. 

These profiles also include some cool new data views - namely RSS and Atom (using ActivityStreams extensions). This means you can use these feeds on your site, your blog, on other social networks or your favourite feedreader. It makes it really easy to add what you recommend on the BBC into Google Buzz for example.

By allowing you to make public recommendations we're allowing you to say to the world "I think this is great!". It adds a whole new dimension to the way you can navigate the BBC site. Imagine seeing other people who've recommended similar things to you - and find even more great content through them. We call that Social Discovery - and we think its a fantastic new way of finding new stuff to watch, listen or read.

Having public recommendations is also key in allowing us to support social networks like Twitter which have what's called an asymmetric friendship model. This means its possible to 'follow' a user without them having to follow you back (Facebook on the other hand uses a symmetric friendship model - each of your Facebook friends has agreed to be your friend - so you both have each other as friends). Imagine if your recommendations were only visible to people who follow you - there'd be an odd scenario where someone who follows you on Twitter, that you don't follow, could see your BBC recommendations - effectively making them public anyway.

Other services like Spotify have taken exactly the same approach to this problem in their social integrations. And like Spotify, you have complete control over your Public Profile. You can delete individual recommendations, you can change or remove your public profile picture, and you can choose any BBC iD DisplayName you want if you prefer to keep your anonymity. We hope the benefits and simplicity of allowing you to make public recommendations is balanced by the control you have over what you recommend, and what you can remove.

What next?

It's very early days for the social features in iPlayer, but it's fair to say this is exciting new ground for us - and we're monitoring how people use these new features closely. We're actively asking for feedback at this stage (via the admittedly verbose #BBCiPlayerfeedback hashtag on Twitter or comments at the bottom of this entry).

We're looking at improvements we can make to the social features in BBC iPlayer, but we're also looking at other parts of the BBC site where this kind of deeply-embedded social functionality would make a real benefit to some of our users. If it goes down well, and people find it useful, expect to see the ability to Recommend content outwards, and see what your Facebook and Twitter friends have been recommending, spread through more parts of the BBC website.

Simon Cross is Executive Product Manager, BBC iD and Flow,

A few people have posted blog comments asking why we've done this, and what what it means for the future. I thought I'd write this to help explain what we're doing and why.

So, why did we build BBC iD?
The simple answer is that our old system - called 'Single Sign On', or SSO - needed replacing. It had been around for nearly 6 years, skillfully powering all the BBC's online services which required authentication, but 6 years is a long time on the web. SSO has been showing its age in some very specific ways:

The technology platform
SSO was built on Perl and MySQL. Good technologies for their time, but the BBC is moving towards a new online architecture (internally called 'Forge') which uses Java and PHP on top of MySQL, Apache and Memcached. Soon, the old Perl-based system will be turned off. SSO would have to have been ported to Forge anyway, so it was a good time to completely refresh it from the ground up.

Performance
SSO used a single MySQL database instance. Forge allows applications to have multiple partitioned databases - which helps to make it horizontally scalable. This means that as BBC iD gets used more and more, we can make it perform simply by adding more servers. Until recently, you only signed in to small pockets of the BBC - the odd messageboard here, a one-off application there.

However, with the advent of BBC iD, nearly every page on BBC Online will know if you're signed in or not, and will be able to adjust itself accordingly. This new level of personalisation will allow BBC Online to grow and personalise around you in ways that were never before possible. But this level of integration, and load, will needed a totally new architecture which made heavy use of partitioned (sharded) databases, Memcache, and load balancing.

Internationalisation
BBC Online continues to grow its audience internationally, and has a staggering number of language sites. As these sites want to do things like personalisation, they need sign in features in their native language. Adding features like these retroactively to a product is really hard - they have to be built in from the start. One more reason why we knew SSO had to be replaced.

Although the first versions of BBC iD are english-only, under the hood, it's been designed with internationalisation in mind. For example, every bit of text you see isn't embedded into the code, it comes from a language specific package. We're now working on increasing the number of supported locales. This will eventualy include not only the main UK languages like Welsh and Gaelic, but languages with different characters (like cyrillic in Russian) and right-to-left text (persian etc) - in fact, anything you can throw at Unicode.

Security
Since SSO was developed, security techniques and technologies have moved on a lot. For example, a while back it was impossible to support the loads we needed to support and encrypt data both in transit and on disk. Now, that's possible. As such, BBC iD has been built from the ground up with very secure architecture in mind. All personal data is stored on disk encrypted, all personal data is transferred over https, and inside the BBC there are strict access controls put in place to make sure only the staff who are authorised have access to it. While SSO was good for its time, the security model had to be thoroughly rethought.

But why build your own sign-in system at all?
OpenID, Facebook Connect, OAuth - the modern web is full of distributed, decentralised identity systems. We could have just forgotten about building our own system, and just implemented one, or all, of these.

Well, the good news is they're on their way! BBC iD was built from the ground up to be compatible with OpenID and other distributed authentication systems and later this year, we'll be introducing the ability for you to sign in to BBC Online using your Facebook login via Facebook Connect, and your Google and Yahoo logins (and more) via OpenID.

However, we still felt we needed our own base-level sign in system, both for those users who don't have external logins they want to use, and also for those who just don't want these things linked together. As the BBC has a mandate to serve all licence fee payers, building our own standalone system was a necessary evil.

Truly, Single Sign On
The biggest problem with the old SSO system was that, although it was actually a bbc-wide sign on system, almost none of our users realised this. It was mainly down to some user-experience descisions within the SSO interface. While a tiny percentage did use their SSO account for more than one service, nearly everyone created a new SSO account for each BBC service they registered for. We're trying to move BBC Online to become a more social, more coherent website. As such, it's essential that our users realise they're signing into the whole BBC site - not just a part of it.

With the old SSO model, we had ghettos of interactivity which didn't connect with each other or the rest of the site; each had their own users, their own rules and their own user interfaces. This made it impossible to represent users on every part of BBC Online consistently.

BBC iD solves this problem in two ways.

Firstly, you can only have one BBC iD per email address. This is made clear as soon as you try and create a second BBC iD with the same email address. A single BBC iD can be used across BBC Online and a person can have more than one BBC iD, but they'll need a separate personal email address to register with for each one. Contrary to some comments on our blogs, BBC iDs are not limited by IP address, so you can have more than one per household. The email address is the important unique field.

Secondly, we created a 'brand' for our login. We're not the first to do this, Yahoo, Google, Apple all do it. And remember Microsoft Passport? We'd rather not have called it anything, but we did lots of testing that showed that people didn't realise their login was global across our site unless we branded it. We've been careful to keep is a 'soft' brand though. It's represented by colour, language and iconography. This consistent message should remind users where ever they see the 'Cid' symbol (Cid's the bod on the badges pictured above, derived from BBC iD) and the words 'sign in', that they can use the same sign in details they use elsewhere on BBC Online.

By contrast, SSO's sign in and register pages were branded to match the service you came from - further reinforcing the impression that SSO was service-specific sign in.

But it's a pain to upgrade
Yes it is. Transitioning users from the old system to the new system is not easy. We could have just copied all the old user data from SSO into our new system, but that would have meant millions, literally millions, of old, dead unused accounts in our nice, clean, new system. Instead, we chose to allow our users to 'upgrade' their old SSO accounts to BBC iD. While this is a little annoying for some users, it is a one-time only process, and means the users we have in BBC iD have new, clean data - and best of all, it means people can register with sensible usernames again. With 13 million accounts created over 8 years, SSO was full of old, bad data.

We take our users' experiences very seriously, so we've done all we can to make the upgrade process simple, reliable and quick. There will always be some people who experience problems, but we monitor our stats and our help email addresses very closely and try and help each and every one of our users who has problems.

Will it be worth it?
The short answer is, yes.

Change is often disruptive, but necessary. The rollout of BBC iD across BBC Online will allow our site to do incredible new things - more personalisation, better interactivity and provide more security to our users. Without this move to use BBC iD, BBC Online would not be able to build, grow and become a properly modern interactive, coherent site.

Simon Cross is the Product Manager for BBC iD.

First, we did the iPhone, which was swish, and very zeitgeisty. But we always said we'd be doing the same for other devices, so here's our first simple-but-elegant iteration of this.

Read more and comment at the Radio Labs blog.

Simon Cross is Senior Client Side Developer, Audio & Music.