Technology + Creativity at the BBC Feed

Looks good to me: Making code reviews better for remote-first teams

2021-02-23T10:44:31+00:00

The Russian version of the BBC News website.

Why do you do code reviews? Perhaps it’s company policy, just an automatic part of your process, but have you ever sat down with your team and asked what everyone hopes to get out of it?

As a developer, has it ever felt like playing a strange board game where the rules are secret and keep on changing? Or as a delivery manager, have you ever been puzzled why reviews sometimes seems to take longer than writing the code itself?

These are some of the things we were asking ourselves at BBC News a year ago. We’re not sure we’ve found all the answers yet. But we think what we’ve learned so far has improved our engineering culture and helped to make code reviews a better experience for everyone.
In this post I’ll share why we started asking these questions, and some of the things we found out along the way.

Our first taste of remote-first working

Throughout 2019 the teams working on the BBC News and World Service websites went through a period of rapid expansion and changing priorities. What had started out as a close-knit team of five building a reimagined articles page for BBC News ended up with 35 engineers across five teams contributing to the same product.

Suddenly we weren’t just building one page type, we were rebuilding all 41 World Service language websites from scratch.

There were other changes. At the start of the year, everyone was co-located around the same bank of desks at Broadcasting House in London. Instant feedback and advice were usually available by turning to the person sat next to you, and pairing came relatively easily.

By December however, the engineers were split across six geographical locations in three time zones. While nobody could have imagined the upheavals of 2020 that lay ahead, we found ourselves having to adapt to a remote-first culture rapidly.

Growing pains

Despite the stresses of scaling sevenfold in a matter of months, the teams made a great success of the changes: by mid-2020 we had successfully rebuilt a set of sites that deliver free and impartial journalism to 35m unique visitors globally per week (for an overview of the BBC’s cloud journey see Matthew Clark’s recent post, and for the performance improvements in the new sites see Chris Hinds’ post here).

But there were definitely challenges along the way, and one area that was often raised as a source of difficulties was peer code reviews. A large influx of engineers from other teams, most of whom were learning about the codebases for the first time, started opening pull requests (PRs) to get feedback on their work.

We saw that these PRs would sometimes spend considerable time in the code review stage, often cycling between review and re-work multiple times. Approaches to giving and receiving reviews seemed inconsistent. Some developers reported that they found the process unclear, and at times stressful.

It was clear it was time to talk and reflect as an engineering community. We started asking what this ‘code review’ thing we were doing automatically was actually for. We decided we needed a kind of charter for code reviews.

Creating our very own charter

Why make our own? After all, there are already numerous excellent online resources available (such as Gergely Orosz’s blog post and April Wensel’s talk).

The answer was twofold. Firstly, we wanted to create a shared, living document that would evolve over time as we learned more about what works and does not work for our teams.

Secondly, while existing resources offer many useful global principles, a lot of the questions we had required local answers, specific to the structure of our organisation, our codebases and the design of our automated tooling and Continuous Delivery processes. For example: who can ‘approve’ a PR? Do all types of changes get reviewed in the same way (documentation, config, infrastructure code etc.)?

As Gergely Orosz points out, having an engineer-initiated guide to code reviewing is one hallmark of organisational support for the practice. So we started an anonymous collaborative document to source ideas, frank observations and anecdotes about code reviews from engineers of all levels of experience. Then, with feedback and advice from a range of viewpoints we gradually distilled that advice into our guide.

We wanted to make sure that the guide was easy to discover and would continue to be updated, so we put it right into our repositories and linked to it from our top-level READMEs and PR templates.

And as an added bonus, the project we were working on is open source, meaning that potential contributors from outside the BBC benefit from more transparency about how we aim to review contributions.

A quick quiz

What’s the goal of doing code reviews? Is it:

To catch defects before they reach production
To share knowledge about helpful patterns and best practices
To discuss alternative approaches and viewpoints
To allow developers of all levels of experience to learn
To link to useful documentation and other resources
To distribute knowledge across the team
To ask questions and check understanding
To improve readability and maintainability of the code
To identify documentation needs
To ensure that work meets quality standards
To record the rationale for certain changes
To coach and mentor junior developers
To promote sharing ownership of the codebase
To notify other affected teams of changes that are being made
All of the above

If you answered “all of the above,” you’ve come to the same conclusion that we did.

We’ve seen over and again that a good code review can achieve all of the above and more (which is not to say that code reviews are the only way to achieve these things). Participating in code reviews in a spirit of open, egoless collaboration is the key to unlocking all of these benefits.

In fact, we sometimes wonder if catching defects is one of the weakest arguments for code reviews, at least in the relatively unstructured way we do them. Think about the last time a bug hit production and caused you a problem, perhaps because of something as trivial as a typo in some config. Could it realistically have been caught during code review? Was it? The truth is that busy developers are not always great at playing ‘spot the difference’.

By the way, the idea that there is a discrepancy between what we expect from code reviews and what they actually achieve is not a new one. An empirical analysis of hundreds of code reviews at Microsoft in 2013 found that sharing understanding and social communication were more common outcomes than finding defects.

What we learned from the process

A few of the most interesting themes that emerged from our reading and internal discussions are summarised below. For more context, see the full guide linked to at the end of this section.

Reviewing code is a first-class activity

We think learning to review code well is as important as learning to write it well. Code reviews are not a second-class activity, to be squeezed in between ‘real’ work. We allow sufficient time in our planning and estimates to allow it to take place, and we don’t neglect it in emergencies, as the cost of haste at these times can be even greater. We actively nurture and develop the skill of reviewing in engineers of all levels.

Communication is at the heart of code review

As mentioned earlier, often the real benefits of code reviews are communication and understanding. GitHub is a powerful collaboration tool but not the only one at our disposal. Jumping on a Zoom or Teams call to talk it through can be friendlier and more efficient than back-and-forth debates on PRs.

This is especially important for growing, distributed teams where there is greater scope for confusion and misunderstandings. We also find group (aka swarm) code reviews effective, especially for significant new abstractions.

Don’t miss the chance to learn

To quote from our guide:

Our primary aim in participating in code reviews is to learn from each other, increasing our understanding of the codebase we are responsible for and of the technologies we use.

Code reviews aren’t just about getting a rubber stamp of approval from someone (often written LGTM, or Looks Good To Me). Every code review is an opportunity to ask questions, share knowledge and consider alternative ways of doing things. Done properly, both authors and reviewers can expect to learn and grow from this process.

And if potential bugs are caught along the way, that’s awesome too.

Remember to be kind

Code reviews can be emotionally demanding for both sides. Pride can come into play — authors may have spent a lot of time preparing their changes, and others may be protective of a codebase.

Taking a cue from the Agile retrospectives Prime Directive, we assume that everyone did the best job they could, given what they knew at the time and the resources available. As we found, trying always to understand the other person’s perspective is particularly critical when new team members are contributing to an established codebase.

Language is also important (Philipp Hauer provides some excellent examples of this). Careless use of evaluations and jargon can undermine the spirit of collaboration that ought to underpin code review. For the same reason we also make a conscious effort to to replace terminology that has racist, sexist or ableist associations.

Keep things in proportion

Many of us have seen cases where a review hasn’t gone so well. A high comment count, multiple revision cycles and comments that have forgotten to be kind are all warning signs that team leads should be watching out for.

Time spent on code review should be kept in proportion with other stages of the development life cycle including testing and accessibility and UX review.

We find that as a rule of thumb, if a change has spent longer in review than it took to implement the code that might also point to a wider problem that needs attention. Are team members prioritising reviews correctly? If there are lengthy conversations, perhaps some of them could have happened earlier in the process?

It may also be helpful for dev teams to regularly reflect on reviews which were harder or less effective than they should have been.

Look for every opportunity to make reviews easier

We’ve noticed that changes that have been authored by two engineers pairing or a larger group swarming often seem to spend less time in code review, with fewer review cycles. This makes sense given the above points about communication, as a lot of potential questions and concerns can be pre-empted, and the change should have already benefited from a more diverse range of skills and viewpoints.

And as April Wensel suggests, automate what you can, for example using linters effectively to minimise trivial nitpicks.

More detail on the above can be found in the full guide at https://github.com/bbc/simorgh/blob/latest/docs/Code-Reviews.md. Bear in mind that some of it is intentionally quite specific to how we work at BBC News and may not apply in your context.

Ask difficult questions!

There are many potential benefits to doing code reviews, but they are not always what people expect, and there are pitfalls along the way too. Even so, not all teams will need or want to go through the process of reflection described above. Perhaps tacit assumptions about code reviews are working great for you, or you find one of the existing online resources sufficient.

But in the spirit of continuous improvement and trying to make the way we collaborate more transparent, we found writing our guide a really positive experience. And with remote working now the norm, having a clear shared understanding of these activities is more vital than ever.

The document is now part of our onboarding process for new engineers, and we’ve seen promising feedback about the cultural benefits that talking frankly about code reviews has made.

But the journey doesn’t end there — we’ll continue to update our guide as we learn more about what works for our teams. Sometimes it’s only by asking difficult questions about every stage in our development process that we can really improve the way we build software together.

BBC World Service: Migrating 31 million readers and an 83% improvement in page performance

2020-11-25T10:47:40+00:00

The BBC World Service publishes news stories in over 40 languages globally. Stories are written by journalists around the world in their native language instead of using translations. World Service covers everything from local to global news and content is delivered in multiple formats, including text, video and audio.

As mentioned in Moving BBC Online to the cloud the frontend (and many backend services) powering the World Service websites were previously written mostly in PHP, hosted by BBC-owned data centres. Over the last couple of years, teams within BBC Online have been working tirelessly to migrate their services to the cloud and the BBC World Service has nearly completed this transition.

Over the past 12 months we’ve migrated our pages which are spread across 41 discrete sites from a legacy PHP monolith to a new React based application. This application is called Simorgh, an open source, isomorphic single page application developed by the World Service languages team.

What do we mean by Single page application and Isomorphic?

Single Page Application (SPA)

A single page application (SPA) is a web application that works solely in the browser, removing the need to refresh or reload the page. This creates an outstanding user experience that feels close to that of a native mobile application. Some common services you may use on a daily basis make use of this technology, including Gmail, GitHub, Facebook and Google Maps.

Isomorphic

An isomorphic (sometimes referred to as “universal”) app is a web app that can run on both the server and the client. The idea being that the first request to a web page e.g. https://www.bbc.com/mundo will be rendered on the web server delivering server side rendered HTML to the readers browser. Once the rendered page reaches the client and the JavaScript is downloaded and parsed the browser is able to take control and then treat subsequent page views as a single page application. In React this is handled via the React hydrate function which “hydrates” the client side DOM with the data that was used to render the page on the server. In most cases the reader does not notice this phase as React is performing a diff on the DOM between the server side render and the client side render. For the most part these will be identical, however, at this point React is in control of the page rendering in the browser.

Simorgh (The React SPA built by the BBC World Service)

Simorgh is the rendering platform built by the BBC World Service web team using the technologies described above. What made Simorgh challenging to build wasn’t the technology we used, but the specific requirements of BBC World Service. When building Simorgh to replace our dated PHP solution we had to bear in mind the following:

Performance — The websites must be as performant as they can be. Many of our readers are on lower end smart/feature phones on networks with low bandwidth rates and high data costs, slow connections and patchy coverage.
Accessibility — The BBC aims to provide a fully accessible web platform, ensuring that anyone can access our websites using any assistive technology.
Support for multiple languages — BBC World Service currently supports 41 different languages, each language site has its own editorial team and from the outside this is seen as 41 separate websites.
Huge volumes of traffic — The World Service currently serves 31m weekly readers. Simorgh despite being behind many different caching/routing layers is rendering on average 1 million unique pages per day with an average of 11 million daily renders across the 41 languages.
First class AMP support — Offering AMP variants of all supported pages. This allows us to move away from the previously separate AMP rendering system that was built on an internal Ruby based framework for static rendering.

We are planning to post a dedicated writeup on the history of Simorgh and the technologies chosen in the near future so keep an eye out for that.

So where are we today?

Simorgh currently supports twelve different page types:

Front Page: https://www.bbc.com/mundo
Story Page (Current CMS): https://www.bbc.com/mundo/noticias-54840845
Articles (New CMS): https://www.bbc.com/kyrgyz/articles/c41p9ll9n0po
Media Asset Page: https://www.bbc.com/mundo/noticias-internacional-54866398
Photo Gallery Page: https://www.bbc.com/mundo/noticias-america-latina-54689363
Feature Indexes: https://www.bbc.com/mundo/noticias-36795069
Index Page: https://www.bbc.com/persian/afghanistan
MostRead Page: https://www.bbc.com/persian/popular/read
On Demand Radio Page: https://www.bbc.com/persian/bbc_dari_radio/programmes/p0340v0s
On Demand TV Page: https://www.bbc.com/persian/bbc_persian_tv/tv_programmes/w13xttnr
Live Radio Page: https://www.bbc.com/persian/bbc_persian_radio/liveradio

These pages may appear visually similar, but our internal content management system treats them as different page types. One of the biggest advantages we have seen by rebuilding our platform is the ability to reuse code as much as possible. Many of these pages share the same code and React components. These components are located in our open source React component library https://github.com/bbc/psammead (Storybook https://bbc.github.io/psammead).

We knew that we wanted to focus on improving web performance for the World Service, but this was difficult with the previous PHP platform. So one of the main goals of the migration was to build a platform that would enable this type of rapid prototyping, allowing us to make changes and improve the feedback loop of those pages.

Web performance wasn’t the number one goal when we created Simorgh, but as we followed best practice in developing the new platform, we did see vast improvements compared to the old one. Some of this was attributed to some early design decisions such as no blocking JS, minimal layout shift, server side rendering etc.

We released pages in batches, grouped by the language and the first language we released onto the new platform saw huge gains performance in many areas.

Lighthouse performance score saw a 224% increase from 24 > 94
Lighthouse best practice score saw a 27% increase from 79 > 100
Total number of requests dropped by 85% to 17 down from 112
Blocking JS requests dropped by 100% from 9 to 0
JS requests dropped by 79%
Total page weight is now 60% smaller than before
JS size dropped by 61%
Dom Content Loaded is 85% faster at just 0.4s down from 2.6s
Visually complete time dropped by 62% down to just 1.8s vs the previous 4.7

These performance metrics were captured using SpeedCurve comparing the same url on the old platform and the new.

So as you can see we have already made a great improvement in our frontend web performance — but we won’t stop here. A large proportion of the BBC World Service audience are on slower 2g and 3g networks, they use lower end budget-friendly android handsets or feature phones. In some of our supported regions network coverage is patchy at best, some readers may only have network access whilst traveling to work or whilst at work. We must continue to make improvements in every way we can to make our pages some of the most accessible web pages in the news category both in terms of accessibility requirements and performance.

This video demonstrates the performance improvement between the old and new platforms.

This external content is available at its source: pal v Simorgh load time

Post migration improvements

Since the migration we have already released a number of new features that aim to help improve performance, perhaps the most notable one was the lazyloading of social embeds (Tweets, Instagram posts and YouTube videos).

Social embeds are often a key part of telling a story. We have found that many of our journalists add a number of social embeds to each page. For instance one language always embeds 2–3 YouTube videos at the bottom of each story. When looking into the performance metrics for these pages we noticed upwards of 500Kb of JS (that was more than the entire Simorgh application) was being loaded by YouTube and that some of this JS was actually blocking the rendering of our page as it was being parsed. In one extreme example the Time to First Byte (TTFB) was at 12s.

This content had to be on the page as it was part of the onward journeys experience. However not every reader would scroll down the page to where these embeds were rendered, so they shouldn't have to download the extra JavaScript or use the extra data allowance when they may never interact with these social embeds.

The Solution?

Lazyloading of third party content - We already do this with any images that are outside of viewport so why not for social embeds? A quick pull request later and we were lazyloading social embeds, no new library, no JS size increase, just using an already existing feature on the platform. Soon after releasing we saw a wide variety of results as these were dependent on where the social embeds were in the story and how many a given story had.

In most cases we were seeing a 10–15% improvement in TTI, as well as reducing, if not eliminating the render blocking time. Where I was most impressed though was in the story mentioned earlier. We had taken the TTI from 12s down to 6s. 6s is still a long time, however, this was a story with many different social embeds and in some ways a worst case scenario. In any-case a 50% improvement in just a few lines of code is phenomenal. This kind of change would not have been possible, at least not so quickly on the previous platform.

How are we monitoring web performance?

Now that the migration is complete, we are in a position to start making more improvements to web performance and changes to the platform. Before we can make many meaningful improvements to the application we need to be able to monitor web performance.

There are two common ways of monitoring web performance;

Synthetic Testing

Synthetic testing is great for catching regressions during the development lifecycle. We use Lighthouse, SpeedCurve and WebPageTest to measure our web page performance.

RUM (Real User Monitoring)

RUM testing is a method of capturing performance metrics from our users. RUM is generally more expensive in comparison to synthetic testing, however it provides a vital look into how real users are experiencing our site.

We use a combination of Synthetic and RUM monitoring for Simorgh. During development, Lighthouse runs on every pull request/feature branch. Lighthouse tests a subset of pages and for the most part is looking at the Accessibility, PWA and BestPractices audits.

Lighthouse is also used in our continuous delivery pipeline. After we deploy to the test environment, we run Lighthouse against the environment and can choose to fail the build if the audits fail. This same test will then also run against the live environment once the deployment is complete.

SpeedCurve runs daily tests against a smaller subset of URLs. SpeedCurve is a tool that essentially wraps around WebPageTest and Lighthouse, providing a fantastic UI on top of those underlying tools. These tests give us an insight into performance of our pages from different regions around the world.

Core Web-Vitals

A recent initiative from Google is the Core Web-Vitals. The idea behind these metrics is that they are a way to determine/monitor the user experience of your site. Google collects the metrics themselves from popular sites and publishes the CRUX dataset (Chrome User Experience). These metrics include things like; Time to first byte, First input delay, Cumulative Layout Shift.

Through a new package in our component library we are now able to collect these same metrics ourselves if the user has opted into performance tracking via the BBC cookies settings page.

In comparison to procuring a 3rd party tool, this is a cost-effective way for us to be collecting real user metrics. RUM is very important for us in the BBC World Service as our users are situated all around the world, they all use different devices with different capabilities and run on a wide variety of different networks. Getting this sort of test coverage with just synthetic testing would be impossible. This new data will allow us to start making informed production decisions about where we need to improve the web pages directly affecting the readers experience.

Screenshot of our custom RUM solution reporting on Web Vitals

We hope to publish a dedicated post in the near future about how we collect and use Web-Vitals.

Conclusion

It’s been a busy period for many teams at the BBC this year but we are seeing the light at the end of the tunnel. The World Service migration has been a great success thus far. We have migrated to a modern platform that is open source, and faster than ever both in terms of product/feature iteration and web performance.

Our journey has only just begun. Simorgh represents a new beginning for the BBC World Service, and we will continue to improve the performance and accessibility of our news web pages for our global audiences.

Leveraging the Tor Network to circumvent blocking of BBC News content

2019-10-30T08:18:33+00:00

BBC News and Tor logos

The BBC World Service's news content became available on the Tor network last week in a move that attracted wide media attention.

The decision to go ahead with setting this service up came at a time when BBC News is either blocked or restricted in several parts of the world.

For example, in Egypt, Iran and China, our audiences are finding it either impossible or difficult to access our content without the use of a circumvention tool, such as a VPN.

The Tor network is an overlay network on the internet, which provides increased security and is resistant to blocking.

The BBC is not the first leading organisation to have a direct presence on the Tor network. Facebook has been there since 2014; the implementation of the social media platform on the network was built by Facebook engineer Alec Muffett, who later left Facebook and subsequently assisted the New York Times in setting up their own Tor site.

As a result of his experiences, Alec created the Enterprise Onion Toolkit (EOTK), which makes it easier for any organisation to set themselves up on the Tor Network.

With help from the BBC Online Technology Group, Alec prototyped a solution based on the EOTK for the BBC World Service. The BBC has an unusually complex domain name configuration, and the prototype proved that the EOTK could handle this complexity well.

The implementation for the BBC was carried out by the Open Technology Fund (OTF) and Alec continues to be a key contributor. The OTF is one of the leading Internet freedom organisations in the world, who have found prominence through funding and vetting numerous information security and internet freedom projects.

Why an Onion service?

From a technical standpoint, the Tor network is a subset of the internet we know and use every day, and is accessed by users using a modified browser. The key feature of the Tor network is that it is fully encrypted. That’s to say, it hides the location of users, and the protocol it uses is continuously updated to maintain resistance to blocking.

This explains why it is a strong solution for the problem of internet censorship and secure communications and why it is being used by a large number of journalists, bloggers and internet users who live in muzzled media environments.

Users can already access the BBC (for example https://bbc.com/persian) on the Tor Browser to circumvent blocking. The user’s connection enters the Tor Network in one country, runs through at least three servers, then exits the Tor Network to the BBC website from another country. While successful in circumventing blocking, this route is exposed to censors who might monitor activity on the last exit server, which is unencrypted, or even tamper with it.

An alternative, called an Onion service, uses the Tor Network’s own address scheme where domain names end with “.onion”. In this case, traffic is directed to a dedicated node on the Tor Network for that service.

This allows the traffic between the Tor Network and the content provider to travel a trustworthy path. This also removes the risk associated with exit nodes.

An additional benefit is that the routing within the Tor Network is simplified when using an Onion service. This provides a much higher performance, which is especially noticeable when watching video.

The Tor Browser is available for Windows, MacBook and Linux computers, as well as Android phones. Alternative browsers, such as Brave or the Onion Browser (for iPhones) can also be used. These browsers can be used for both .onion and classic URLs.

The BBC homepage, with a URL accessed through the Tor network.

Is there different BBC content on the Tor network?

The BBC content on the Tor network is not different from that which is accessible to our international audiences under normal conditions.

The experience is similar to being in Ireland or the East Coast in the USA for example. Users will be able to access World Service radio, TV and websites in over 40 languages, as well as the news in English.

Content which is not available internationally, such as BBC iPlayer, will continue to be unavailable on the BBC Onion service. Users within the UK appear as international users when they use the BBC Onion service.

Technical risks

An aspect of setting up an Onion service for the BBC was the question of whether technical BBC assets will be placed on the Tor Network or whether the Onion service needs to be technically trusted by the BBC in any way.

Onion services are https-based and therefore do require their own server certificates and the certificate for the BBC Onion domain is separate from other BBC certificates. This allows users to trust that they are actually reaching BBC content.

The Onion service has to rewrite all of the URLs in order to make the BBC site work inside the Tor Network. It is therefore essential that the Onion service is operated securely and by a trusted team.

The work done by the EOTK platform does not involve placing any BBC assets on the Tor network itself. Neither does it need to be provisioned with any passwords or certificates to access BBC systems. To the BBC, it appears like a normal group of international users.

Content on the Tor network is therefore proxied through the Onion service and there is no additional web hosting commitment.

The BBC's duty of care

Some countries, such as Russia, China and the UAE, have passed laws to regulate the sale and distribution of tools such as VPNs.

For example, the UAE prohibits the use of VPNs to access illegal content. However, BBC content is not illegal in the UAE.

The promotion of the Onion site by the different BBC services will include clear warnings that users should be aware of their legal environments and should not use it if it might put them or those close to them under any risk or danger.

Information controls then and now

Controls placed by governments on access to information and trusted news are not new at all.

During the Cold War, some governments used to jam the shortwave radio broadcasts of the BBC World Service to stop their populations from listening to BBC. Then, the BBC circumvented these measures by providing new frequencies or changing frequency values to confuse jammers.

These controls are now moving on to the internet. At a time when internet freedom has been declining consistently and online information controls are growing, the BBC World Service continues to pursue its mission by providing an additional online news presence on the Tor Network.

The BBC News Onion site can be accessed at https://www.bbcnewsd73hkzno2ini43t4gblxvycyac5aw4gnv7t2rccijh7745uqd.onion (Link updated October 2021)

BBC Voice + AI: An insider's perspective

2019-10-24T10:42:53+00:00

I'm not the same journalist I was a year ago

If you had told me then that I would soon find myself doing a job that involved talking to a small cylinder all day, I probably wouldn’t have believed you. But here I am, surrounded by smart speakers. I’m getting on a first name basis with voice assistants like Alexa and Siri, learning all about voice technology and artificial intelligence.

Why? I’ve been working on a project to help better deliver BBC News via voice assistants. This is part of a wider Voice + AI project led by executive editor Mukul Devichand, to help the BBC operate in the best possible way as millions of people embrace this technology.

Everyday I ask questions like “If our audience could have a conversation with the BBC about what’s going on in the news, what would that be like?”. It’s a mixture of blue sky thinking, design sprints, workshops, audience testing, prototyping and swathes of post-it notes … all with a big dollop of fun on top.

I thought I’d share some observations from my journey so far.

Alexa, give me BBC News

The main project I have been involved in is the launch of a more interactive version of BBC News.

For now it’s available via Amazon Alexa: to hear it just say “Alexa, give me BBC News.” The listener can skip stories by saying next if they’ve heard enough, they can go back, pause, or ask for more information.

Fun behind-the-scenes fact for you, this product was originally given the (now legendary) nickname “Skippy” by the team at BBC News Labs because of the ability to skip through stories. Shout out to their brilliant software engineer, Lei He, who first started piloting Skippy back in 2017 and stayed with us in Voice + AI until recently to see it through.

In terms of structure, the service has two layers: main stories on top with deeper dives attached. And this is where it gets really interesting, when you ask for more information you get a richer piece of BBC content – think expert analysis or an exclusive interview. This is appealing because it would be pretty impossible to listen to all of the audio the BBC makes each day, so someone has done the work for you. Our research has shown that people don’t always want to or can’t interact because they’re busy doing something else, so we’ve designed the service to also suit someone who just wants a passive listen. Say nothing and you will just get the main stories.

News on a new platform

A team of journalists has been put together from within the main BBC newsroom to make the linear and interactive BBC News briefings. We know that people are using their smart speakers in their homes and so we have tried to adapt our tone and style to match this personal setting. We made a conscious decision to move away from broadcasting at people and try to use everyday natural language.

A recent workshop exploring what conversations between the audience and BBC News might sound like

During an early round of user testing one person said that being able to control the experience made it feel like a personalised news bulletin with minimal effort. We were very happy with that, but we also know that this voice space is still in its infancy and we’ll continue to listen to audience feedback and innovate as we go.

Some key learnings so far:

The new tone has fans. One person said that it is “more in line with the casual relationship you might have with a voice assistant at home”. Thumbs up. Of course, there’s a fine line between friendly and over-familiarity.
We need to pay attention to how we write for an interactive audio service. We had some users who would see the headlines right at the start as a menu and would then ask to jump to a story. After trying out quite a few things, in the end we settled on three brief headlines, just a line or two each. We now use what I call the ‘X, Y, Z format’ - e.g. “Today we’ve got X and Y, but first Z.”
Audiences change the way they interact with the service at different points in the day. In the morning they want a shorter, snappier experience, whereas in the evening they are likely to have more time to dig deeper into stories.

Settling In

I’ve really enjoyed the tales from our journalists about settling in to the new medium. In order not to be bad new neighbours they’ve been coming up with stealthy ways of listening back to their Voice content quietly. There’s the romantic technique, which involves hunching over the speaker murmuring sweet nothings to it and, what I like to call, the casino roller, where you hold the device up to your face and whisper at it as if blowing on dice for luck. They’ve also been dubbed by some in the Newsroom as “the team with the voice machine”.

Discussing how to write for Voice platforms

Learning the Lingo

As a reporter I covered a lot of technology stories but I wasn’t prepared for all the acronyms and unfamiliar terms our Design + Engineering teams had up their sleeves. It was like arriving on a new planet and not speaking the language. I still get thrown sometimes. The other day I got an email invite to a spike workshop and accepted without being entirely sure what I was agreeing to. Nothing sinister as it turns out, a spike is a time-limited investigation into whether some software would work for us, and, after being mocked by our Voice journalists the other day for using the word ‘deck’ instead of ‘presentation’, I reckon I’m now almost fluent.

Testing with students at University College London

Perhaps the most cutting edge sign at the BBC? Made with a DIY sticker

You're a journalist right? What do you do?

A developer asked me this a few months ago about my role within BBC Voice + AI and it stayed with me because it’s actually something I’ve been asking myself in a broader sense - what does it mean to be a journalist in a world where AI Assistants are a point of access for our work? Are we all going to need to learn how to code in order to stay in work? Considering how both the internet and social media have rocked the way we communicate and consume media, it’s mind-boggling to consider the impact something like conversational AI might have on our lives in 10, 20 or even 50 years time.

I don’t have the answer to the question above. No one does. But I’ve decided not to worry about it. And in my humble opinion, neither should any journalists out there.

Being resilient is in our DNA. Whatever the tools or the platform - notebook, microphone, thumbs for breaking news on social media - we are all storytellers.

This post originally appeared on the BBC News Labs website. See here to find out more about what BBC News Labs does.

Seeing isn't always believing

2018-11-15T10:15:00+00:00

Much has been written about the societal impact of AI but there has been far less penned about its creative potential.

This blog post will focus on an AI experiment conducted in support of the BBC’s ‘Beyond Fake News’ season.

Our experiment took inspiration from this viral ‘Fake Obama’ clip produced at the University of Washington. Researchers used AI to precisely model how President Obama moves his mouth when he speaks.

This external content is available at its source: https://www.youtube.com/watch?v=AmUC4m6w1wo&t=13s

This image synthesis technique is more popularly known as ‘Deepfake’. The term ‘Deepfake’ (a portmanteau of deep learning and fake) can be unhelpful and confusing as the underlying technology has potential for both creative and nefarious use. It is the malicious use of the technology that grabs our attention, often cited examples have ranged from fake news to porn.

So why is this problem important for the BBC? Video reanimation can confuse (and impress) audiences, challenge our notion of truth and has the potential to sow widespread civil discord. It’s crucial for organisations like the BBC to get under the skin of the technology by understanding what it takes to create a compelling video reanimation and researching what can be done to detect manipulated media.

For our experiment, we wanted to push the technological creative boundaries by exploring whether a presenter could appear to be seamlessly speaking several languages. To make this happen we asked BBC World News presenter, Matthew Amroliwala, to record a short 20 second script. We then asked three different presenters from the BBC World Service Hindi, Mandarin and Spanish services to record the same script but in their native languages. We deliberately picked diverse languages in order to test how effective the technology is.

For the modelling and synthesis work we partnered with London AI startup Synthesia. Before recording his 20 second piece, we asked Matthew to read a prepared script which would tease out all of his facial movements. This was used as training data for the deep learning and computer vision algorithms. A generative network (this is a network used to generate new images of a person) was then trained to produce photorealistic images of Matthew’s face which would form the basis of his new digital face.

Finally, to bring the digital face to life, the facial expression and audio track from our World Service colleagues is transferred onto the new digital face - a process called digital puppeteering.

And that’s it. Take a look at the video below and see how convincing our reanimated video is.

So, what did I conclude about our experiment? Spanish Matthew looks convincing to me. However, is there a feeling that something is not quite right when viewing the Hindi and Mandarin Matthew? Is the reanimation not quite as finessed, or is my brain so unused to seeing him speak mandarin that the suspension-of-disbelief is broken? Or is transferring non-European languages trickier technically?

But consider this: we now have a flexible digital copy of Matthew’s face. It would be possible for him to record a new video (perhaps in his kitchen) and for us to reanimate those words onto any other recording of Matthew - in the studio or reporting on location. The implications for a trusted broadcaster like the BBC are serious.

Technology is at a point where it’s possible to cheaply and quickly manipulate video and make it difficult to tell the difference from an original. We will need tools that can verify the authenticity of a video and be able to prove this to the audience.

But what mechanism would instil confidence in our audiences? We are seeing academia and technology companies working on the problem of authenticity, but there is some way to go. For now, for the audience, there needs to be a heightened awareness of this technology’s capability. Seeing isn’t always believing.

You can see Matthew Amroliwala's reaction to the technology on the BBC News Click programme.

BBC News on HTTPS

2018-07-09T12:14:00+00:00

A few weeks ago the BBC News website finished transitioning to HTTPS. The green padlock you should now see next to the web address is probably the biggest publicly visible technical change to the site since it relocated from news.bbc.co.uk in 2011. Even so, a question we’re often asked is “why did it take so long?”

Before answering that, it’s worth remembering why HTTPS (or more accurately, TLS) has come to be seen as a must-have feature for all web applications. In the early days, secure technologies such as SSL were largely the preserve of e-commerce websites. The padlock assured the user of both the site’s authenticity and the encryption of their credit card details in transit. The use of these technologies has expanded in recent years, with campaigns such as HTTPS Everywhere and Secure the News promoting the adoption of HTTPS across the board. Meanwhile, browser vendors such as Google are taking steps to identify sites that do not use HTTPS as ‘Not secure’. Clearly changes to the web landscape and user expectations mean that universal HTTPS is here to stay.

As a public service, we have to ensure that BBC News is available to the widest possible audience, regardless of device, browser or use of assistive technology. We champion the ideal of graceful degradation of service as far as possible. But in a climate of anxiety around fake news, it’s vital that users are able to determine that articles have not been tampered with and that their browsing history is private to them. HTTPS achieves both of these as it makes it far more difficult for ISPs to track which articles and videos you’re looking at or selectively suppress individual pieces of content. We've seen cases outside the UK, with some of our World Service sites where foreign governments have tried to do this.

Our plan for migrating the News website was relatively straightforward, built on extensive groundwork already done to move World Service sites (such as BBC Hindi) to HTTPS. Until recently, anyone accessing BBC News over HTTPS was redirected (‘downgraded’) to HTTP. This changed in March when we enabled access via both protocols and began an iterative process of chasing down a multitude of bugs, while we worked on updating links, feeds and metadata to reflect the new address. Colleagues in BBC bureaux around the world helped us detect access issues in different geographical areas early (we discovered, for example, that in India a government-mandated network block initially made the site totally inaccessible).

At the same time, we compared the page load performance of real users across HTTP and HTTPS, which revealed that many of those on HTTPS received a slower experience, due to the relatively large number of domains our assets are served from and the overhead of negotiating multiple TLS connections. To balance out this impact, we decided to extend the project to include some performance improvements to the site. Our final step was to reapply the redirect in the other direction, ‘upgrading’ HTTP users to HTTPS in sections (though even here we had to proceed with caution, initially making the redirect temporary in case it had to be reversed).

There were other challenges. The work had to be fitted around major events that place restrictions on our platforms, including a Royal Wedding and local elections in the UK.

Many of the bugs mentioned above fall into the class of ‘mixed content’, where the browser detects non-HTTPS assets being loaded on an otherwise secure page. This is a particular challenge for BBC News due to the site’s long and complex history, since almost every page published since the site launched in 1997 is still available. Though it appears externally to be a single website, it is really a patchwork of technical architectures, mainly because of differing requirements. Our election coverage demands real-time updates combined with scalability to cope with huge traffic levels, while one-off interactives need a flexibility and richness of experience that goes beyond our standard templates.

Over the last twenty years, publishing systems for content on News pages have come and gone, having been replaced or made obsolete. Although newer content is published through dynamic web applications that can be readily modified, what lies beneath this sometimes resembles layers of sedimentary rock. This means in practice that tracking down historical mixed content and working out how to change it is not always straightforward. We developed our own ‘crawler’ to help us find such problems, and had to come up with some crafty workarounds to address some of the most inaccessible bugs, and a number of these tasks are still in progress. We also have a major ongoing project to convert some older audiovisual content to a format that can be delivered securely, but this will take time.

Even then, some mixed content just cannot be fixed economically, and one or two errors will remain. Such pages still work, with the occasional browser warning, similar to how BBC News pages from the late 90s are still usable. We confined our efforts to content available on www.bbc.co.uk, leaving older domains as a historical record. We think users would rather we spent more of our time on building the future of the website.

BBC News is now only available over HTTPS, and the padlock (combined with the web address) hopefully gives users of the site confidence that what they read and watch was published by the BBC and is private to them. We hope you agree it was worth the wait.

Digital news trends for 2018

2018-06-21T10:24:06+00:00

Report author Nic Newman reveals this years news trends

Social media as a source for news is in decline for the first time, according to an international poll which was revealed to the BBC this week. Most of the drop was due to a growing distrust in Facebook as a news platform. Meanwhile people's trust in the news in general has stayed stable with just over half of people saying they trust the news they use themselves.

The Reuters Institute Digital News Report is an annual survey of digital news usage across the world, and this year it polled 74,000 people in 37 countries.

It found that usage of Facebook as a source of news had dropped for the first time, most noticeably in the US (down 9%) but also most other countries including the UK (down 2% to 27%).

Other trends that emerged were:

58% of those polled in the UK were concerned about fake news. This percentage is higher in other countries where there's a higher level of polarised opinion like Brazil (85%) with upcoming elections and Spain (69%) after the Catalan independence vote
There's a higher proportion of people wanting government intervention to stop fake news in Europe (60%) than in the US (41%)
Social platforms are least trusted in the UK of all countries surveyed (12%), with higher trust for "mainstream" media such as broadcast and quality newspapers
More people are using messaging apps such as Whatsapp to share news, particularly in Malaysia (54%) and Brazil (48%) however the take-up in the UK is still relatively small (5%)
There's a gradual increase in people paying for online news subscriptions - 16% up in the US, while for the UK it's 7%
Podcasts are becoming more popular (18%), particularly among younger people
The same is true for voice-activated speakers. Usage has doubled in the UK, with just under half using them to access news.

You can find more details here.

Decommission of BBC News App v2

2018-05-23T12:39:05+00:00

Three years ago, we launched a new BBC News app in the UK and internationally which gives you a more personalised news experience on mobiles and tablets.

Behind the scenes, the new app was one element of a technical project to move BBC News services onto a cloud infrastructure, as part of our effort to deliver the best value for money.

We have been running the old app for Kindle Fire devices in parallel in the Amazon Kindle Store, but now, because the technology on which that old app is based is part of a legacy system, we are no longer able to support it.

The BBC is retiring this technology, which means the old app will no longer be supported, from the 25th of May you will no longer be able to download the BBC News app from the Amazon Kindle Store.

We are also ending support for the older version of the app across iOS and Android. In the following weeks any existing installations on these devices will stop updating with new content.

For those of you that haven’t downloaded the latest app, it’s available for Apple devices running iOS 9.0 and above and Android devices running 4.1 and above. In the UK, it can be downloaded from the App Store and Play Store respectively.

Since we launched the new app in 2015, we have continued to develop it - introducing notification features, building experiences for vertical video - optimised for watching on your mobile - and providing quicker access to interactive content and long-read articles.

The app also provides better access to a broader range of content than the previous version, including local news. It has lists of most read and most viewed content across BBC News and improved photo galleries and video playback – letting us surface much more of BBC News’ video content. For the first time, the app can also be personalised – with a ‘My News’ section, highlighting news topics and stories you want to follow.

If your device is not supported and you can’t download the app, please go to our responsive site.

Outside the UK, you can download the new app from the Play Store and App Store , or go to our responsive site.

New service for business market data

2018-04-17T12:54:48+00:00

In recent days we have launched our new market data service under the Business section of BBC News Online. There are multiple reasons for making this change, not least the need to provide an improved user experience that could be accessed on all devices and browsers and by users with specific accessibility needs.

When BBC News Online was transitioned to a responsively-designed website back in 2012, market data pages were sadly left behind meaning that the service provided a very poor user experience on anything other than a desktop device, and was very difficult for the BBC to maintain or improve due to an outdated codebase. In addition, the previous market data pages were hosted on servers and by a data centre that were fast becoming defunct and in dire need of replacement; indeed, we were experiencing regular outages causing loss of service to our audience that we were unable to address in a timely fashion due to the ageing of the market data hardware and software.

Finally, BBC News is a news organisation and not a data provider, yet our old market data pages provided no news context for the data movements on display. As such, it was imperative that we build a new market data service that is hosted on the Cloud, is responsive to users’ devices, and does a better job of providing our general public audiences in both the UK and abroad with key information on market movements and the news behind them.

In making such changes we have consolidated the number of market instruments that we offer data on in order to bring audiences an editorially-led experience that focuses on those entities that are newsworthy and drive the markets. In short, this means our broad audience now benefits from an improved user experience albeit covering a scaled-back amount of market data and related pages. This simplified offering ensures market data is explained through our editorial, and our editorial is illustrated by our data.

The new market data section ensures our audience can access market data information on all devices, in a fully responsive design, with interactive price charts across a range of time periods, and supported by BBC Business editorial. As part of the project to refresh our market data service, we have launched new pages on more than 500 popular company shares, and provided our Business editorial team with the ability to create new company share price pages in minutes as the news agenda demands.

Accompanying the share price data and company news are 15 major global index pages covering the UK, Europe, US and Asian markets, five commodities pages providing information and news on global oil, gas and gold prices, and pages on four of the world’s main currencies, namely pound sterling, the euro, the US dollar and the Japanese yen. A quick overview of all this data and the day’s upward or downward trends is provided on an overview page.

Clicking on any item in the overview table, or searching for a share price in the company search box, presents the user with a dedicated page to that company, currency, commodity or market that features a detailed chart covering data movements over the current day (intraday pricing), the past month, the past three months, past year and past five years for comparison purposes.

Having launched the first iteration of this new market data service, we are now building a Company A-Z page, investigating ways to make the interactive nature of our new pages more obvious, considering the introduction of top share price risers and fallers as well as sector listings, and working on a new editorial tool that will enable our journalists to more easily embed a range of market data within stories.

A developer's guide to BBC News on Android

2018-04-09T12:43:32+00:00

If you’re living in the UK - and for many of you around the world - you’ll probably recognise the staccato chime and burst of drums from your mobile, telling you that something important has happened in the world. Natural disasters, election results, the death of infamous celebrities, and royal nuptials; all these gifts of knowledge are delivered to you by the BBC News mobile app.

And, you won’t be alone. The News app has over 9 million users each week; that’s roughly equal to the population of London. The Android app, which we’ll put in the spotlight in this article, is available in a number of flavours, all built from a single codebase. Outside of the UK we offer an ad-supported version for global English-speaking audiences, as well as five other language variants: Arabic, Hindi, Mundo (for Spanish speakers in South America), Russian, and Welsh. Our users are loyal, too, with a third of visitors using the app daily.

We’re responsible for keeping people informed about what’s going on in the world around them, so our app has to be stable and reliable. It needs to be simple to use and accessible to all. But we also try to present our content in creative and engaging ways, which means taking advantage of new libraries and APIs as they become available.

Keeping things fresh

We release updates to the app each month, something we’d like to make even more frequent! This, together with A/B testing, allows us to evolve the user experience incrementally. Change is still controversial, but we aim to avoid alterations that are too wide-ranging - the sort of thing that’s provoked a slightly negative reaction in the past:

We truly value all user feedback, and are flattered that to some the app has reached points where it’s impossible to improve. But we can!

We’re committed to supporting the latest the platform has to offer, and try to target new Android versions as soon as they’re out of beta. This isn’t always a painless process. As an example: the Oreo update exposed issues with our use of content providers, and the new notification channel interaction model continues to present us with challenges. But this commitment to staying up-to-date allows us to deliver features like Launcher Shortcuts soon after release, as well as taking advantage of performance and stability improvements.

As developers, we try to keep our coding practices current, too.

Writing great code

Like many other Android teams, we find a lot that’s useful in the basic Android development environment, but think some parts can be improved. Hence we augment the standard toolkit with a range of open source libraries.

We make heavy use of RxJava in our asynchronous code. This gives numerous benefits when compared to standard Android patterns like AsyncTask. Chains of functional operators allow us to transform data with fewer side effects. Multi-threading is easy to implement, and straightforward to stub in unit tests. And errors can be properly handled. For network requests we combine RxJava with OkHttp to retrieve and cache data in a flexible, robust and performant way.

A complex app means complex object hierarchies to manage. Here the dependency injection framework Dagger (well, Dagger 2) helps us out. Components are either annotated directly or referenced in configuration classes, and Dagger does the hard work of generating the boilerplate code to instantiate objects. It also can provide and maintain singleton instances of classes, allowing shared state without the testability issues of a classic approach.

We’ve also introduced Kotlin to the project. New modules that don’t incorporate legacy code are being implemented entirely in Kotlin; older modules use a pragmatic mix. Even in areas using Java heavily we’re seeing benefits; for example, using data classes to replace code that would previously have been generated using annotation-based libraries (in our case, AutoValue).

However, third party libraries don’t just allow us to write better code. They can also allow us to deliver user-facing features in a richer and more efficient way.

Better features, built faster

A great example of this is the approach we’re taking to add motion to the app. While the Android animation API has improved hugely over time, adding movement to buttons and other parts of the user interface still involves significant effort and understanding.

Here we make use of Lottie. Created by Airbnb, Lottie renders After Effects files as native animations that are included as easily as traditional static assets, and can even be hosted remotely. This allows the designers - an integral part of our team - to create complex transitions between UI states using software they’re familiar with. The exported animation is then simply referenced by the developer in an instance of LottieAnimationView. This supports animator-like interactions, so the ability to start, pause, resume and cancel. It also lets us observe state changes using AnimatorListener and AnimatorUpdateListener.

Hence we can add animations quickly and precisely, and in a way that allows changes to be made easily in the future.

Onwards and upwards

Our lives as developers are not, however, without challenges. The codebase of the current app began life in 2013, and was first released in 2015. To put that in perspective, your phone at the time probably looked something like one of these:

The Android build plugin at the time lacked unit testing support, multi-module builds were slow and required a lot of bespoke configuration, and singletons were fashionably cool. Well, maybe not cool - but they were certainly heavily used! As a result the current team inherited a monolithic codebase with minimal unit tests and a fairly loose architecture.

As discussed earlier, we’ve introduced techniques and components into the project that weren’t available during the incubation of the original codebase. Singletons have been replaced with injected dependencies, and we’ve made a lot of progress towards a cleaner codebase that’s easier to test and more resilient to change.

We’re now embarking on a new phase. The project is being restructured based on clean architecture principles with clear layers of responsibility. Distinct software functions - so things like data retrieval, business logic processing, or presentation - are finding new homes in separate decoupled modules.

Architecture is hard

So why are we doing this?

A big motivator is the ability to scale. As consumption has shifted from desktop to mobile, so the size and needs of our user base have grown. This creates a perfect storm: app complexity, team size, and the number of new features required have all increased - and so too has the risk associated with making changes across a monolithic, tightly coupled codebase.

A modular approach allows change to happen in isolation, with behaviour changes limited by strong contracts between components. Functionality can be tested with much less complexity: stub implementations replace real dependencies, with no need to worry about their inner workings.

Modularisation has other benefits, too. We’re able to create our different app flavours in a more flexible way; optional modules - maybe even built by a different team - can be included in a particular language variant through composition. Build time is reduced through improved incremental compilation. And it’s consistent with the approach required to create Instant Apps, something we’re excited about trying out in the future.

And it’s goodnight from me…

So that’s a brief look at the way we do News on Android at the BBC. We’ve explored the way we write code, and the challenges we meet along the way. But it’s worth returning to the why: our users. Meeting the needs of a diverse audience can be tough, but will always be outweighed by the reward of delivering trusted news to a user base of millions around the world.

Plus, we get to work with some great people. Even the ones with Apple phones!

Wish you were here?

If you’re an Android Developer, and all this sounds like an interesting challenge - come and check out our available roles! We do percent time, go to conferences, and get lots of training, so there’s plenty of opportunity for growth. There’s also often cake!

The interactive game training the next generation of news reporters

2018-03-29T10:10:08+00:00

BBC News, BBC Design + Engineering and Aardman Animations have teamed up to produce an interactive game, giving children a taste of newsroom decisions and pressures. Senior product manager Andrew Leimdorfer explains how it's evolved.

The iReporter game aims to give players the chance to take on the role of a journalist in the BBC newsroom. The game was commissioned by the BBC News School Report team, who's mission is to help 11-18 year olds develop their media literacy skills, and is part of a set of resources to use in classrooms across the UK.

The "choose your own adventure" game challenges you to make your own decisions on which sources, political claims, social media comments and pictures should be trusted as you contribute to the day's news output.

The game features video interviews in social media type environments

The idea to use the vehicle of a game to develop these skills first surfaced in Autumn 2017 and, after a competitive tender process, Aardman Animatons were selected as the right partner for the project. Although the basic structure of the game and the key learning points were defined from the outset, the script and gameplay were developed throughout the creative process and user-tested with groups from schools in Bristol and Coventry. The feedback from the youngsters was then incorporated into the design.

Players are presented with a series of editorial choices

While setting out to deliver key editorial learning points, such as the importance of thinking twice before sharing, working out the motives of your sources and spotting obvious fakes and mistakes, the brief was to keep the game engaging and fun for the target audience. The use of social media style interfaces as the main way of delivering the content helped ensure the interactions felt familiar. Aardman also did a great job to keep humour in the story, while delivering the educational points.

Players are under pressure to meet different deadlines

We know from feedback on School Report Day that the initial response has been great. It takes around 20 minutes to complete the game, which is a significant commitment for online content, but we're already seeing high levels of engagement and significant numbers of users playing through the whole game.

However, the success of the project will be judged over the longer term as schools across the UK use the game in the classroom and new cohorts of School Reporters are educated about the pitfalls of fake news, but also inspired by the fast-pace challenge of working in a newsroom.

Shifting gear with the World Service

2018-03-26T15:16:48+00:00

Just over a year ago I took this photo of Fran Unsworth and Jonathan Chapman standing in in the middle of an empty concrete shell of a office floor in Nairobi where we had just signed a lease for our new bureau in Kenya. It was exciting to think of what could be done with the space, albeit with a daunting level of work needed in Kenya, Nigeria and India to realise the potential of the investment in the World Service through the World 2020 programme.

Our new Nairobi bureau as it was this time last year

A year of building

A year on, thanks to the hard work of multiple teams across the BBC, we have three brand new bureaus in Delhi, Lagos and Nairobi staffed with hundreds of new journalists broadcasting and publishing in 12 new languages across Africa and Asia. We have also updated or provisioned new facilities in another 50 sites overseas as part of the 2020 expansion programme. It means the BBC is now broadcasting and publishing in over 40 languages ranging from Pidgin to Korean and our new bureau in Lagos will be launched next week.

I’ve been fortunate to visit three of our new sites in the last 12 months and meet some of the incredible people running those new services. These are really diverse teams, as we publish in 3 or 4 different regional languages from each office.

Part of the Delhi editorial team

In Delhi I watched a brand new team of journalists make news bulletins in Hindi, Telugu, and Tamil one after another; something they do every night. An amazingly complex task, especially as the teams had only been working together at the BBC for 6 months.

The News bulletin production team in Delhi working in Hindi, Telugu, and Tamil

There are very few organisations anywhere in the world capable of mobilising this level of broadcast and digital infrastructure, let alone the scale of quality journalism in so many different languages. It is one of the things that makes working in technology in the BBC uniquely interesting and rewarding.

Whilst we’re proud of what we’ve achieved this year, we’re only just getting started.

Although we’ve launched the 12 new websites, created countless programmes in 3 new major bureaus, and hired of hundreds of journalists, we’ve only just begun to transform how the World Service works. The bulk of our audience remains on TV and Radio; to be sustainable we have to deliver engaging digital services that local audiences find relevant and valuable to their lives.

Adjusting our approach

We have a big goal to achieve: the BBC as a whole is aim to reach 500M weekly users by 2022, and World Service expansion is one of the drivers of growth that we hope will drive an additional 80M users. To get there we’re changing our approach to building digital products and ways of working.

The launch of 13 new languages sites were the starting point of our digital plans for the World Service

In the last year, we had to constantly manage risk to achieve the deadlines we’ve hit. To meet tight delivery schedules for the new websites and building fit out, we often had to reign in the scope of our ambition for our digital products at initial launch. Similarly, we had to be 100% sure that the studios and production systems would work from day one, and so we deployed existing combinations of production tools, albeit now virtualised and IP capable, rather than experimenting with completely new technologies.

We’re now turning our focus to a more agile, data and performance driven phase of delivery.

In the near term that will focus on speed of digital performance and user engagement:

our content doesn’t load quick enough on mobile; we want to ensure all our sites load as quickly as possible in all our WS markets, even on slower mobile connections.
we also have a lot of users who come to us via search and social, read one article and then leave again. We’re experimenting with machine learning driven recommendations to increase the number of users who click on one more thing to read or watch.

World Service Dev Team together with some of the developers from our partner Andela

In parallel with the core delivery our News Labs team have been able to experiment with new tools that leverage the latest in languages technology. One of those tools, called “Stitch”, started as a rapid prototype created in a weekend by a tech savvy journalist. The News Labs team scaled it rapidly to a point where it is used to create over a 1,000 videos a week. It’s helped us publish 4 times as much digital video a week as we did before the World 2020 programme started.

With our journalists, we’ll continue rolling out improvements to the tools and dashboards they use; letting them to connect better with each other, so they can better connect with audiences.

We’ll start blogging soon about some of the new prototypes of technology we’re using to transform how we plan, gather, edit and publish the news across the World Service and the rest of BBC News.

If you are interested to join the work we are doing to expand and tranform the World Service do get in touch. We are looking for fantastic front end and frameworks developers and data scientists to join the team.

Digital Creativity team trains photojournalists for School Report

2018-03-20T12:57:48+00:00

BBC D+E’s Digital Creativity team helped transform youngsters from four schools into photojournalists for the day last week.

It was all part of the BBC’s School Report News Day that involves 30,000 school children and 900 schools around the country. Around 130 students and 30 teaching staff from schools across the north came into Media City for workshops to enhance their digital and journalistic skills.

The D+E team collaborated with the BBC Academy to run a series of workshops that ended with the youngsters taking their own photographs around the Salford base and then publishing them on to the BBC’s creativity platform Mixital.

The theme of the day was accuracy and authenticity in news reporting – two values that are crucial to the BBC. The purpose of the workshop was how to apply the same values to tell stories accurately and fairly with photos.

Academy photographer Danielle Baguley explained some of the principles of photography by taking the groups through her portfolio that included photos from sporting events to Will.i.am. The youngsters were introduced to the principles of photojournalism and how to tell a story with pictures.

At the end of the workshops the youngsters were set the challenge of gathering their own photos around Media City to tell a story of urban regeneration. Back in the newsroom, the youngsters edited their photos, selected the best and wrote captions. They then published them on Mixital and you can see them here.

24 hour Diversity Hackathon highlights

2018-02-05T14:51:27+00:00

Last month we reported on the 24 hour Diversity Hackathon, involving 100+ staff from BBC News and BBC Design & Engineering. You can find out about it here and read Head of Data Science and Architecture Gabriel Straub's blog, but now we can also bring you the video highlights below. Enjoy.

Can data help journalists better represent the variety of voices and interests across the UK?

2018-01-24T18:34:35+00:00

Gabriel Straub from BBC Design and Engineering's Technology, Strategy and Architecture team, lists all the ingredients that went into last week's 24 hour Hackathon.

“Go hack” went the cry, and with that staff from across the BBC began the first 24 hour Diversity Hackathon, which set out to explore ways that data can support BBC News journalists to make content that better represents the diverse range of interests and voices in the UK.

At 10.00am on Thursday 18th January, over 100 BBC staff from Design & Engineering, News, Monitoring and R&D gathered in the BBC’s central London headquarters to voluntarily run this first BBC 24 hour coding event.

The Hackathon Brief

The aim was to build prototypes that could support:

• Our Editorial Choices – how can we better discover and report those important topics that are not part of the continuous cycle of breaking news and the more traditional news stories and sources?

• Our Coverage – how can we better understand what ‘underserved’ audience groups want from a story?

Fiona Campbell, Controller BBC News Mobile & Online, sets out the challenge

The Hack Teams

The BBC staff volunteers were grouped into 14 diverse teams comprising journalists, engineers, architects, data scientists, graduate trainees, audience researchers and marketing professionals, and the response was fantastic. Our ambition from the outset was to break down the silos between departments, cement new friendships across the organisation, encourage people with different skills, from different parts of the BBC to all come together successfully as a team…and we did it!

And this is how….

24 hours to go

10:00 Introduction by Jatin Aythora, BBC Chief Architect, TS&A followed by Problem Presentation by Fiona Campbell, Controller of BBC News Mobile & Online

10:45 Hacking starts and continues for next 24 hours!

14:00 Lunchtime check in and first vote. First 3 minute pitch presentations from the teams

22:00 Late Dinner check in and second vote. Second 3 minute pitch presentations from the teams

08:00 Breakfast check in and vote. Final 5 announced!

10: 00 Demos and judging panel. Winner is announced.

Throughout the 24 hours, to give people a break from a hard day (and night) of work, we organised some inspiring entertainment, as well as an open bar and food round the clock. Among the highlights were a workshop with steel band Ebony to learn how to play the steelpan, a chance to pitch your wits against renowned computer scientist and academic Dr Sue Black who hosted a quiz, a variety of stand-up comedians and musicians, yoga classes to stretch out and relax, table tennis, fussball and Jenga for the more energetic, and large squashy bean bags for those needing a power nap.

Late night yoga to keep teams going

A late night lesson in steelpan

The event ended with the demos and judging session hosted by former Tomorrow’s World presenter Maggie Philbin. The judging panel of senior editorial and technology staff from BBC Design & Engineering and BBC News, were impressed by the quality of entries. I was impressed by the sustained enthusiasm, creativity and frenzied coding amongst all the teams – some even took to the streets to test out their idea on the public!

Watch this space for how we plan to take their ideas to the next stage. But one thing that’s certain – as a result of the fantastic feedback we’ve received both from the judges and participants, we’ll be holding more Hackathons in the future.

The Hackathon was not just about demonstrating how technology can help open new paths into understanding the diverse interests, opinions and perspectives of our audiences - as a data scientist I would say that - but, for me it was also about showing how working and innovating with people across the organisation can produce amazing ideas which could help transform the way we serve our audiences.

The event in code…

• 102 participants
• 14 teams
• 7 judges
• 2 event spaces
• 1,171 emails about logistics!
• 16 weeks of planning
• 12 oversized beanbags
• 9 inspiring entertainment acts
• 0 hours of sleep (for some)
• 40 bags of assorted sweets
• 24 hours
• = 1 successful Hackathon!

The final word must go to our BBC sponsors across Design & Engineering departments, News and Monitoring without whose amazing support this 24hr Diversity Hackathon could not have gone ahead. Thank you!