Earlier this week, I posted a brief history of the BBC Blogs Network and provided some insight into the technical dilemmas we've faced since launch 18 months ago. Today's post looks at the editorial challenges.

I don't believe that every BBC television or radio programme or personality should have a blog, nor do I know of anyone within the BBC who would want this, yet hardly a week goes by without at least one request for a new blog.

There's a process for the proposal of new blogs which, currently, doesn't capture whether or not our would be bloggers have what the Guardian's Emily Bell calls one's "inner blogger". Without an author who has a "drive to blog", a blog that looks great on paper ends up lacking in substance and appears soulless.

Many people don't have an inner blogger and not every BBC presenter, reporter, producer or editor wants to blog. Indeed, there are some who think the BBC shouldn't be blogging at all. For example, earlier this year, the BBC's Senior News Correspondent Kate Adie said to a blogger:

"... journalists shouldn't have any time to blog - there are too many stories waiting to be told!"

Adie has a point - blogging does take time - but I disagree that it necessarily takes time away from more fruitful journalistic and production activities. In fact, I'd argue that, where blogging is an integral part of the process of journalism or production, it's quite beneficial to both the final product and audience's understanding of it.

• Allow us to join in conversations about the topics we cover and programmes we make
• Bring some of the BBC's best and most widely recognised talent closer to their audiences
• Make it easier for journalists and programme makers to gain exposure to and learn from the knowledge and experiences of our audiences
• Make our editorial decisions and policies, as well as our production values and techniques, more transparent and those who make them more accountable

These remain the core reasons why the BBC Blog Network exists and, because our blogs each tend to have a different style, voice and audience, they each tend to focus more or less strongly on each of the goals above.

For example, Radio 2 Presenter Chris Evans' blog is much like a personal diary, bringing audiences closer to the man behind the microphone. As one blogger said of Evans' blog:

"He actually comes over as being quite like how he is on TV, which is to say that he seems to be a happy-go-lucky, decent bloke. All in all, the kind of chap you wouldn't mind spending some time in the company of."

Newsnight, Radio 5 Live's Pods and Blogs, and the new Radio 4 programme iPM are all using their blogs in a different way - to engage with and tap into the knowledge and creativity of their audiences.

The Editors, Sport Editors and this, the BBC Internet blog, are all about bringing transparency and accountability to the services provided by different divisions of the BBC.

There have also been many less obvious, but no less important, more general successes - some of which have brought concrete changes to the content and services we provide online and on-air. For example:

• The first use on bbc.co.uk of embedded videos from youtube and other video sharing websites took place on the blogs
• The BBC's first experiments of one-click links to add pages to social bookmarking and recommendation services was on our blogs, with all BBC News and BBC Sport pages now carrying such links
• We first started experimenting with the use of third party photo sharing sites such as Flickr as a way to ingest and manage audience photo submissions on the blogs, a technique now used more widely

I've bookmarked many other examples, as well as bloggers' reaction to some of what we've been doing, here.

In addition to having recently commissioned a technical review of the infrastructure behind the BBC Blogs Network, something I wrote about on Monday, many people around the BBC have recently been taking stock of the blogs they're responsible for and thinking about the future for them.

People do, of course, define "blog" in many different ways, but most bloggers I know tend to see their blog not just as a publishing tool but as a tool enables a one-to-one or many-to-many conversation. One way to do this is via the comment facility at the bottom of each blog post.

Steve Herrmann, Editor of the BBC News Website, is enthusiastic about some of the exciting things he's seen on the BBC News Blogs, but admits that:

"Responding to comments consistently across the blogs continues to be one of the biggest challenges for all concerned."

Another way to engage in the conversation is to seek out and link to the thoughts and content of other bloggers - and here, except for in a few examples, we're doing pretty poorly. The preliminary results of the technical review being conducted by Headshift has revealed that only about 1 in 8 posts on a BBC Blog link out. Based on the many conversations I've had with the BBC's bloggers, only a handful utilise tools like Technorati to proactively track and engage in conversations their blog could be a part of.

So we're pretty good at using blogs as a publishing platform. But we're not doing as well as we could engaging with the conversations our blogs could, and in many instances should, be a part of.

We also have some difficult challenges ahead. We need to refresh our current technical infrastructure, and improve the way we deal with comment spam and engage with legitimate comments. We need to ensure that ideas for good blogs make it through whatever editorial approval process is required whilst keeping the bad ideas from seeing the light of day. We need to close some of our less well looked after existing blogs, the ones which don't benefit from the nurturing hand of someone with an "inner blogger".

In the coming weeks I expect to be able to return with some more insight into the technical review and resulting recommendations I've alluded to in this post.

We've been canvassing opinion (with the unexpected affect that one of our internal sessions affected the output of a flagship BBC News programme) both inside and outside the BBC. Now it's time to hear from you.

What sort of blogs do you think the BBC should, or shouldn't, do in the future? What process do you think should be applied in making those decisions? How might we improve the way we use our blogs not just to publish content but to participate in the wider conversations taking place around BBC content and the topics and stories we cover?

I look forward to your thoughts...

Robin Hamann is a senior community producer.

The BBC Blogs Network has been up and running for 18 months. This milestone provides a good opportunity to give you some insight into what we've been doing, how we think our efforts measure up, and where we might be headed in the future, editorially and technically.

That's a lot to cover, so we're splitting the discussion in two. Today's post deals with the technical aspects of our Blogs Network and later this week, I'll return with a second post looking at editorial challenges.
So, first: some history. In early 2006, following several forays into blogging by the BBC, most notably BBC Scotland's Island Blogging, Ouch! and Nick Robinson's Newslog, the decision was made to customise and install an off-the-shelf blogging solution and create the BBC Blogs Network, which launched in April.

The graph below provides four snapshots, taken at six month intervals, of the unique visitors and visitor sessions for the BBC Blogs Network, starting with that first month and ending with October 2007.

The graph illustrates that the BBC blogs have, in aggregate, found a large and steadily growing audience. Additionally, it's worth noting that the average time of each visit has remained fairly static at just under four minutes.

The downside of this success has been our technical infrastructure becoming increasingly unstable. It was built quickly and involved a number of customisations to the software which effectively ended our ability to easily install software patches and bug fixes provided by the vendor in order to deal with some of the technical issues we've faced.

Our technical woes will have been invisible to most people visiting a BBC blog, but this will be of little comfort to those who have encountered them first-hand, including Newsnight editor Peter Barron who recently posted on The Editors: "Often I try to respond to a comment or complaint about the programme and end up gnawing my knuckles in frustration as the response either doesn't appear for many hours or fails to materialise at all. Hardly the best way to have a free flowing dialogue with our viewers."

Eddie Mair wrote a post at 7am on Saturday, the launch day of his new programme iPM: "I do want to say what a bang-up job the team have done. As you'll see from the iPM Blog, they have done a ton of great work - I find it really rewarding to have a rummage round the blog." Later that morning, he updated the post to say: "By the way we are aware that the 502 error messages are back. Great day for it to happen, no?"

We are aware of the problems and of how frustrating they are for anyone who takes the time and effort to compose a thoughtful comment only to see that effort result in a 502 error or an automatically generated message from the server saying, wrongly, that they've been banned. Here's where we're at in our efforts to diagnose and resolve them.

The early results of a full technical review we've commissioned from an outside agency - as well as our own investigations - indicate that these technical problems are almost certainly down to the bottleneck that exists between our three front-end web servers and the database server. The database is what ingests, stores, and allows the management (including moderation) of comments and then outputs those comments and the other data that make up the web pages you look at.

The graph below [click for large] shows the percentage of CPU usage on our blogs database server over a week. Clearly visible are the spikes, sometimes reaching as high as 100%. It's during these spikes that you are most likely to encounter problems commenting.

The primary cause of these spikes is spam attack. Worryingly, the amount of spam comments submitted is on the increase and, across the network, we can now get as many as 50,000 spam comments per week. In one particularly bad day last week, the comment cgi was hit well over 30,000 times.

Each time a spam comment is submitted, usually by a bot, the database CPU crunches the data. We're aware that the version of the application we use is less efficient in dealing with spam than newer versions but, as I mentioned above, customisations we made to the application have effectively ended our upgrade path, so we've been unable to take advantage of upgrades since made available by the vendor.

That's the bad news. The good news is that our new(ish) applications engineer, David, has been busy firefighting and implementing various anti-spam measures. We've also commissioned Headshift to conduct a full technical review of our blogging platform, the results of which will be delivered on November 19th. I don't want to pre-empt that report but I can say that, in addition to reviewing our existing platform, the report will also present us with a number of suggestions for improving our technical infrastructure and these are likely to include a mix of software and hardware upgrades or replacements.

My apologies if you've encountered problems when using the BBC's blogs. When the review is delivered next week, we'll begin the process of planning and implementing a robust solution over the following weeks and months. In the meantime, we'll carry on with our daily fire-fighting and will continue to provide as good a service as we can - although there are bound to be a few problems from time to time.

I'd be glad for any thoughts you might have and hope to see you on Wednesday for Part Two.

Robin Hamman is a Senior Community Producer.