Frequently asked questions

New for 2012
The FT Bowen Craggs Index now covers 81 very large web estates, with associated social media channels, mobile and apps. We look at the entire presence, covering non-English content where relevant. Chinese, Japanese and Russian speaking associates are used to check relevant areas.

It is the most detailed mass study of large websites produced, and has three aims. First, to help organisations know where to look for ideas. Second, to pick out trends in online comms. Finally, to allow the companies in the Index – broadly, the largest quoted companies in the world – to see how they are performing against their peers.

The Index is unlike other rankings for two reasons. First it takes a broad approach. Rather than concentrating on a particular element, it looks at the jobs a web presence is asked to do, and sees how well it does them – this is what we call effectiveness. Second, our approach is ‘expert’ rather than ‘check box’: our analysts – experienced in both business and the web – make judgements at every stage.

Main changes this year:

• Explicitly analysing mobile websites and corporate apps.
• Upgrading accessibility testing¹

What is the Index?
The Index is a ranking – in fact it is many rankings – but the real aim is not to stimulate praise, blame or panic, but to show what should be done (and what should not be done) to make a website as effective as possible. It is not designed to say ‘these are the best corporate websites’ – rather that this is how this group performs. But it does include the largest companies in the world and the sample is big enough to give a good idea of trends. It also allows people responsible for corporate (or, indeed, non-corporate) sites to see which way others are going, to spot ideas – and, if they wish, to hijack some of them.

How are the companies selected?
Companies included are taken from the FT Global 500, ranked by market capitalisation. We started with the top 25 from each of the US, Europe, and the rest of the world, and added in the six companies from the top half of last year’s Index that would no longer qualify. The only company excluded is Berkshire Hathaway, which chooses not to use the web in a conventional way. Our aim is not to say ‘these are the best websites’, but to provide a detailed picture of the world of corporate websites with a mass of best practice – our database keeps reviews from previous years, so the resource will get ever more substantial.

What’s the timetable for reviews?
We start towards the end of the year, but do the great bulk of the work from January to March. We ask companies in advance if they are planning updates, and hold back reviews to get the most up-to-date analysis possible. Our database is updated on a rolling basis, but there still will be once-a-year publication to show the state of the corporate web.

Who does the reviews?
We have eight people working on the Index, either full- or part-time. They all have a background in the web and business, and most have being working on the Index since it started, in 2007. Our analysts are all experts on the corporate web and are trained to ask first, what should be provided, and only then to judge how well it is provided. ‘Appropriate ‘is the key word. We look at the entire presence, covering non-English content where relevant. Chinese, Japanese and Russian speaking associates are used to check relevant areas.

What’s the methodology?
Our starting point is that the Index has to be credible – for us that means a checkbox approach cannot be used, because companies vary so much in what they can and should be doing online.

Bowen Craggs has developed a methodology to benchmark corporate sites in great detail. The FT Bowen Craggs Index uses the same methodology, but at a lighter level. The metrics are divided into two groups: overall and specific. Specific metrics concentrate on how well the site serves different groups. Within the overall metrics, Construction covers navigation and coherence, Message looks at the visual and content messages the site transmits, while Contact covers both ease of making contact and use of FAQs.

Some of the questions we ask are broad (Is the look and feel appropriate? Does the site tackle relevant controversial issues?) – here analysts have to make a judgement based on their knowledge of the company, industry and the web. Others are quite specific (How well are CSR management and measuring systems explained? What format are provided for quarterly results?).

Is this an objective exercise?

No. We always consider the profile and needs of the companies being examined: in that sense our benchmarking is not objective, nor should it be. For the same reason, it is not a ‘tickbox’ exercise: every metric is judged by its relevance and its utility to the client, rather than ‘Is it there or is it not?’. The key questions are much more likely to be ‘How appropriate is it?’ and ‘How well is it done?’.

Are the scores strictly comparable from year to year?
Almost, but not quite. We adjust the questions to take account of changes in the world of the web. The big change this year is that we have woven several proxy social media-related questions into the metrics. The aim is to see how well companies are integrating the web and social media (for example, Is there social media integration in the home page?) and also whether they are making active use of it (Does the company use social media for case studies, reputation management, conversation?).

How long do the reviews take?
A new review takes 16-20 hours on average and is carefully documented. For the 81 companies there is a total of 800,000 words of analysis in our database to support the score charts. Where a site has been analysed previously, we go through it carefully for changes.

Don’t you have conflicts of interest?
We have carried out consultancy and benchmarking work for almost a quarter of the companies in the Index. Most do well. This is not because we favour them, but because they are also the companies that take the web most seriously, so are more likely to employ us. We separate consultancy from benchmarking, but, more important, we could not possibly mark clients up (or down). Even if we wanted to, we would soon be found out – especially now that all our underlying research is available on subscription in our database. We are also aware that web managers do not like to be over-scored – they know if a score feels right, and want a credible judgement they can support internally.

My company or organisation is not in the Index: how would it score?
Whether your company features in the Index or not you can order your own Index report or subscribe to our database.*

¹ The methodology has been developed by Peter Abrahams, accessibility practice leader at Bloor Research, an IT research company, and founding member of the OneVoice for Accessible ICT Coalition