Museum of London and social software: research methodology for analysing effectiveness of Museum blog
November 5, 2009 Blogs, Social media, WebsitesThere are a number of ways in which I gathered information and analysed the data to evaluate the effectiveness of social software in increasing visits to Museum of London and the Museum’s main website. I used a combination of quantitative and qualitative data, including drawing upon my own experience as Web Content Manager working on these websites.
In today’s post, I will explain some of the methodology I used to answer the set of questions I identified at the start of my research to help me measure the effectiveness of the Museum of London blog (this site referred to as MyMOL) on the Museum of London website. These questions can be seen in my first blog post on this subject.
Primary research method for analysing effectiveness of Museum blog (MyMOL)
First method: use of web analytics
I used the website logs to analyse and compare the patterns in visit and visitor numbers of the Museum of London website with that of MyMOL. The web statistics gathered, analysed and compared were for the duration of one year (1 April 2008 to 31 March 2009). This period of time allowed for patterns to emerge, and correlations between the sites to be discerned, if any.
The web statistics gathered for each site were the:
- number of unique visits,
- number of unique page views,
- number of unique visitors,
- number of repeat visitors,
- number of new visitors,
- average duration of visits, and
- number of referrals made between sites
(Terminologies above will be described in greater details in the research findings.)
To ensure website statistics gathered were comparative for the Museum of London website and MyMOL, I used statistics from Google Analytics for both.
The percentage of new and repeat visitors and the duration of visits were used to demonstrate how engaged visitors were with each site and its content, which also provided an insight into how valuable visitors found the information (though these values have limitations – see ‘Recognised benefits and limitations of web analytics’ below for more on this).
To help answer the question of whether MyMOL was attracting new audience and encouraging more visits and engagement with the Museum of London website, or whether it was only taking visitors away from it, I looked at any referrals that resulted in visits made from the Museum of London website to MyMOL, and from MyMOL to the Museum of London website, for the same period of time.
Second method: Museum of London website emails and MyMOL blog comments
The second research method I used for finding out how engaged visitors were with the Museum of London website and MyMOL, was to compare the number of enquiries that come through the Museum of London website with the number of comments made on MyMOL blog entries for the duration of the year.
As it was difficult to track every single enquiry that came via the website due to the fact that email addresses are explicitly published on all Museum of London websites and offline publications, I only counted enquiries submitted to the info@museumoflondon.org.uk email, which is the generic email address for all web enquiries, including emails via the contact forms on Museum of London website.
I calculated the number of enquiries received with number of visits made to Museum of London website to arrive at an average number of enquiries per visit, per month.
To compare this with MyMOL, I looked at the number of comments made to blog entries and calculated the average number of comments per visit, per month.
This indicated how engaged visitors were with the sites, both individually, as well as in comparison with each other.
Recognised benefits and limitations of web analytics
Web Analytics is the statistical measure of a visitor’s journey through websites and is described as “the measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage” (Web Analytics Association 2009).
Canadian Heritage Information Network claims that success of a website is “determined by its usage”. Web analytics helps measure this usage by enabling site managers to gather information about where visitors come from, what pages they view and how long they stay on the site. Web analytics also enable site managers to find out how many visitors are visiting the site over a period of time and if those visitors make repeat visits.
However, whilst this is true, there are problems with how the information is gathered in the logs for analysis. For example, visits and visitor numbers are not accurate. “Lots of tools use IP address to assign visitor status, but when a customer is using an ISP such as AOL they are on a dynamic IP. This means that if they come to your site today on IP ‘A’, tomorrow they might come on IP ‘B’. This would be tracked as 2 visitors each with 1 visit, even though it was really 1 visitor with 2 visits” (Kyrnin 2009). On the other hand, “some visitors may use more than one computer on a given day” and “more than one visitor may use the same computer” but it all appears “as one unique visitor” (Dash 2006). In addition, if cookies are used to track visits and visitors, the result is compromised as many people delete cookies or refuse to accept them (Kyrnin 2009).
Page views can also be misleading as people may see information on other sites instead of the original website, and so pages will not be counted. Also, the duration of visits could show up as longer then is true if visitors look at one site but keep open the browser for another site, to which they come back to after a length of time. On the other hand, if visitors view only one page, duration of time will not be logged at all.
There is also an assumption that processes for measurements “relate to an action by a human visitor”(Web Analytics Association’s Web Analytics Definitions). However, sometimes website crawlers visit websites to index pages for search engines. Some web analytic software includes these visits, whereas Google Analytics excludes them, which gives a more accurate result. However, if JavaScript is turned off on a person’s browser, Google Analytics will not be able to gather any data for any visits from that browser, thus decreasing the visitor and visit numbers, although switching off JavaScript is not a common occurrence.
Even with the recognised limitations of web analytics, other then carrying out intensive web surveys (which has limitations of its own), using the only means currently available to measure the success of a site, is still very strong. As the limitations apply to both the Museum of London website and MyMOL web statistics, and as both analysis are done using Google Analytics, I thought the results will balance against each other and as long as I am flexible and take into account these limitations, the information will still prove to be valuable.
References
Canadian Heritage Information Network. “Web Analytics ? Measuring for Success.” Canadian Heritage Information Network. 15 January 2009. Web. Accessed 21 August 2009. <http://www.chin.gc.ca/English/Digital_Content/Web_Analytics/index.html>
Dash, R.K. “What Is Web Analytics? Web Metrics?” Weblog post. Chameleon TechnoBabble. 26 April 2006. Web. Accessed 21 August 2009. <http://www.chameleonintegration.com/2006/04/26/what-is-web-analytics-web-metrics/>
Kyrnin, J. “What Web Analytics Can be Tracked: An Overview of Metrics That Can Be Tracked.” About.com. Web. Accessed 21 August 2009. <http://webdesign.about.com/od/analytics/a/what_can_track.htm>
Web Analytics Association. (a) “About Web Analytics Association.” Web Analytics Association. Web. Accessed 21 August 2009. <http://www.webanalyticsassociation.org/aboutus/>
Web Analytics Association. (b) “Web Analytics Definitions.” Web Analytics Association. 22 September. 2008. Web. Accessed 21 August 2009. <http://www.webanalyticsassociation.org/attachments/committees/5/WAA_Web_Analytics_Definitions_20080922_For_Public_Comment.pdf> p.7
The working life of Museum of London « fucktorymuseum 2.0 :
Date: November 11, 2009 @ 2:17 pm
[...] Il già citato Bilkis Mosoddik continua a presentare il suo lavoro di ricerca: l’ultimo post si intitola “Museum of London and social software: research methodology for analysing effectiveness of Mus… [...]
richard @ London Recycling :
Date: October 12, 2010 @ 8:07 pm
Bilkis, I’ve got to applaud you for this post, your research was very analytical in it’s processing of the data. However, I’m hoping that I can add some additional light on your data that you gleaned. Google’s got some further nifty tools that help to refine a website. Using tracking data such as “bounce rate”, you can see just how long users remained on a site. This data should be fed back into the website’s design so that it can be refined further- ie are users clicking out of a site within 30 seconds- in that case look at the pageload speed and keep in mind the fact that home users (your key audience) may be on a slower connection than you. If users log 1-2 pages per visit in a 45 second timeframe, look at the content of the site- is it engaging enough? Is the site easy enough to navigate? Furthermre, take a look at browser capabilities (another nifty feature)- can a large percentage of your users actually see certain things on a page? Is flash disabled (or as you pointed out Javascript). Does your site depend on javascript? It’s then worth considering such things as visitors with ie6- this may be a larger issue for home users than business users- and could hinder CSS, html, java and flash (let alone png files!). Now, this is where I start the real criticism- YOUR SITE IS SLOW!!!. Firebug reports a load time of 4.6 seconds for a 5.5 meg broadband connection, based in Cambridge, so I’m geographically local and have good download speed, but the site’s got far too much data to load. I see over 30 elements loading on a page and the web server (any server)and my high end intel core2duo macbook pro can only handle 3 requests at a time- consider consolidating your CSS files, using sprites for image files and consolidating javascript files into one file. You need to think about your visitors and how long they have to sit there between page load- it’ll reduce your bounce rate and improve your visitor retention! If your visitor numbers are high, (I see some 114,000 per month, which is quite high!), try using a content delivery network- host your images elsewhere, that way, you’ve got more than one pipe feeding the person visitng the site. However, google shows a bounce rate of 47% which isn’t too bad and visitor page views of over 4 1/2- again quite damn good. I’d suggest that the above could quite easily push page views above 5 per visit and your bounce rate closer to 40% (WHY? you’re missing out on a percentage of visitors who can’t download your content fast enough, so they just click out of your site!
Bilkis Mosoddik :
Date: October 13, 2010 @ 9:34 am
Dear Richard,
Thank you for your comment. Much of what you said makes sense and actually since my initial research, I have been looking a lot more at our websites and how they are used.
You raise a number of really good points and although at this present moment in time, I can’t say we are addressing every one of those, I can promise you that we are addressing at least some of those.
We are in the final stage of developing a redesigned website that has a new information architecture and consolidating much of our information in a single place. During this process, we are also addressing the, sometimes, conflicts in our different css files and bringing them together as well.
This is part of a larger project that brings our collections online and addresses the problem of having various small micro-sites that hold the same data in different formats.
We are in what we are calling the phase 2 of the project. Once the new designed and restructured goes live and we iron out any bugs, we shall move on to phase 3, which addresses the microsites. After that, we shall be doing a full evaluation of the site(s) that will ensure that some of the problems you have picked up on is fixed (if it hasn’t already been done so in phase 2 and 3).
Thanks again for your comment. It is very much appreciated. I have noted the things you mentioned and I will be adding it to my list of things to check for the new site.
Regards,
Bilkis.