Research Survival Guide Part 2: Evaluating Resources

This post follows on from Part 1: The Literature Search

Once you’ve gathered results you need to consider what to do with them. In most instances there is no way you can read everything on the list, and some of it won’t turn out to be relevant. So, ask first, is it for an essay or for a dissertation? This will impact on how many sources you should look at.

You then need to evaluate your sources.

For books and ebooks use Heritage or LibrarySearch to find copies. If you are unsure whether the book will be useful, take a look at the ebook where one is available. This will save having to get hold of a physical copy of the book from the Library.

Ask questions such as:

Why was the book written? – was it to inform, to persuade, to entertain, to teach or to provide an overview of an area?
Who published the book? – was it a university press, commercial publisher, government, professional association, a campaign group?
Is the book well organised? Do the contents or index indicate you are likely to find material you need?
When was it published? Consider the date if this is relevant to your subject. For English literature, history, theology, age may be no barrier to good content, for science subjects and law etc., making sure you have the most up to date information on the subject is key to a good piece of work.
Has the book got a bibliography? If so, does it cover primary sources? If you find a good book that has a bibliography don’t forget to use that as another source for finding useful material.
For what sort of audience is the book aimed? i.e general reader, students, specialists, researchers?

Many of the questions above apply equally well to journal articles. We recommend accessing articles via LibrarySearch+, and within the results you can limit your search to ‘peer reviewed’ articles which have been evaluated by other academics. You can also use the options on the left to limit your search in a number of other ways, including date ranges.

Bear in mind that the majority of articles in LibrarySearch+ are available to you, but for some you can only access an abstract. Again, if in doubt ask a librarian to avoid paying wherever possible.

In some subjects, particularly the social science subjects, you made need to find more current information than journal articles can provide. Newspapers are a good source of recent information, and you can find newspaper articles on LibrarySearch+ and also through the LexisNexis database.

Bias in newspapers is definitely to be considered, particularly political bias. Most papers are fairly openly and obviously in support of one political party over another, so bear that in mind and look at multiple sources rather than relying on just one.

Google and Google Scholar can be incredibly powerful tools for gathering together resources that might be useful for your literature search. But what you need to recognise and compensate for, is that Google, and even Google Scholar, have no academic filters. They will bring you results, but with no indication of how trustworthy or accurate the contents are.

However you can use Google’s Advanced Search to limit searches. A particularly good one for some of you will be to limit the results to sites with certain web address endings such as .ac.uk and .edu for University sites, or .gov, .gov.uk for official government sites. .org sites are more complicated – they tend to signify charities or other non-profit organisations, but this does not mean the content is without quite heavy bias at times.

So what can you do when faced with websites that you need to assess?

Let’s say you need to find out information about Homeopathy. A quick google search will bring up lots of results, including quite a number of official sounding bodies. Let’s have a closer look at some of these sites to see how you can assess them.

Start off with some of the same questions you asked of a book:

Why was the site written?
Who wrote the site?
When was it last updated?
Who is the target audience?

Let’s start with Wikipedia:

We’ve got a long article here, but in the opening paragraphs we can see homeopathy described as “nonsense, quackery or a sham”. That’s a fairly good indicator of the attitude of this article, but of course bear in mind that many people can edit this page, so the bias of it may change at intervals. Usually Wikipedia notes at the top of a page if the article seems to be particularly contentious, but don’t assume because the information is crowd sourced and indeed well-referenced at times, that it is entirely correct. It can change in an instant.

So how about the NHS, their page must be neutral surely?

Well yes, pretty much, but of course the NHS bases its opinion on scientific evidence and medical opinion, hence their description of homeopathy as a ‘treatment’ in inverted commas…

So what about other official organisations?

The Homeopathy Research Institute has an entire research database listing relevant articles:

But consider, what journals have these articles been published in? And are they peer-reviewed or likely to have a bias. Also, does this research database contain references to any articles which give negative conclusions?

How about other groups with a viewpoint? The 10.23 group has a clear bias that you can’t miss…

Their information is one-sided and not referenced with any sources. However, it may be worth taking note of their mass overdose protest as a form of evidence against the effects of homeopathy.

So what are we to conclude? Well, for homeopathy at least, most websites have an intentional or discrete bias, so looking at a number of sites is advisable. This will be true for any even slightly contentious topic – consider political research into the Arab Spring, the continuing, though discredited, furore around the MMR vaccine and autism, or even historical portrayals of prominent figures like Richard III.

It’s important to ask why a webpage has been created? What is the motivation of the author? And it’s always worth clicking around a few pages of a site to understand their argument or confirm their neutrality. If there is an about us or FAQ page that may quickly provide you with that information.

What else? These days, the design quality of a website says nothing for the credibility of content, lots of people have the ability to put together a shiny website. Neither does the web address. A healthy level of cynicism is the best way to approach websites, even something like the BBC.

One final thing to note with websites. Statistics in particular are something to be wary of online, unless coming from an official website (and even then keep a critical head on as to how they may have been manipulated to show the best result), or from a website clearly citing their sources or numbers. Wherever possible try to locate the raw data from which the statistics have been derived. Remember, 74% of statistics are made up on the spot.*