To assess the reliability of existing data, CAFOD has developed the following tool:
Aim: To assess available data to decide how reliable and useful it is.
Context: This tool can be used individually or with a group, although it is probably best suited for individual research. If you have many sources of data to assess, members of a network or team could use the steps below and apply them to different data sets.
How to use this tool:
Step 1: Establish the source of the information. For example, was it gathered by a government department, by independent researchers, consultants or a CSO? What do you know about their respective reputations?
Step 2: Pinpoint the timeframe for the data. How long ago was it gathered? What time period does the information apply to? How does this relate to the time period you have chosen for your monitoring?
Step 3: Find out how the information in the document was gathered. What methods were used? How reliable were these methods?
Step 4: Assess whether the data is representative. What or who does it represent?
Step 5: If the source includes a discussion or analysis of the data, consider from what perspective this has been done. Can the data be interpreted in other ways?
Step 6: Decide whether the data reflect any clear contradictions or discrepancies. Can these be explained to your satisfaction?
Step 7: Draw a conclusion. Based on your answers to the questions above, what can you say about the reliability, credibility and legitimacy of this data?
Source: CAFOD, Christian Aid and Trocaire (2007) Monitoring Government Policies: A toolkit for civil rights organisations in Africa: p.51.