![A](../images/A.jpg) |
Activity |
![I](../images/I.jpg) |
Information |
![P](../images/P.jpg) |
Presentation |
![W](../images/W.jpg) |
Website |
8 Evaluation Analysis
Having collected all your data and before you are in a position to disseminate
your findings, you need to analyse the information you have collected
and consider what messages are emerging. It is worth highlighting that
the ideas on this page are a basic introduction to the issues of analysis,
see 8A Evaluation Analysis: Further reading.
Factors influencing analysis
There are many ways to analyse data and the methods you choose will
depend amongst other things on the:
- type of data you collect
quantitative (largely numerical
or answers to closed questions) or;
qualitative (answers open questions,
observations, materials produced during an activity, which may include
written, audio, visual, video evidence)
- way you collect and store the data
- experience and preferences of those involved in the data analysis,
for instance it maybe quicker and more effective to commission some
one from outside with experience of statistics to analyse large quantitative
data sets
- time you have available which might be influenced by the numbers of
people within your team, if you have commissioned someone else to help
with the analysis
- access you have to equipment and software packages for instance, the
time required to transcribe an interview will depend on use of a transcribing
machine, or the use of software to support the sorting, coding and analysis
of the data.
![I](../images/W.jpg) |
Evaluation Analysis: Further
Reading 8A (pdf
slides 290kB)
This tool includes further details about and links to websites and
reading materials relating to data analysis |
![back to top of page](../images/top.gif)
![To do sign](../images/To%20do%20sign.jpg) |
Things
to do
The first piece of advice is a ‘don’t’ rather
than a ‘to do’! Don’t leave analysis till the
end; think about it from the beginning. This involves:
Identifying who will contribute to the process of analysing data
- WHO
Thinking about the data you collect - WHAT
Considering how you will store, retrieve and analyse the data
- HOW
WHO?:
Decide who will be involved in which steps of the data analysis
process. If you haven’t time, experience, or interest in this
aspect of the evaluation process consider working with a consultant
to support the process of analysis. However, try to avoid simply
handing over this element of the evaluation but discuss practical
ways in which you can be involved in contributing to the sense-making
process.
WHAT?:
The approaches used for data analysis will depend on whether it
is QUANTITATIVE or QUALITATIVE data. See 8A for
more details about the Learnhigher CETL ‘Analyse this!’
that includes a short interactive
quiz to aid understanding about the difference.
HOW?:
Think about how you will collect the data and how you will record
and store it for the purposes of data analysis. For instance:
quantitative data gathered from paper questionnaires need entering
into a spreadsheet; who will have responsibility for data entering?
Is it possible for them to submit answers to an electronic survey?
developing a coding system for recording answers can speed the
process; however, it is important to ensure that you use consistent
codes and approaches. For an example and template for setting
up basic Excel and SPSS spreadsheets to help record questionnaire
data for analysis see Greater
Merseyside Aimhigher Evaluation Toolkit
Analysing qualitative data also requires preparation; see the
Evaluation analysis tools:
8B (using a matrix or template) which is a
simple and straightforward method whereby you record the answers
from a series of semi-structured interviews, or participants
in a focus group discussion
8C (using a thematic approach) whereby you
analyse the data using a priori (existing) codes or generate
codes using a grounded theory approach based on emerging ideas.
|
![back to top of page](../images/top.gif)
![I](../images/P.jpg) |
Evaluation Analysis: Using a Matrix 8B
(pdf slides kB)
This powerpoint presentation goes through the steps and gives examples
of analysing data using a matrix |
![P](../images/I.jpg) |
Evaluation Analysis: Using a thematic
8C (pdf slides kB)
This information sheet works through the steps taken to analyse qualitative
data using a thematic approach. |
Achieving a balance and finding time
As with all aspects of the evaluation process the challenge is achieving
the balance between what is desirable and what is feasible. The aim is
to try and make sense of the data. This sense-making (Knight, 2002) process
involves you in trying to extract messages from the raw data you collect
and represent them in a format that is suitable for decision making and
dissemination.
There are advantages and disadvantages with all approaches to data analysis.
For example:
- How you analyse the data will depend on what you have collected.
- What is convincing evidence for answering a question such as: ‘who
participated in the activity?’ is not going to answer the question,
‘why did they participate in the activity?’
The process of analysis is also the stage at which you will bring the
different pieces of the jigsaw (data) together to help you answer your
original questions. Remember it is not necessary to evaluate everything.
It is important you make clear what you have done and that you do that
well. When you have analysed your data it is vital that you do not make
claims for your data that you cannot justify.
Although analysis of the data predominantly happens towards the end of
the evaluation cycle it is important that you leave enough time to analyse
your findings and disseminate them to the appropriate audiences. This
needs careful planning and it is sometimes necessary to recognise the
need to be selective when collecting your data and focus your analysis
and evaluation on answering the questions that will allow you to USE your
findings demonstrate your accountability, aid your development or contribute
to knowledge according to your original PURPOSE.
Return to 'Toolkit' Structure:
Ten features of evaluation
|