Once the question has been defined, it is a great deal easier to design the research programme. It is important to note that the quality of the findings at the end of the research are in part a reflection of how well these questions have been addressed.

For existing rail passengers, there are a number of areas which are more relevant to evaluate than for non-users. For example ‘behavioural change’ is more relevant to non-users, as researchers will be seeking to understand what makes them switch to rail travel. However, it may be of interest when considering how to retain existing passengers. In this case it is often important to think about the passenger and how they perceive the quality of the service. Therefore questions should assess their level of satisfaction with stations, information and services.

Feedback and understanding your target group
In addition to gauging satisfaction, research on passengers provides an indication of which aspects of the services and facilities are good and which can be improved.

The approach of being receptive to your passengers’ thoughts and feelings serves more than one purpose. Other than evaluation, it also gives a good impression that passengers are listened to, their thoughts and feelings matter and this is part of the ethos of Citizens’ Rail.

To provide answers, some form of data may be required. Data may be qualitative or quantitative. It reflects the subject of the research question, so if the question involves assessing passenger movements, frequencies their journeys on trains, purposes or times, then data should focus on numbers and statistics.

If peoples’ attitudes towards aspects of rail travel are of interest, then the data should accurately convey their thoughts and feelings. For instance the quotes in the below graphic are from the qualitative questions from the Citizens’ Rail evaluation in the Devon area. These quotes are qualitative data and can be used in an evaluation to understand more about rail passengers’ ideas regarding the services.

A number of considerations are important here.

• The time and resources available to the evaluation programme: this includes monetary budget, suitably trained staff and the amount of time available to collect the data.
• How to access the data required. Are the target audience easy to reach? Does the researcher visit certain stations / areas or trains to collect data?
• How to ensure the data represents the target audience accurately (see the following section on sampling).

The first of these, available resources, is very important to consider before proceeding. Often a trade-off will occur and if resources are low, it might not be possible to get the best accuracy in your results.

A range of data collection methods exist to draw insights into rail passengers, and the choice of these depends on the aims of the evaluator:

• Survey interviews
• Self-completed survey questionnaires
• Qualitative interviews
• Focus groups

Example of a survey questionnaire form (below). This could be administered by interview, either face to face, by telephone or by email. Alternatively, it could be ‘self-completed’, and handed to the respondent for return by post.

Example of a qualitative interview question (below). The wording of the question encourages open-ended answers allowing the researcher to understand more about the individual’s circumstances, thoughts and feelings in depth, hence the longer answers. An interview may be recorded and transcribed in order for the researcher to go back and analyse the conversation more thoroughly.

Question 1) ‘In the questionnaire, you gave us some information about your most frequent journey. In order for us to find out more about this, we’d like you to let us know the surrounding context of this journey. Why do you travel by rail? Are there any other ways you could travel and what are your thoughts about making the decision to choose rail over the others? Please go into as much detail as you can.’

Example of a focus group question and answers (below). Focus group questions can be similar to those used in interviews, allowing an open-ended in-depth response. However, the difference is that, by having a number of people present, a conversation can develop. Again, recording the conversation, and transcribing it allows the researcher to be able to see what the issues are.

Focus group facilitator / researcher: ‘Think about your most recent journey by rail. I was wondering what people consider to be the most crucial part of their travel experience. The difference between using the rail over other modes of transport. What keeps you using rail? And what could be improved? ’
Respondent 1: ‘The fact that I’m not tied to anything. I don’t have to worry about a parking space….I can work on the train, or just look out of the window….’
Respondent 2: ‘I agree, I like the flexibility in some ways….I can meet colleagues for drinks after work and not have to worry’.
Respondent 3: ‘Sometimes when trains are late I wish that I could travel by car….but overall I’m satisfied.’
Respondent 1: ‘It’s a choice for me…I like not having to worry about a car…’

If a sample is completely (100%) accurate, the data will reflect that of the wider population exactly. This is not going to be the case unless an entire population is sampled, which is an unlikely event. Usually, there will always be some ‘bias’, a term which means that the data is different from the true picture. A sampling strategy should seek to reduce that bias as much as possible.

To understand sampling concerns further, ask the five questions: who, what, how, where, when.

• Who are the target audience?
• What is the scale (geographical or otherwise) of the population?
• How do you reach the numbers of people you need?
• Where do you sample to get the most representative data?
• When are you going to collect data in order to ensure your sample accuracy is maximised?

This is just the example for a survey. For other research methods the sampling strategy is different. For instance, interviews typically only need 30 or less respondents. The researcher therefore has to consider how to access the most suitable people to contact for interviews.

One scenario which became apparent in the Citizens’ Rail evaluation was that a number of people stated they did not always use rail because of certain factors on their most frequent journey. This prompted the research team to issue another follow-up survey, this time by email, to respondents who had agreed to be contacted further. The email contained a few short questions which asked people to describe the circumstances of their most frequent journey in more detail.

They way in which this is done will depend on whether the research is qualitative, quantitative or both.

Quantitative analysis
Using a software package such as Excel or SPSS, the data can be assessed firstly by performing a descriptive analysis for each question asked. This will provide an idea of how the sample is made up. Are they mainly older or younger? What are their current travel habits? And so on…… The following examples provide information on how the descriptive analysis of rail passengers for Citizens’ Rail was carried out.

From the above tables the following results can be deduced:

• From 136 rail passengers, the average distance of their most frequent journey is approximately 94 kilometers
• The majority of these trips are for work, and weekly (30%) or 5 times a week (24.3%)
• There is a fairly even split between people who have heard about local plans for rail improvement and people who have not heard of the plans
• Press is the most visible source of such plans

Qualitative analysis
When analysing interview or focus group transcripts, the researcher is can be more flexible in their analysis, based on their aims and preferred research style.

The Citizens’ Rail research produced a number of key findings, including:

• The need for less complex more flexible ticketing
• More human presence at stations to provide information and help passengers feel safe
• The importance of being involved in local railways
• More flexible times and less crowding on trains

Qualitative research can explore themes such as these in depth and give the evaluator a broader understanding of the issues. The way in which the data is interpreted which decides which themes are important, which is why there will always be some ‘subjectivity’ (judgement is affected by personal thoughts and feelings) in the findings.

REPORT: Reporting results and findings for rail passengers

There are a number of ways in which findings can be reported but it is important to think clearly about where the information will appear. Here we return to the graphic from the first section, which becomes important again at this point:

• Who: Who are the target audience? Practitioners? The general public? Policy makers?
• What: What are the outcomes? Funding? Policy change? Marketing?
• Where: What are the outlets for the findings? Reports? Website? Dissemination events such as conferences?
• Why? Go back to the original aims . Are you fulfilling them? What was the overall purpose of doing the research in the first place?
• How? How can the research be used in the future? Does it solve the initial issues, such as understanding the non-users in the target population? Can others draw from the findings? How can they be shared?

Whether it’s through charts, photographs, text or tables, both qualitative and quantitative data can be using an increasing number of visual and audio means. It is important to ensure the maximum reach to your intended target audience through documentation of the findings, face to face dissemination through presentations and dissemination events, and social media.

Example of a quantitative output:

Question: How much would the following encourage you to use trains more?

Whilst a chart can help the reader to make their own interpretation, be careful to ensure that a valid explanation is given. For example, the above chart details responses from people who use the train, and their evaluation of the importance of car parking at the station. This is crucial in determining whether car users may continue to use rail.

Example of a qualitative output:

‘I have found the railway service to be effective and an enjoyable experience – in many ways my preferred mode of transport. The main barrier is the journey to the station, Parking at the station is very limited, which precludes driving to the station then taking the train from there. I would suggest a more integrated service with frequent links to stations from a variety of areas would be one measure to improve services, preferably with an integrated fare structure – one ticket for all forms of transport.’

‘From the perspective at Torre station in particular access is difficult to one of the platforms for anyone with mobility difficulties or families with children in pushchairs etc because everyone has to use the bridge with steps. Level access and additional car parking together with a ticket machine would help. I know this can only be a long term plan but there is land to rear of the station that could be used for the purpose.’

‘If ticket prices were simplified with only one price for the same journey with maybe the only difference would be that after 9.00am it was cheaper, people would use the train much much more. Like paying a fair and getting on a bus, one price whether it’s as you step aboard or get your ticket 3 months ahead. Have a system like the tube that you can’t get on without a ticket to stop the fair dodgers.’

Example of a PowerPoint presentation: