Select Language
Supported by EU Interreg IVB | Accessibility | Cookies

Citizens’ Rail toolkit  > Evaluate your success > Understand the background of evaluation

Understand the background of evaluation


Why evaluate?
Evaluation allows you to measure a specific impact comparing a predefined target and the actual state. It deepens the understanding of current situations, processes and structures and contributes to their future improvement.


What is evaluation?
Considering the given railway context we understand evaluation as a systematic process of gathering data to assess the success of a railway project in meeting its objectives and to answer the question, if your project makes a difference12. Possible evaluation subjects are:

Evaluation subject Regional railway example
Product Use new rolling stock, Implement new rail connections or improve frequency
Service Modernise ticketing system, Employ more staff at stations
Measure, action Refurbish station together with citizens, Develop a citizens outreach and engagement strategy
Project Build or improve station, Develop and implement new marketing concept
Participation, Governance, Organisation Involve citizens in line or station development, Implement discussion forum with rail stakeholders, Establish a community rail partnership
Programme, strategy Adopt a regional rail funding strategy, Reform public transport strategy
Policy Pass rail privatisation regulation, Change rail regulation structures
Technology, research Use mobile devices for customer information and ticketing, Use online participation tools

Subjects of evaluation
Source: Citizens’ Rail project team based on “Standards für Evaluation”, DeGEval – Gesellschaft für Evaluation, 2008, p.15


Evaluation comes up with its own vocabulary. Sometimes the definitions for the expressions are used in different ways, which can be rather confusing than helpful. Make clear to use the terms in a consistent way for your project. This table is meant to make you familiar with some terms that are crucial in the context of evaluation.


How to evaluate?

If you like to evaluate in the context of railway projects, you should try to find answers for the following guiding questions13, which also represent certain evaluation types:

1. What and why (for which reason)?

This question is strongly related with the sections above (i.e. evaluation subjects). We also already mentioned that evaluation is part of the railway project process in the section “Railway projects, participation and evaluation- How do they relate to each other?” However, evaluation is carried out in a process design, itself. At the beginning you should clarify general circumstances e.g. what´s your specific goal of evaluation. When it comes closer to the evaluation implementation your questions need to be more detailed, e.g. what type of information can tell me about participation, which data do I need to collect for this, which methods will I apply? Keep in mind how you will deal with the evaluation results. Will there be any feasible consequences of the evaluation for the project? Your evaluation process might look like this:

Evaluation workflow
Source: Citizens’ Rail project team, based on Nabatchi (2012)


2. When and why (issues)?

Regarding the ‘when’ question, often a distinction is made between ex ante and ex post evaluation approaches. Depending on the specific timeline of your project, it is essential to identify whether it is possible to evaluate measures or their impacts ex-post, once they are completed. Otherwise an ex-ante approach can be utilised before completion in order to cope with the objective of an accompanying evaluation. Regarding the ‘why’ (evaluation purpose) in combination with the ‘when’ question, you can distinguish between formative and summative types of evaluation. In addition, there are other types of evaluation in between them.

Evaluation types
Source: National Centre for Sustainability NCS & Swinburne University (2011), p. 5 14: 5 (after Owen/ Rogers (1999); Government Social Research (GSR) Unit (2007), p.4f


3. How and with what (methods, indicators and data)?
In the majority of cases, railway evaluation implies a strong focus on quantitative and exclusively transport related data such as passenger data, revenues or costs. In context of citizen-oriented railway it is rewarding to broaden the perspective in order to understand the interplay of society and local railway. Therefore it is crucial to develop a mixed method approach which also includes qualitative aspects. Examples are given in the other parts of the section “Evaluate your Success”:

Read more about evaluation methods, indicators and data







Back to the Evaluate Your Success page