STAGE 6. Defining the evaluation plan
What steps should be completed?
Step 1: Select the type of evaluation
The strategy of this evaluation design consists of taking a series of measurements of a subject or group of subjects, before and after the application of an intervention, so it is also known as repeated measures design. The measurements recorded reveal a pattern or trend before and after the intervention, which is used to evaluate the changes that have occurred in that period of time. Whenever the pattern or trend changes, it can be assumed that this change is due to the intervention.
The subjects (or study units: community, treatment centre, etc.) in this design act as their own control and there is no need for a control group nor are there any minimum size requirements for the sample. When measurements are taken before and after treatment, any potential trend or process occurring during the intervention can be established and effects can be monitored to see if they continue, stabilise or disappear with time. Biases attributed to the historical development of the phenomena (maturing or learning effects) can therefore be discarded. In this design, threats to internal validity decrease as the number of measurements increases.
A limitation of this design is poor control of potential external influences that may be introduced when the treatment is applied. This limitation can be minimised by introducing a control group, which would restrict the potential confusion variables, as both groups would be subject to the same influence.
Time series designs allow inferences on the effects of an intervention to be made, but do not establish a cause-effect relation, so its explanatory potential is medium. However, they allow two of the conditions that often increase the cost and difficulty of experimental and quasi-experimental designs to be overcome: finding a control group equivalent to the intervention group and having samples that are large enough for the results to be extrapolated to the population as a whole. They can therefore be a useful design to evaluate preventive community interventions, treatment interventions, and even the effect of public policies.
References:
1 Arnau J. (1978). Psicología experimental: un enfoque metodológico [Experimental psychology: a methodological approach]. México: Editorial Trillas.
2 Biglan A, Ary D & Wagenaar AC. (2000). The Value of Interrupted Time-Series Experiments for Community Intervention Research. Prevention Science. 1(1): 31-49.
López MJ, Marí-Dell’Olmo M, Pérez-Giménez A & Nebot M. (2011). Diseños evaluativos en salud pública: aspectos metodológicos
3 [Evaluation designs in public health: methodological aspects]. Gaceta Sanitaria. 25 (Supl.1): 9-16.
4 Escudero, JR, Vallejo G. (2000). Comparación de tres métodos alternativos para el análisis de series temporales interrumpidas [Comparison of three alternative methods to analyse interrupted time series]. Psicothema. 12 (3): 480-486.
© COPOLAD. Cooperation Programme between Latin America, the Caribbean and the European Union on Drugs Policies.
