Cannot deselect all options for a given filter.
+
The Early Grade Reading Assessment (EGRA) is a good tool used to measure the impact of early grade reading programs on students’ basic literacy outcomes.
Evaluation studies often implement a reading program in one area and compare student reading abilities in that area to those of students in another, similar area where no program is being implemented (called the control area).
Using EGRA, we can assess the difference in reading ability between students who were part of the program and students who were not. The most commonly used EGRA score metric to test this is oral reading fluency, commonly known as “ORF.” This difference in ORF is the impact of the program.
How Is Intervention Impact Calculated?
Intervention impact, or the effect of a reading program, is calculated by the difference between the gain in learning outcomes at the program and the control schools.
In the example below, the mean ORF score was 5 for both the program group and control group at baseline, or the beginning of the program. Between the baseline and endline, the average reading fluency gain was +6 (5 to 11 correct words per minute [cwpm]), while in the same period of time the control group average gain was +2 (5 to 7 cwpm). The net gain for students at the program schools was +4 cwpm. This is known as the intervention impact.
Effect Size
An important consideration when looking at intervention impacts is the effect size. An effect size is the difference between two averages converted into standard deviations. This allows us to compare the size of one difference with other differences also converted into standard deviations.
Different countries, contexts, languages, and grades will all result in different scores on an EGRA, and therefore different average impacts. To compare across different contexts such as countries, languages, and grades, we standardize the impact score using an effect size. In the Nepal 2020 example, the effect size is small, but decent.
We interpret the impact of a program on student reading outcomes using the following rule on effect sizes:
Adding Context
It is important to answer why there is impact (or not). For example, we should consider the scale of a reading program, its cost, and how effectively stakeholders are implementing it, along with any effect size or scores. To properly interpret early grade reading program impact, we always consider the whole picture, which we can get through additional measures such as stakeholder interviews and classroom observations.