Nepal
Stories How do you quickly and safely assess students’ early-grade learning during a global pandemic? USAID’s Early Grade Reading Program II (EGRP II) in Nepal faced this exact challenge when designing the program’s evaluation in 2020. The solution: a local, scalable tool with strong Nepal government buy-in called the classroom-based Early Grade Reading Assessment (CB-EGRA). Additionally, using this tool to assess impact was 80% less expensive than deploying a traditional EGRA.
Explore the CB-EGRA results:
CB-EGRA History and Rationale
A national-level EGRA in 2014 identified substantial shortcomings in early-grade reading outcomes across Nepal. In response, the government began rolling out a National Early Grade Reading Program (NEGRP) to improve learning in grades 1-3 in 2015. As part of the NEGRP, the government identified that teachers, education administrators, and policymakers needed a simple tool to measure student learning so they could adjust curricular materials and instruction accordingly. To respond to this need, the CB-EGRA was developed by Nepal’s Education Review Office (ERO) in 2017 with technical assistance from USAID’s first Early Grade Reading Program (EGRP). CB-EGRA is similar to the Early Grade Reading Assessment (EGRA) that is typically used to measure outcomes in international education interventions, but CB-EGRA is typically shorter and much less intensive and expensive to administer. CB-EGRA has since been used by thousands of teachers across Nepal and has also informed the development of national-level assessment tools that can be used for measuring the Sustainable Development Goal (SDG) 4 indicator on minimum proficiency in reading.
Benefits of classroom-based EGRA:
80-85%
less expensive than an EGRA
Scalable
one teacher can administer to an entire class
Localized
developed by Nepal's Ministry of Education, Science and Technology
CB-EGRA in the EGRP II Learning Evaluation
Beginning in June 2020 and running through May 2022, EGRP II continued EGRP’s tradition of technical assistance to the Government of Nepal to build capacity for NEGRP implementation. EGRP II’s overall activities included:
- Supporting the development and rollout of the Government of Nepal's “Integrated Curriculum” and continuing to scale up the government's NEGRP.
- Building municipal- and provincial-level capacity for delivering early grade reading services.
- Improving local governments' ability to provide effective teacher professional support.
- Supporting continuity of learning in response to the COVID-19 pandemic.
To understand the impact of these interventions on student learning, the program coordinated with ERO to conduct a baseline study in February and March 2021, followed by an endline in February and March 2022. This evaluation was implemented during a time of substantial disruptions due to the pandemic, including prolonged school closures and restrictions on travel. To adapt to these circumstances, EGRP II developed a novel approach to summative assessment that enabled the program to reduce the sample size and time required for data collection. This was achieved through a process of statistical linking: equating student scores on an early-grade reading assessment (EGRA) with a CB-EGRA.
CB-EGRA is a multiple-choice test administered by teachers to groups of students in grades 1-3 to measure their reading abilities. It includes assessment tasks such as word and sentence reading, vocabulary, word separation, dictation, listening comprehension, reading comprehension, and calendar reading. However, since CB-EGRA is group administered, it does not capture oral reading fluency rates for individual students. This skill must be measured and reported for USAID’s standard indicators in the education sector and is also often used for global SDG reporting.
During the baseline evaluation, EGRP II worked with ERO to conduct a small pilot study to statistically link1 student scores on the CB-EGRA to their scores on specific EGRA subtasks. Using assessment linking in the case of EGRP II enabled the program to report on USAID standard indicators related to oral reading fluency while also adapting to the pandemic circumstances and tapping into an existing classroom-based assessment tool that already had broad buy-in in Nepal.
EGRP II conducted the equating process by assessing 758 grade 2 and 804 grade 3 students in a sample of 45 schools using CB-EGRA at baseline while at the same time administering the EGRA oral reading fluency and comprehension tasks with a subsample of 229 grade 2 students and 229 grade 3 students from the same schools. The program team then used statistical techniques to identify equivalencies between student composite scores on the CB-EGRA and their scores on the EGRA tasks. For example, using this statistical model, Nepal’s national benchmark for fluent reading (45 correct words per minute at the time) had an equivalent CB-EGRA score of 59%. After establishing these equivalencies, the program was able to assess children using only CB-EGRA at endline and still report on the percentages of children at different oral reading fluency proficiency levels.2
CB-EGRA composite vs Oral Reading Fluency (Grade 3)
Learning Evaluation Results
The program team analyzed the overall EGRP II impact on the proportion of students achieving the following levels of reading proficiency, which aligned with Government of Nepal’s reading proficiency benchmarks that were in effect at the time:
- Low Reader — a student correctly reading between 0 and 14 words per minute.
- Emergent Reader — a student correctly reading between 15 and 44 words per minute.
- Fluent Reader — a student correctly reading at least 45 correct words per minute.
EGRP II identified a 7.5 percentage point increase in the proportion of fluent readers in grade 3.
Grade 3 Program Impact on Levels of Reading Proficiency
No significant improvements were identified in grade 2, although there was no further learning loss in that grade despite significant disruptions caused by the COVID-19 pandemic.
Grade 2 Program Impact on Levels of Reading Proficiency
These assessment findings demonstrate that the experience of equating CB-EGRA and EGRA proved effective in enabling EGRP II to report on USAID indicators related to reading proficiency levels without using an EGRA. The CB-EGRA also has the benefit of being less technically complex for teachers; they are generally familiar with group-based assessments as part of their everyday practice. Since it has broad buy-in in Nepal, and because equivalent scores are now established, the Nepal government and other actors can use CB-EGRA to report on indicators that require reporting on oral reading fluency.
Conclusion
The pioneering assessment approach in EGRP II demonstrates that early-grade reading assessments can be done less expensively and less intensively but still provide critical information for decision-making within education systems. This approach could be continued in Nepal based on strong local buy-in and adapted in other settings to benefit early-grade learners in the post-pandemic period and beyond.
Note: Some of the content in this blog was adapted from the following briefing paper developed using RTI International’s Independent Research and Development funds: Fonseca, J. and Shrestha, R. (2022). Learning and resilience in response to COVID-19: Innovative approaches to formative and summative assessment. Research Triangle Park, NC: RTI International.
[1] One of the most well-known examples of this type of “assessment linking” approach is found in the United States, where equivalencies are used to establish a one-to-one relationship between ACT and SAT exams so scores can be compared across students.
[2] Swadesh Maharjan, Buddhi Chaudhary, Manoj Bhandari, Simon King, Jodie Fonseca, Wendi Ralaingita, USAID’s Early Grade Reading Program II (EGRP II) in Nepal Endline Report: Program Impact on Student Reading Performance in the Early Grades (Research Triangle Park: RTI International, 2022).