Moving Forward the Education Data Frontier in Household Surveys

Education is a key dimension in poverty measurement. Not only it is valuable on itself, but it also enables the achievement of other valuable goals.

From a rights perspective it has a similarly double role: not only is it a recognised human right on its own (since 1948) but it is also an enabling human right.

Education empowers people to understand the complex world in which we live and to make more informed decisions in multiple areas of life – health, education, finances, family planning, work, to name a few. In other words, it allows people to take control over their lives.

How education is currently measured

Education is a constituent dimension of the global Multidimensional Poverty Index (MPI) introduced in 2010. In the global MPI a household is deprived in education if no household member who could have completed six years of education has done so, or if any school-aged child is not attending school (up to class 8).

All national MPIs developed so far, covering over 40 countries around the world, include an education dimension. Most commonly, the indicators used are years of schooling and school attendance, but usually using more demanding thresholds than the global MPI. These two indicators are included in most MPIs because the required data to construct them are available in virtually all household surveys.

The schooling achievement and school attendance indicators provide fundamental information for two reasons. Firstly, in many countries, there is still a long way to go to achieve universal primary enrolment and an even longer way to achieve universal secondary enrolment.

Secondly, schooling is a necessary condition for literacy and numeracy. Despite this, the schooling achievement and school attendance indicators fall short of capturing the quality of education. This is not a minor issue. The World Bank, in its 2018 Report,  raises the issue of learning poverty: “for millions, schooling is not producing enough learning. (…) Globally, 125 million children are not acquiring functional literacy or numeracy, even after spending at least four years in school.”

Notably, the learning crisis is not exclusive to poor countries. Even in countries with universal or almost universal coverage, substantial proportions of in-school children do not achieve minimum reading comprehension skills: for example, 59% in Argentina, 43% in Uruguay, and 69% in Egypt. Some of the poorest countries, such as Zambia and Niger for example, exhibit not only high schooling deprivation rates but also extremely high learning deprivation rates (Zambia 15% and 98% correspondingly; Niger 33% and 85% correspondingly).(1) The learning deprivation rates are even higher for secondary school aged children. In summary, when looking at indicators beyond attendance and completed years of schooling, such as those reflecting cognitive assessment, educational deprivations are exposed for many countries with very low global MPI values. Furthermore, for countries with higher MPI values, deprivations in education are even more crudely revealed.

Source: Author’s own elaboration with data from 2018 PISA (OECD) and MPI data from OPHI.

 

Capturing cognitive skills

However, collecting data on cognitive skills in household surveys is challenging.

This is because, for the data to be accurate and relevant, it needs to be at the individual level. This takes time and implies that all household members should be assessed (or at least all children), with the additional complexity that relevant cognitive skills naturally vary with age.

Not coincidentally, the most common way to assess cognitive skills so far has been through school-based standardised tests. While being an incredibly valuable source of information, the data from these assessments refers only to in-school children.

Furthermore, it cannot easily be linked to household surveys and the amount of information on the child’s household and other household members is quite limited. Therefore, there is still the outstanding need to integrate some measurement of cognitive skills into household surveys, which would allow for a more accurate measurement of multidimensional poverty.

In fact, household surveys have the power to integrate indicators on diverse dimensions into the same tool, allowing the identification of joint deprivations. They also have the power of regularity.

Data for multidimensional poverty measurement would be highly enriched with information on cognitive skills, even if only at a minimum level. This data would bring a new lens on deprivations in education across all levels of development, enabling a better tuning of public policy. Naturally, this enriched information would serve to monitor not only the first SDG but also the fourth, which is focused on ensuring inclusive and equitable quality education and promoting lifelong learning opportunities for all.

In recent years, assessment tools called “citizen-led assessments” (CLAs) have been designed for the household survey format. CLAs have been developed and implemented with the aim of ensuring the inclusion of all children, irrespective of schooling status.

Based on these initiatives, a proposal was made at the OPHI Workshop in February to develop a ‘Minimum Cognitive Skills Assessment Tool’ (MCSAT) for children and another for adults. The proposal is to test all school-aged children in the household and only the most educated adult in the household, whenever this adult has not completed the equivalent of a bachelor’s degree.

For children, we propose to use the Annual Status of Education Report (ASER) assessment instrument, which was developed by the ASER Centre, established in 2008 as part of the Pratham network in India. ASER was the first CLA (PAL Network). ASER can be administered on paper, but it is adaptive. In the reading section, it starts with the ‘paragraph level’. If the child is not at that level, the evaluator moves to the recognition of words and if this does not work either, it moves to the recognition of letters. If the child succeeds in reading the paragraph, the evaluator moves to the ‘storytelling’ level.

Similarly, the numeracy test starts at the subtraction level and, according to the child’s performance, the evaluator moves down or up in level. The instrument is easy and quick to administer, taking 10 minutes on average. The proposal is to implement this tool for children 8–12, and, to avoid the so called ‘ceiling effect’, to develop a slightly more challenging version for children aged 13–17.

If implementing the ASER test within a multi-topic household survey is unfeasible for time (and cost) reasons, the minimum desirable assessment would be a fluency test even though this would miss testing numeracy skills. There is strong evidence that the number of correct words read per minute closely correlates with reading comprehension, and that children must reach adequate fluency to comprehend passages.

For adults, the minimum assessment of skills undertaken should include literacy, as is currently undertaken by DHS and MICS through reading a sentence on a card. This is certainly better than no testing at all. However, the recommendation is to try a more ambitious approach which would test adults in two key dimensions, health literacy and financial literacy, at some minimum threshold. A household in which the most educated adult cannot deal with these basic daily issues can be considered deprived in education.

The health questions we propose have been borrowed from health literacy questionnaires. They test, for example, whether the person understands the timetable for taking a medicine from a prescription. The questions on financial literacy have been taken from OECD. They test understanding of the implications of inflation on an individual’s purchasing power, and the notion of a certain amount of money earning an interest rate.

These are preliminary proposals which need to be scrutinised and piloted. However, we emphasise the importance of international development agencies and national statistical offices exploring these possibilities.

Any progress in household data collection on cognitive skills, even at minimum levels, would be a leap forward compared to what is currently being collected, with the potential of substantially enriching multidimensional poverty data and related policy design.

 

(1) These estimates come from World Bank (2018).

 

 

This article was published in Dimensions 16