Coneval: Institution-building for multidimensional poverty measurement in Mexico

PDF version available here

 
In the early 2000s, Mexico launched a process of institution-building for its social development policy and the formulation of an official poverty measure, which led to the creation of the National Council for the Evaluation of Social Development Policy (CONEVAL) and the establishment of the first official multidimensional poverty measure in the world. During its first years of work, CONEVAL worked on the methodological definition of the new official measure of multidimensional poverty and the collection of data that made this measure possible. Today, CONEVAL generates official multidimensional poverty estimates with representative data every two years at the state level and every five at the municipal level. Moreover, since its creation, it has published 1,500 evaluations of social policies and programs based on the multidimensional measure and four reports on the social development policy. CONEVAL’s measurements and evaluations also have an impact on the work of subnational entities.

An Institution for Transparency

Until 2002, Mexico did not have an official poverty measure. Different public entities and academic institutions had carried out various estimates, and each government had presented its figures based on its own calculations. Likewise, evaluations of social policies were carried out by the institutions in charge of implementing them. In 2001, as part of a restructuring process and seeking greater credibility in their poverty figures, the federal government consulted with national and international experts to establish standard criteria for the measurement of poverty. After gathering their recommendations, the federal government published for the first time in 2002 official figures on monetary poverty. [1]

However, for various reasons, these figures were met with distrust by the opposition parties represented in Congress. For instance, poverty was said to have fallen between 2000 and 2002 although the country’s economy was contracting during that period; these results appeared at odds with the process of democratic strengthening the country was undergoing, which demanded greater transparency from the civil service.

This identification of an official poverty measure overlapped with other relevant factors. The government began encouraging a more ordered and institutionalized system for the design and implementation of social policies. At the same time, different political parties in Congress began mobilizing to establish a law defining state policy on social development, as there was a significant gap between a series of rights established in the Constitution of 1917 and the rules regulating the public policies to enforce them. In addition, they began discussing the need to separate the managerial and evaluation tasks of social policies that until then had been concentrated in the federal government.

All of this converged in 2004, when the Congress of Mexico adopted, with a unanimous consensus of all political forces represented in it, the General Law of Social Development (LGDS in Spanish). This law established several key aspects of current social policy in Mexico, including the definition of the National Policy of Social Development and the National System of Social Development, which established institutionalism and ordered social policy. For example, the use of official poverty measures is now mandatory for all social development programs focused on poverty reduction. As part of this, an autonomous evaluation body was established, the National Council for the Evaluation of Social Development Policy (CONEVAL in Spanish), which was given the responsibility of estimating poverty and evaluating social policies. Moreover, access to social rights was established as the common thread of public policy in the Mexican State. Finally, the need for poverty to be measured multidimensionally[2] was advanced; this measure was to be periodic (every two years at the state level and every five years at the municipal level) in order to systematically monitor the evolution of poverty in all its dimensions and for it to be representative of the different subnational entities (32 federal entities – 31 states and Mexico City – and 2,457 municipalities).

CONEVAL was created as a decentralised public body with technical and managerial autonomy. Its objective is to provide information to policy-makers and citizens about the state of social policies; multidimensional poverty measurement and policy evaluation; and the strategies, programs, and social development actions undertaken to reduce poverty. To help guarantee the independence of CONEVAL’s work (and thus generate public confidence and credibility) a Governing Board, CONEVAL’s Steering Committee, was created. The Board is composed of six academic career researchers who must be elected democratically by the federal government, the states of Mexico, representatives of the municipal authorities and the Congress of the Union through an open public call. These researchers must be or have been members of the National System of Researchers, have extensive experience in the subject, and collaborate with higher education or research institutions registered in the excellence register of the Mexican National Council of Science and Technology. This is crucial because it guarantees the suitability of the researchers as well as their academic and financial independence, as their academic wages are still covered by the universities to which they are attached. The Steering Committee also includes a representative of the Secretariat of Finance and Public Credit and the head of the Secretariat for Social Development (SEDESOL) who presides over it. There are a total of eight committee members, of whom two are public servants and the rest are academics.

CONEVAL began operating in 2006, and, during its first three years, it was dedicated to providing answers to the LGDS mandates, concentrating on two big research areas in poverty measurement. The first focused on defining the theoretical and methodological framework of multidimensional poverty measurement, while the second was oriented towards generating the necessary data to implement it.

 

The process of generating an official multidimensional poverty measure

The definition of Mexico’s multidimensional poverty measure was a product of an investigation and consultation process with national and international specialists in poverty measurement (see Chart 1), as well as political actors, state institutions, and international organizations.[3] Immediately after CONEVAL’s establishment in 2006, a series of studies and seminars with national and international specialists were carried out to learn more about the most advanced multidimensional poverty measure proposals. Afterwards, CONEVAL requested five specific methodological proposals, which were based on guidelines established by the LGDS, from national and international experts. These five proposals were entrusted to Julio Boltvinik, Satya Chakravarty, James Foster, David Gordon, Rubén Hernández, and Humberto Soto, experts known for distinctive methodological proposals on multidimensional measurement.[4] These proposals were presented in two workshops and one international seminar in 2007 and enriched with commentaries from a wide range of specialists in poverty measurement from Mexican public institutions, universities, and international organizations, as well as from civil society organizations. This consultation work identified the methodological challenges and laid the foundation for the creation of a Mexican multidimensional measure.

Following this consultation phase, CONEVAL developed its own internal methodological proposal, complying with the objectives defined in the LGDS. The methodology was intensely debated and agreed upon within CONEVAL and then presented in two seminars with national and international researchers. The proposal included important innovations compared to the other dominant approaches in multidimensional measurement. Among them were innovations in defining poverty status based on the intersection of deprivations in economic wellbeing (defined by the households’ income) and deprivations in social rights (defined by the social rights established in the LGDS)[5], and also an innovation in income measurement that incorporated important methodological advances in this sphere.

The resulting methodology of this process was made official in mid-2010, when it was published in the Official Gazette of the Federation, making Mexico the first country in the world with an official multidimensional poverty measure. It was also established that this measure was not to be modified for at least ten years in order to guarantee the continuity of the measure, facilitate its institutionalization, and generate credibility among the citizenry and decision-makers. The intense consultation and research process was crucial to fulfilling CONEVAL’s objectives because it guaranteed the measure’s technical soundness and its credibility. At the same time, the participation of the Mexican State and of a wide variety of political actors helped the measure to be institutionalized within the Mexican State.[6]

 

Collecting Data that is Representative, Periodic, and Public

At the same time CONEVAL was working on the measure’s methodological definition, it started designing a tool that permitted the collection of information required by the LGDS. The tool had to provide adequate information about the different dimensions of poverty previously cited and comply with the established periodicity and territorial representativeness. To that end, CONEVAL worked closely for two years with the National Statistics and Geography Institute (INEGI) on the elaboration of the National Household Income and Expense Survey Socioeconomic Conditions Module (MCS-ENIGH). This collaboration included an intense effort to define every one of the indicators to be used to inform the multidimensional measure – including the addition of innovative aspects with respect to the treatment of income – as well as the design, testing, and consolidation of the module. The module was implemented for the first time in 2008, after broadening the sample of the previous round of the survey from 20,000 to 64,000 households to fulfil the representativeness requirements for every federal entity.

Additionally, in order to increase the credibility of the poverty measure, a transparent estimate mechanism for the multidimensional poverty results was created (see Policy Report 11). To do so, CONEVAL made several timely decisions. It used data from the Socioeconomic Conditions Module, which was produced by an independent body (INEGI) and available to the public on the institution’s website. It also gave through its website free and open access to the calculation formula designed for the multidimensional measure. Moreover, this formula is available in different statistical packages to facilitate its use. This way, any researcher can easily verify the figures provided by CONEVAL and guarantee the transparency and accuracy of the information reported.

 

CONEVAL’s Consolidation

Once the initial phase of methodological definitions and data collection was over, CONEVAL continued strengthening its evaluations of social development policies and programs and its measurement and analysis of poverty from a multidimensional perspective. Regarding evaluation, CONEVAL is dedicated to independently monitoring program results, identifying problems and suggesting concrete actions to improve processes. It also defines the evaluation guidelines that entities responsible for social programs must follow.

As mentioned in the introduction, CONEVAL has published more than 1,500 evaluations of social policies and programs, and four reports on social development policy. In turn, CONEVAL’s poverty measures have been used in public policy in various ways since 2012: helping establish goals, objectives, and strategies under the framework of the National Social Development Policy; guiding the National Strategy for Inclusion; informing the budgetary process; helping in the identification of beneficiaries of the National Crusade Against Hunger; and as the base formula of the Social Infrastructure Contribution Fund (FAIS). In addition, CONEVAL offers courses and consultancies on its issues to other state and civil society organizations, and has collaboration agreements with federal entities and other national and international agencies.

 

Decentralization of information on multidimensional poverty

The mandate given to CONEVAL by the Congress of the Union to obtain figures that can be broken down by territory resulted in a significant achievement, providing for the first time clear maps of poverty by subnational entities through an official measurement estimated by an independent agency. This decision had a very relevant additional effect: the governors of the country’s states demanded multidimensional poverty information.

The periodic estimates made by CONEVAL showed not only each state’s poverty situation but also exposed the behaviour of poverty rates at the state-level over time.[7] In addition to visualizing a state’s situation compared to other states, these measurements evaluated and publicized the results of states’ poverty reduction programs. Because of this, some governors started paying attention to CONEVAL’s results and requesting clarifications of the situation and consultancies on how to direct their social policy to obtain better poverty reduction results in the biannual measurements. Some states even began to develop working teams to do their own multidimensional poverty analysis and evaluations of their policies.

Today, CONEVAL provides counselling and training to the states on poverty and evaluation issues based on specific demands.[8] This work has led to several results. According to CONEVAL, governors not only talk about “poverty” but also about the “causes” of poverty. Additionally, the multidimensional measure has helped organize social policies by focusing on the topics prioritized by the index, and multidimensional poverty analyses have started to appear in the National Development Plan.[9]

The work at the municipal level (2457) using multidimensional poverty measurements has been slower, but it is already showing important results. For example, resources from FAIS, one of the most important municipal and federal entities’ financing fund, are assigned on the basis of CONEVAL’s poverty measure.[10] Furthermore, in recent years this fund has been modified to allocate resources to municipalities also according to their contribution to mitigating the social deprivations of the population living in extreme poverty. FAIS assigns specific percentages for achievements in poverty reduction, which greatly incentivizes the measure’s use and improves the living conditions of the population and the exercise of their social rights.

 

Conclusion

CONEVAL’s creation and the application of Mexico’s multidimensional poverty measure have had many important achievements. To begin with, they managed to establish and consolidate a solid institution for the measurement of poverty and the evaluation of social development policies. Within ten years, Mexico shifted from having unofficial poverty measurements made by the same actor in charge of carrying forward the social policies to being the first country in the world to have an official multidimensional poverty measurement with an autonomous entity in charge of measuring poverty and evaluating social policy. The measurement’s creation was, moreover, a product of a very rich consultation process amongst national and international experts, as well as a comprehensive discussion with all the national agencies and institutions responsible for social policy. All this, together with the transparency and replicability of the information, helped to begin addressing the credibility problem around poverty figures and improve social policy at federal, state, and municipal level.

 

Chart 1: Chronology of the Development of the Multidimensional Poverty Measure Methodology

 

 

Suggested Readings

National Council for the Evaluation of Social Development Policy. (2010). Metodología para la medición multidimensional de la pobreza en México. México, DF: CONEVAL.

National Council for the Evaluation of Social Development Policy. (2016). “¿Cómo se logró construir la medición de pobreza del CONEVAL?”

Mora, Minor (coord.). (2010), Medición multidimensional de la pobreza en México. México, El Colegio de México/CONEVAL.

 

[1] For a review of the different subjects discussed during this process, see Székely, Miguel (ed.) (2005), Números que mueven al mundo: la medición de la pobreza en México, México, Miguel Ángel Porrúa.

[2] The dimensions defined by the law were the following: current income per capita, average household educational gap index, access to social security, housing quality and spaces, access to basic services at home, access to food, and degree of social cohesion.

[3] For a detailed description of the process of obtaining this definition, see “Cómo se logró construir la medición de la pobreza en CONEVAL?” Among the specialists consulted during different stages of the process are Sabina Alkire, Louis-Marie Asselin, Julio Boltvinik, François Bourguignon, Rodolfo de la Torre, Jean-Yves Duclos, Juan Carlos Feres, James Foster, Samuel Freije, David Gordon, Enrique Hernández Laos, Luis Felipe López-Calva, Clara Jusidman, Nanak Kakwani, Shahid Khander, Gladys López-Acevedo, Nora Lustig, Xavier Mancero, Susan Parker, Rafael Pirez, Sanjay G. Reddy, Luis Rubalcava, Rosa María Rubalcava, Javier Ruiz, John Scott, Emmanuel Skoufias, Erik Thorbecke, José Vences, and Michael Walton.

[4] For a review of the different proposals, see Mora, Minor (coord.). (2010). Medición multidimensional de la pobreza en México, México, El Colegio de México/CONEVAL.

[5] For a description of the Mexican methodology, see National Council for the Evaluation of Social Development Policy. (2010). Metodología para la medición multidimensional de la pobreza en México. México, DF: CONEVAL.

[6] See the conversation with Gonzalo Hernández Licona, Executive Secretary of CONEVAL, in Dimensions magazine, 2016.

[7] For more information on CONEVAL documents with other federal entities, see http://www.coneval.org.mx/InformesPublicaciones/InformesPublicaciones/Paginas/Colaboracion-con-Entidades-Federativas.aspx.

[8] For more information on the different types of collaborations CONEVAL has had with the different federal entities, see http://www.coneval.org.mx/InformesPublicaciones/Documents/CONEVAL_Entidades_Federativas_v04_web.pdf .

[9] Interviews with Gonzalo Hernandez, Executive Secretary; Édgar Martínez, Deputy Director General of Coordination; and Nereida Hernández, Technical Services Manager.

[10] FAIS Guidelines.