Household Survey and National Statistics
No matter how ambitious international global goals may be the most important efforts towards poverty reduction will no doubt be down at the national level. Successful assessment of poverty progress will only be possible if recipient countries assume ownership of the process because the responsibility to monitor poverty progress is ultimately of individual countries. To understand if poverty has increased or decreased authorities rely mainly on household surveys that provide consumption and expenditure data that are used to construct poverty profiles. Over the past two decades there has been a significant improvement in the availability of consumption data which has supported the remarkable poverty reduction achievements. Starting with 22 surveys in 1990, the world counts today with 1000 household surveys (Ravallion, Datt, and van de Walle 1991). But albeit this privileged accumulation of knowledge, much has been said on aid coordination but not much has been done in terms of harmonization of data and there are serious challenges in household surveys that need to be overcome in the future to properly monitor poverty. One fundamental condition to infer if poverty has decreased or not, is to have comparable measures of well-being at multiple points in time. But experience has shown that albeit the improvements, the heterogeneity of instruments (for e.g. questionnaires) and methodologies used in surveys can seriously dampen their quality jeopardizing the ability to compare results rigorously. Lack of consensus in survey design for example in questionnaires can have serious consequences in terms of comparability. The questionnaire is the soul of a survey, what we ask is what we get, so changes in questionnaires have dramatic consequences in the capacity to compare poverty throughout time that can lead to misleading conclusions and doubt if poverty has indeed increased or not. For example research in data collection has widely established that factors such as the recall period and the number of food items listed have a large effect on the consumption estimated. For example findings from Beegle in Tanzania show that comparing different recall period to collect data if we increase the recall period of personal daily diary from one week to two weeks poverty also increases from 55% to 63% (Beegle 2012). So sensitivity analysis has shown that the more we increase the recall period of questionnaires the higher are poverty estimates. On the other hand short and collapsed lists of food categories compared with long detailed list of food consumption have greater poverty estimates. In fact on average a 7 day recall with a long list of food items performs better compared with more expensive and onerous methods working as the gold standard. (Beegle et all, 2012)
The implication is that innocuous changes in the survey design have significant impact on poverty estimates, so any change in data collection methods should be looked at with caution to avoid difficulties in comparability and mis-evaluations of poverty. This does not mean that customizing surveys to specificities of the country is an unrecommendable practice, for example tailoring questionnaires with a list of food that reflect local consumption patterns will result in good quality of the measure of food security, but arbitrary changes of household surveys over time will result in spurious estimates of change of absolute poverty. Indeed reality is that in many countries there is no systematic effort to collect and distribute survey data. Most household surveys are collected on an ad hoc basis driven by specific requests of governments or ministry and depending on the availability of donor´s funding. Indeed these very expensive data collection initiatives craving for donors’ funding tend to be customized to donors’ strategic interests rather than produced systematically in a rigorous way. Furthermore new instruments and variables can also be used just due to a turnover of cabinet personnel willing to change questionnaires to improve informational data forgetting the cost they incur in data comparability. Pure administrative issues related with the quality of the training, supervision, enumeration and data entry can also undermine the reliability of the results of household surveys. Additionally as poverty is crucial to the agenda of most developing countries not surprisingly questionnaires can also be changed to accommodate desired political prospects. There are also issues of confidentiality that make access to survey data restricted even in those cases where survey data has been properly collected and compiled. There is great heterogeneity across countries as to when and to what degree the data are made available to analysts outside national statistics offices which makes poverty estimates although well produced not accountable for monitoring purposes. In other situations the time gap between the fielding of household surveys and the release of the data for analysis makes estimates irrelevant or inconclusive. Many reasons may explain these delays. The lengthy process may be a plausible explanation but sometimes the release of poverty estimates can also be managed according to the political agenda namely elections, particularly if poverty is likely to increase or stagnate.
Another issue that is usually overlooked is also the importance of the timing of the survey. 80% of the worlds’ poor reside in rural areas and most of the poor depend on agricultural activities that typically are seasonal. So not surprisingly in most developing countries welfare fluctuates according to seasonal patterns making poor better-off during harvest periods and worse off in lean season. From this follows that if fieldwork for data collection occurs one year during lean season and in the other after harvest it may lead to the mis-perception that poverty has reduced considerably over time when in fact it reflects the seasonal cycle of poverty and not necessarily improvement over the years.
But comparability challenges do not occur only over-time but also within the country. Spatial differences in the cost of living can be dramatic inside one country. Cost of living can be the double or more in the capital compared with rural areas and in addition prices usually differ according to regional patterns. So failure to accommodate the differences in cost of living will result in mis-identification of the poor particularly what concerns the geographic profile of poverty.
Due to panoply of reasons there is lack of a consistent and predictable flow of new household data. In some cases they are not available or come in late or in other situations they are not reliable because they are product of choices and implementation decisions that seriously dampen comparability across countries and throughout time. These are serious challenges that demand an exhaustive assessment and evaluation of each country’s respective household data to guarantee proper calculation of global poverty.
In the last 30 years if it is true that Povcalnet database has accumulated an impressive stock of more than 1000 surveys representing already 129 countries that only correspond to 20 to 40 new surveys available annually. One of the big challenges of monitoring global goals is that it is necessary annual household survey data but it is not realistic to imagine this occurring in the medium term. What do we do meanwhile? One way to move forward is to improve data collection exploiting new software and technologies. Traditionally household surveys were implemented based on Pencil-And-Paper-Interviewing (PAPI), but advances in mobile technology namely in Computer Assisted Personal Interviewing (CAPI) software provide a viable alternative. Comparing both interviewing methods CAPI significantly reduces the variance of consumption and increases the mean reducing poverty measures (Caeyers, Chalmers and De Weerdt, 2011). CAPI main advantage is that it reduces the time lag between data collection and data analysis but also allows automatic checks and quality control at the entry point.
Cell phones represent also an alternative opportunity in data collection. They will not be able to completely substitute the lengthy interviews of LSMS that take 12 months, but may serve as a potential annual monitoring tool for quality control and follow-up. There are several advantages: rapid collection of high frequency and wide data; cost-effective, flexibility on question formulation; minimization of respondents fatigue reduces attrition and non-responses. (Croke and others 2012). Hybrid initiatives such as the WB Listening to Africa providing a face-to-face baseline survey followed by phone interviews give precious data about the dynamics of poverty.
GPS instruments can also be very useful because they will allow tracking extreme poor located in remote areas or disconnected from markets or other services. Distances of land size normally based on self-reports can be updated rigorously. If issues of confidentiality are addressed by for example collecting data based on enumeration area geocoding can point-track households and improve understanding of access to services, seasonal migration patterns and real-time vulnerabilities establishing innovative causal relationships of poverty based on surgical data.
 For example in the case of China access to microdata is restricted, but aggregated data on the distribution of consumption is published in official statistics reports. It is possible to estimate poverty indirectly but under additional assumptions.