Data error prompts process improvements
27 February 2014
Data error prompts process improvements
The Treasury and Statistics New Zealand are improving their quality assurance processes after data miscalculations affected estimates of disposable income, Treasury Chief Economist Girol Karacaoglu and Statistics New Zealand Deputy Government Statistician Vince Galvin said today.
Estimates of household disposable income (income from all sources after deducting income tax and adding tax credits) were miscalculated for 2007/08 and for 2009/10 to 2011/12 in a way that tended to overestimate the disposable incomes of households on low to middle incomes.
The problem is limited to this particular area of work and has no bearing on other Treasury or Statistics New Zealand work streams or data. Policy advice to the government is unchanged, and there has been no impact on benefit payments or taxation for individuals or households.
“The Treasury and Statistics New Zealand hold themselves to very high standards. We are deeply disappointed to have failed to meet those standards with these data calculation errors,” said Dr Karacaoglu.
“The miscalculations were human error. But the main problem was that communication protocols between the Treasury and Statistics New Zealand were inadequate for this data and neither agency had an overview of the complete process to ensure the anomalies, once identified, were properly explained. Both agencies are determined to fix that,” said Mr Galvin.
“Statistics New Zealand and the Treasury have changed their quality assurance and communications processes to ensure this problem doesn’t happen again. These new processes will be in place for the next round of estimates for 2013, which begin in March.”
The impact on two major pieces of work was assessed in detail:
• In the Ministry of Social
Development’s Household Incomes Report, the trends and
most of the findings remain broadly valid. The main
difference is that the revised figures show that the global
financial crisis had a greater one-off impact on household
incomes in 2009 than the original data showed. This was
especially the case for lower-income households with
children in private rental accommodation.
• Estimates used by the Tax Working Group in 2009 and by the Treasury in Budget 2010 are unchanged within statistical margins of error. The main impact is that corrected estimates show the increase in the number of children in households with less than 50% of median income after the Budget 2010 tax changes is less than previously advised.
“We recognise that issues of estimates of household disposable income are important and we considered carefully what impact the corrected data may have had on the Treasury’s policy advice,” said Dr Karacaoglu. “We concluded that that advice would not have changed, as the effects of the miscalculations on the data we used are too small to have altered our judgements. We consulted with the chair of the Tax Working Group, Professor Bob Buckle, who agreed that the impacts were not material and would have been most unlikely to have changed the Tax Working Group’s findings.
“It is also important to note that there are no ‘real world’ impacts on New Zealanders from the miscalculations. There is zero effect on the individual or household benefit payments and tax credits people get, or the tax they pay,” said Dr Karacaoglu.
The table below summarises the key miscalculation impacts.
|Miscalculation||Scale||Key statistics affected|
|Accommodation Supplement was double-counted in estimates of disposable income for 2009/10, 2010/11 and 2011/12.||$1.2bn||The largest differences are
for families with children. For example:|
the revised 2011/12 rate is 14% rather than 12% of children (150,000 rather than 125,000) in households with less than 50% of median income
using the 60% of median after housing costs measure, the revised 2011/12 rate is 27% rather than 25% (285,000 rather than 265,000).
The proportion of all households with a high ‘outgoings-to-income ratio’ (OTI) increases from 26% to 27%, with lower income households even more likely to have high OTIs, increasing from 38% to 41%.
|Working for Families payments were
overestimated in Treasury estimates of disposable
Accommodation Supplement payments were underestimated in Treasury estimates of disposable income in 2007/08.
combination these miscalculations had no statistically
significant impact on estimates of the number of households
on estimated incomes below 50% and 60% of median
Approximately 11,000 fewer children were in households below 50% of median income after the Budget 2010 tax changes than previously estimated. No statistically significant change to the 60% measure.
The estimates of household disposable income are jointly produced by Statistics New Zealand and the Treasury, and involve modelling of Statistics New Zealand’s Household Economic Survey (HES) figures by Treasury using its Taxwell micro-simulation model. The Taxwell-generated estimates are then added by Statistics New Zealand to the HES customised dataset before being sent on to the Ministry of Social Development and others. Through the Ministry of Social Development’s Household Incomes Report, analysis of this data has become the basis of almost all detailed analysis of income distribution in New Zealand.
The Treasury and Statistics
New Zealand have conducted two thorough reviews into the
• how the errors occurred, including the internal and co-production processes used by Statistics New Zealand and Treasury
• the Taxwell dataset and associated documentation
Both of these reviews have been externally peer-reviewed by Deloitte. All of the reviews are available at http://www.treasury.govt.nz/publications/informationreleases/disposableincome.
The Ministry of Social Development has published on its website corrected figures for the most commonly used statistics, together with a short commentary on the impact of the revised data.