2018 Census External Data Quality Panel
23 September 2019
The Census 2018 External Data Quality Panel (the panel) today released the first of two independent reports on the quality of data from 2018 Census. The second will be published in December.
One in six New Zealand residents did not complete a questionnaire for 2018 Census, largely due to operational failures that made it difficult for them to participate. To address high levels of non-response, Stats NZ embarked on a large-scale census mitigation process that involved, for the first time, the extensive use of alternative government data to fill gaps in the census dataset.
The panel’s initial report contains its findings on the robustness of the methodologies used to complete the final census dataset and the quality of the first release statistics. Established by the Government Statistician in August last year, the eight member panel includes experts on census methods, statistics, Māori data, demography and equity.
Key findings from this initial report include:
• The panel endorses the statistical approaches used to mitigate non-response. It finds that the use of administrative and 2013 Census data has improved the quality of results by reducing the extent of the undercount that is a feature of all censuses of population. This was especially the case for Māori and Pacific ethnic groups in the population.
• However, the unprecedented use of alternative government datasets to augment census data raises questions around ethics, social licence (i.e., tacit approval from the New Zealand public), cultural licence (collective mandate for the trusted use of Māori data), and Māori data sovereignty. While the panel has been advised of the statutory legitimacy of the record linking that has enabled the new methodology to be adopted, members remain unclear about the social and cultural licence for this linking.
• The panel is confident that the measure of the Māori descent population and the electoral population counts needed for electoral purposes meet the accuracy requirements.
• The panel also endorses Stats NZ’s quality ratings for the key statistical variables of age, sex, usual residence count, census night count, Māori descent, usual residence address, and census night address.
• The panel has taken a broader view of the needs of users of ethnicity data than simply the ethnicity variable itself. Members have rated the quality of ethnicity data as ‘moderate’, rather than endorsing the ‘high’ quality rating given by Stats NZ.
• There is significant variability in the quality of ethnicity data by ethnic group. This reflects different patterns of non-response, and the reliance on different alternative data sources. The quality of ethnicity data generally reduces as the level of ethnic and spatial specificity increases.
• The panel endorses Stats NZ’s decision not publish iwi data as official statistics due to insufficient data quality, noting in this regard that Stats NZ have not met their Treaty obligations to Māori.
In addition to these key findings on methodology and first release statistics, the Report notes that that there will be key limitations for users of 2018 Census data, including:
• The very different way the 2018 Census dataset has been derived creates a break in the time series for census data. With regard to ethnicity, comparisons with data prior to 2018 should be undertaken with extreme caution, particularly for Māori and Pacific ethnic groups.
• Household and families data will generally be of low quality and will not enable comparisons with 2013.
• The analyses of many population groups of importance to government, ethnic communities, local authorities, Māori and service providers will be affected by the lower responses to many of the census questions.
• Information that relates people to dwellings will be incomplete as just under eight percent of the population cannot be placed in a specific dwelling.
The panel will continue to assess the quality of 2018 Census data during October and November, and will publish a second report covering all the main variables in the census dataset by the end of 2019.