Kyoto Protocol: Review Of Economic Modelling
Pete Hodgson, Convenor of the Ministerial Group on Climate Change, today released a review of two significant economic models used to analyse the possible effects of ratification of the Kyoto Protocol.
The review has been released in response to an Official Information Act request. It has also been supplied to Parliament’s Foreign Affairs, Defence and Trade Committee, which is examining the government’s National Interest Analysis of ratification of the Protocol, in response to requests from committee members.
Commissioned by the government’s Climate Change Project, the review by Arthur Grimes of GT Research and Consulting investigates economic modelling by the Australian Bureau of Agricultural and Resource Economics (ABARE) for the government, and by the New Zealand Institute of Economic Research (NZIER) for the Greenhouse Policy Coalition and the Petroleum Exploration Association of New Zealand.
AN INVESTIGATION OF THE ABARE & NZIER MODELS USED
TO ANALYSE EFFECTS OF THE KYOTO PROTOCOLS
ON NEW ZEALAND
GT Research & Consulting
10 February 2002
The purpose of this report is to investigate the nature of the models used by ABARE (Australian Bureau of Agricultural and Resource Economics) and NZIER (New Zealand Institute of Economic Research) to analyse the effects on the New Zealand economy of New Zealand ratifying the Kyoto Protocols. The report assesses the suitability of the models for that task. It then suggests ways in which one of the models could be extended to take additional factors into account that may be relevant to, and so further improve, the analysis.
This comparison of models is important in order to give comfort that the model results present a reasonable assessment of the effects on the New Zealand economy of ratifying Kyoto. The comparison is even more important given that the two models appear to give very different assessments of the effects of Kyoto on New Zealand economic outcomes. While the range of scenarios investigated in the two reports vary, the overall conclusion of the ABARE simulations can be taken to imply that Kyoto will have only relatively minor economic effects in aggregate (albeit with some substantial sectoral effects); in contrast, the NZIER assessment can be taken to imply that Kyoto will have strongly negative effects on New Zealand economic outcomes, particularly on GDP over time.
Differences in the scenarios’ assumptions (e.g. on international emission permit prices, industry coverage, inclusion of different countries within the protocol, etc) can explain some of the differences in results. However, the generally more negative conclusions of the NZIER results relative to the ABARE results is likely to be due, at least in part, to different modelling approaches or to differences in the properties of the two models. For this reason it is important to assess which tool (if either) is more suitable for the task of assessing Kyoto’s effects on New Zealand.
The approach adopted here has been to use information presented in the reports of the two agencies. This material has been supplemented by personal interviews with personnel of the two organisations plus additional material supplied by the agencies to gather further detail.
A brief description of the two models is presented below. Most emphasis in the report is placed on particular issues which indicate suitability or lack of suitability for the task at hand. In the ABARE case, just one model has to be assessed; in the NZIER, the two models used in their report are assessed.
The report does not comment specifically on the scenario assumptions within each of the reports. It is an investigation of the tools rather than of the inputs. Thus no comment is made about the reasonableness of the final results (which depend both on the tools and the scenario inputs). The report can, however, be used to determine which tool (or variant of a tool) is best placed to analyse scenarios based on given policy assumptions.
2. ABARE REPORT
The ABARE report uses the GTEM model. GTEM is an international, dynamic 33-sector general equilibrium (GE) model with considerable disaggregation in industries (including agriculture) that will potentially be affected in a major way by the Kyoto protocol. It is a standard GE-type model with considerable documentation available (including on the website: www.abareconomics.com).
In keeping with other GE models, GTEM assumes that each of capital, labour, land and natural resources are used in production of goods and services in each sector of the economy. Prices adjust to ensure full employment of resources. Hence it is an equilibrium model which does not attempt to model transitional effects of policy and so cannot model “temporary” unemployment or other disequilibrium phenomena arising from policy changes.
The standard GTEM model has been extended to include natural resources in order to model Kyoto issues. To extend the model, ABARE uses the GTAP database (version 4.0e). As noted in the NZIER discussion below, the data which ABARE uses for New Zealand (e.g. input-output weights) is not fully up-to-date and may miss some points that are country-specific to New Zealand. This may have some effects on its results.
Emission response functions (ERFs) within sectors are modelled. The ERFs enable reductions in carbon dioxide emissions through energy efficiency improvements and fuel switching and from reductions in noncombustion sources through adoption of new technologies and management practices. These responses occur as a reaction to price changes. This is an important feature to model.
Overall, subject to the points discussed below, the model is appropriate for a long-run analysis of the issues at hand.
Specific Points Related to the ABARE Model
- Each of the scenarios is modelled relative to a reference case where the Kyoto protocol is not ratified. Overall, the reference case looks reasonable; export and import growth rates appear a little low (at approximately the same rate as GDP growth, whereas historically the rate of trade growth is higher) but this should not lead to any obvious bias in the scenario results one way or another.
- It is important to keep in mind that the ABARE results are long run. While the tables refer to results for 2010, there is nothing in the model which suggests that the long-run transitions depicted will be completed by 2010. Indeed, given many countries’ experiences with major relative price changes after deregulation (including in New Zealand) it is likely that the new long-run equilibrium industry configuration could take 10-20 years to complete, with many downside adjustments occurring faster than many upside adjustments. Thus the present value of the costs of adjustment could well exceed those documented in the ABARE report.
- Land price effects are incorporated in the ABARE model, which is entirely appropriate and necessary, especially for agricultural industry reallocation. However no detail is given on the size of the land price adjustments. This information is important in assessing whether there would be major bankruptcies occurring (especially in agriculture) and hence whether there would be considerable pressure on the banking system. It is now recognised that the macroeconomic effects of banking problems (caused by factors such as major land price declines) can be severe and long-lasting (e.g. the economic effects of the Asian financial crisis, New Zealand and Australia after 1987, and the Savings and Loans crisis in the USA in the 1980s). If agricultural land prices fell significantly as a result of Kyoto (this depends on how the protocols are implemented in New Zealand) the negative output effects on the whole economy could be considerably worse than modelled by ABARE. (Some government intervention as occurred successfully after the major land-price declines precipitated by the removal of agricultural support in 1984/85 might then be required to prevent such macroeconomic problems.) This issue is unlikely to be germane if agriculture is exempted from Kyoto measures.
- ABARE does not model the effects of new forestry plantings in generating new carbon sink entitlements. However, depending on the nature of the actual policies implemented by government, owners of new plantings may be “rewarded” with emission credits which would act to increase plantings relative to the effects modelled here. This incomplete modelling of the positive effects for forestry of the effects of carbon sinks linked to new plantings is likely to lead to an understatement of the long-run benefits of the Kyoto protocol on forestry and the New Zealand economy generally.
- The assumption that extra government revenues (from the imposition of emission abatement policies) are distributed as a lump-sum tax back to households also leads to an understatement of the benefits of the protocol for the New Zealand economy. In practice, increased revenues would be used to lower existing distortionary taxes (e.g. on labour income, company profits or GST). This lowering in other distortionary taxes would help to bolster economic activity and reduce existing distortions within the economy. If government revenues from emission credits were significant, this factor could assume major importance (and is an element in the “double dividend” environmental literature).
Overall, the ABARE results indicate fairly minor economy-wide effects of the Kyoto protocols on macro-economic variables, but some substantial industry-specific results. This is similar to what has been observed from large tariff changes in the past and so appears reasonable. Again, however, the historical experience is that positive adjustments of sectors not originally “hit’ by the policy change can take considerably more time than the negative impacts on directly affected sectors. Also the effects on the banking system caused by major land price declines (only in those scenarios where these are predicted to occur - i.e. where methane charges are included) could exacerbate the negative short-run adjustments. By contrast, the model appears to under-estimate some long-run positive economic effects (arising from positive effects on forest plantings due to sink credits and from the beneficial economic effects arising from cutting distortionary taxes). Overall, there is no clear direction of bias caused by this range of issues.
3. NZIER REPORT
NZIER uses two modelling approaches: a static general equilibrium model (similar, in principle, to the ABARE model, albeit based on a static rather than dynamic approach) and an econometrically estimated equation (or system of equations) which aims to model a production function with output explained by inputs of labour, capital and energy. The two modelling approaches are quite separate and these approaches need to be examined separately.
In discussing the results, NZIER reports results from each separate model and reports combined results using effects of both models. One model (the GE model) is a reasonable attempt to address the issues and the results of that model deserve attention. There are issues attached to this model which are documented below, but in principle these issues could be overcome and in some respects they provide an important contrast to alternative assumptions within the ABARE model.
By contrast, as discussed further below, the econometric approach has no credibility whatsoever. There is virtually no documentation of these equations available and no staff member is able to discuss, even verbally, what lies behind them. The statistical estimates and data are not available to enable an independent reviewer to ascertain what has been done and how reasonable the results are. Quite simply, in the circumstances they should not have been published and/or used in the study. The results of that approach, and of combined results incorporating that approach, should be entirely disregarded, at least until such time as NZIER fully documents this work and has had it externally peer-reviewed.
Specific comments on the models follow.
Specific Points Related to the NZIER GE Model
- The NZIER GE model is a newly compiled model for which this is its first assignment. As such, it does not have the advantage of comprehensive documentation available for the ABARE model. Some documentation is available in Appendix A of the NZIER report and, while terse, this is a reasonable attempt to explain the content of the model. However, no justification is given for many of the assumptions underlying the model, e.g. that the consumer trades off a composite energy bundle of goods with a non-energy composite bundle; that the income elasticity of all demand for all non-energy goods is unity; that the elasticity of substitution between the composite energy good and the value added bundle for most sectors is 0.5; that the elasticity of substitution between primary factors is unity (whereas other NZ work suggests this should be lower). It would be useful in a new model such as this to document the bases for these assumptions more fully. This is not to say that the assumptions are unreasonable; but it is difficult to assess the reasonableness of all the assumptions without further documentation.
- NZIER uses more up-to-date input-output weights than does ABARE, and it also takes account of the fact that natural gas is not piped to the South Island, unlike ABARE. Given the increased share of agriculture in output over the past decade and given the increasing importance of the South Island dairy industry (a major coal user), inclusion of each of these factors is likely to make the NZIER results an improvement - in this respect - on the ABARE results. It is difficult to judge the magnitude of this effect without further modelling using sensitivity tests.
- The NZIER and ABARE models incorporate different elasticities of substitution between domestic and imported goods. This affects the results. ABARE uses international, historically-based assumptions which may be on the low side for an open economy such as New Zealand. NZIER uses considerably higher assumed elasticities (based on judgement) which, if anything, may be on the high side. Thus the two studies seem to set the bounds of reasonableness for these assumptions; but neither can be considered unreasonable. The empirical work has not been done in NZ at the required level of disaggregation to ascertain better estimates of these elasticities. Sensitivity tests using either or both models would be helpful to uncover the magnitude of difference to the results that occurs as a result.
- The modelling of the cement and iron & steel industries incorporates specific threshold effects which allow for the industry closing down once output falls below a certain level. This contrasts with the ABARE approach which assumes smooth adjustment. The NZIER approach appears quite reasonable for these industries (and this may be a worthwhile addition to the ABARE model), although the reasonableness of the exact specification of the threshold effects is difficult to judge. (For instance, the graph on page 60 indicates that the price of Portland cement is currently approximately 40% below the import price, suggesting that there is quite a margin in costs that needs to occur before the shut-down scenario occurs. By contrast, from the results in Table 19, only a very small margin is allowed for before shut-down occurs.)
- As with ABARE, any revenues gained by government from carbon sink entitlements are redistributed back to households through lump-sum taxes. This ignores the beneficial effects that can be gained by reducing other distortionary taxes and so leads to an overstatement of the costs of Kyoto under some scenarios.
- Also, as with ABARE, NZIER does not model the effects of new forestry plantings in generating new carbon sink entitlements. This is also likely to lead to an understatement of the long-run benefits of the Kyoto protocol on forestry and the New Zealand economy generally.
- Unlike the ABARE report, there are no explicit emission response functions included (although some substitution is allowed between land, labour, capital and energy). For instance, it appears that no response for agricultural emissions of methane and nitrous oxide to a methane charge is included (see page 16), thus implicitly assuming that there are no technologies available currently or in future to reduce these emissions. This may be reasonable for a short to medium term horizon, but not longer term. Thus the results presented are more “medium term” than long run results.
- For the other sectors, since emission response functions are not included, this will lead to an overstatement of the negative effects of Kyoto since ability to adjust within a sector to a less emission-intensive production mechanism is diminished, thus increasing the need to make adjustments between sectors. As an example, discussion on page 55 notes that the wood products sector should be able to save significant amounts of energy through better utilisation of wood waste and waste heat, but this is not allowed for in the modelling. As another example, discussion on page 52 notes that drying milk is energy intensive. It is likely therefore that the dairy industry would shift (at least at the margin) towards more cheese and less milk powder output if energy rises in price, but such a switch is not allowed for in the modelling. As another example, the model does not allow for switches in the transport industry to vehicles with different emissions intensity, but this is a switch that can be expected to occur as energy prices rise.
- It is interesting to note that in the NZIER GE model, energy is solely an intermediate good in the production function (primary factors being land, labour, capital). Thus energy does not contribute to value added - i.e. to GDP (see page 82). This seems reasonable for most sectors.(To the extent that the documentation allows, it appears that this in direct contradiction to the NZIER’s econometric approach.)
- While not specific to the underlying models, I note that much of the NZIER discussion is difficult to follow, and some is potentially misleading. In reporting the GE results, NZIER reports gross output changes rather than value added (contribution to GDP) changes. It is the latter, not the former that is more relevant for New Zealand’s welfare. As noted in the previous bullet point, since NZIER assumes that energy does not contribute to value added in most industries, a change in energy prices can be expected to lead to a drop in gross output (as energy input declines) but to little change in value added. Thus NZIER would be better to discuss value added (GDP) changes rather than gross output changes. It is unclear why they do not do so. Because gross output figures are misleading indicators of welfare changes and because GDP changes are not reported, the best reported figures within the NZIER document to concentrate on in interpreting the outcomes for the NZ economy are those presented for “household consumption”. These figures essentially represent long-run income changes for the country.
Specific Points Related to NZIER’s Econometric Work
- The econometric results (section 2.3 and Appendix C) are extraordinary for their lack of detail. There is no detail provided as to the nature of the production function that has been estimated (e.g. whether it is modelling gross or net output, whether it nests value added within gross output, etc), there are no details on the data sources used, and there are no details on the econometric results. Without access to this detail, none of these results can be given any credence.
- No discussion of how the estimated function(s) relate to other New Zealand empirical work in this area (including recently published Treasury work). This is normal practice to allow the reader to judge the reasonableness of the approach adopted.
- No detail is provided as to how the estimated sector and aggregate production functions match up with the production functions used in the GE model. In the GE model, energy does not contribute to value added, but it is impossible to tell whether this is the same in the econometric model.
- The estimates are all based on just 20 observations (1980-1999). This is a small sample and (depending on the econometric results) is likely to make for quite large confidence intervals around the estimates. Nowhere is this discussed.
- It is stated that an aggregate function is estimated and sectoral production functions are estimated. Nowhere is it stated whether the aggregate function is consistent with the sum of the sectoral functions.
- It is likely that the estimates treat all vintages of capital as the same in terms of energy intensity. However, consistent with the discussion on page 22 of the NZIER document, a “vintage-capital” model should be used to allow new vintages of capital to have different energy intensities and in-built technology than existing vintages. This would have to be taken into account in econometric work for the purpose of modelling the effects of ratifying Kyoto.
Overall, the NZIER’s GE model is a reasonable starting point to model the issues at hand, although it is previously “untried” and the write-up is not fully complete. Some aspects also are not entirely appropriate for modelling Kyoto issues, particularly the lack of emission response functions (unlike ABARE). In other respects, particularly the more up-to-date input-output coefficients and the specific treatment of South Island energy choices, the NZIER model has some advantages over the ABARE model.
In contrast, no attention should be paid to the NZIER’s econometric work or to any discussion based fully or in part on it. It is simply not credible.
Some of the discussion in the NZIER report could be interpreted as being more demanding on the “pro-Kyoto” stance than on the “anti-Kyoto” stance. An example is the straw man on page 35: “The double dividend could theoretically occur where ¡K revenues from [the environmental] tax are sufficient to finance the entire public budget, and hence eliminate all distorting taxes”. For the double dividend to occur, it is not necessary that the environmental tax revenues finance the entire public budget; all that is needed is that they replace some of the existing distortionary taxes. As another example, the discussion about whether leakage could be lessened is one-sided and is odd in assuming that if New Zealand were to impose a “carbon tariff’ this may wreak havoc with the world trading system. This is clearly a gross exaggeration which ignores the usual small country assumption. A third example concerns the sensitivity tests which varies the size of sinks. The study uses 20 Mt as the base-case and then models only the effects of 5 Mt and 10 Mt rather than adopting sensitivity tests which are symmetric about the base-case. This unorthodox approach to sensitivity testing is not justified in the text.
4. COMPARISON OF ABARE & NZIER GE MODELS
The ABARE & NZIER GE models each has strengths and weaknesses. The ABARE model has a well-established pedigree, although it is not specific to New Zealand. In contrast, the NZIER model, while previously untried, attempts to incorporate assumptions specific to New Zealand, including use of more up to date input-output data, and takes account of the lack of natural gas in the South Island. It suffers, relative to ABARE, however, in not incorporating explicit emission response functions. Also it is unclear whether it is able to report value added changes given that these figures do not appear in its report.
The two models differ substantially in some of their elasticity assumptions, especially relating to the Armington elasticities determining consumers’ abilities to substitute between domestic and foreign goods in the same sector. Neither model appears to have unreasonable estimates and sensitivity analysis is required to test the effects on results of the differing estimates.
Neither model incorporates the effects of: new forestry plantings on additional sink credits, land prices on the financial sector and thence on to economic activity, transitional effects, or government reducing distortionary taxes as a result of sink credit revenues. These effects work in opposing directions and, for each model, it is therefore impossible to conclude on the direction of any bias in the results arising from omission of these factors.
5. FURTHER MODELLING APPROACHES
Given that the ABARE model is based on a well established GE model and incorporates emission response functions, it would be useful to use this model to conduct further analysis of the effects of Kyoto ratification.
However, it would be extremely useful to conduct sensitivity tests of its results using, as inputs, some of the approaches used in the NZIER model. In particular, for any given scenario (preferably one close to the government’s most likely policy option) it would be useful to compute the results of ratificiation based on the following model variants:
1. The base case model as used in the previous ABARE work.
2. Base case using up-to-date (NZIER’s/Statistics NZ’s) input-output coefficients.
3. Base case using NZIER’s approach for energy substitution for the South Island.
4. Base case using threshold effects in cement and iron & steel.
5. Base case using higher Armington elasticities to proxy NZIER approach.
6. Base case with ability to generate additional sink credits through new forestry plantings.
7. Base case with government cutting distortionary taxes to recycle extra revenues.
(Note that it is unlikely that the model could handle transition effects or effects of land prices on the banking system.)
After further consideration of the reasonableness of these various approaches, some combinations of variants could be considered. For instance, a combination of cases 2, 3, 4, 6 & 7 could possibly be considered a “best practice” approach and could be well worth applying to investigate the effects of likely policy options.