Peer Review Of Economic Analysis Prepared For The Regulatory Impact Statement On The Emission Trading Scheme

25 June 2009

1 Introduction

We have been asked to peer review the 20 May 2009 report by NZIER and Infometrics to the Ministry for the Environment entitled “Economic Modelling of New Zealand Climate Change Policy”. The report uses computable general equilibrium models (CGE) to assess the relative effects of various climate change policy proposals.

In this review, we ask the following questions:

Are the analytical tools used in the report appropriate to answering the relevant policy questions?

Are the policy options appropriately defined and represented within the model?

Are modelling results reliable and robust?

Do conclusions flow from the modelling results?

2 Are analytical tools suitable?

NZIER and Infometrics use computable general equilibrium models. Their report points out such models have become standard tools for the analysis of climate change policy options. For example, the Australian Treasury based its analysis on the outputs of a range of CGE models. The report also acknowledges that CGE models suffer from well known problems:

It is almost impossible to test them empirically. There is no way of knowing whether the previous studies have been correct, and hence, no way to assess whether the current study is likely to be empirically significant

They generally assume smooth adjustment to economic shocks. Hence, even if CGE model results provide a reasonable representation of where the economy would settle after it has gone through a full adjustment, they provide no useful information on how the transition would unfold.

While setting out these methodological problems, the report implicitly concludes that the models are worth using, and that their results are illuminating despite such weaknesses.

We strongly question this implicit judgement. It is true that under some circumstances CGE models can provide useful insights. Such circumstances would include:

When the decision requires an understanding of sectoral, rather than economy-wide, impacts. Despite their overall empirical weakness, CGE models are pretty much the only practical tool available to economists to track the diffusion of an economic shock through the flow of inputs into the outputs of various sectors

When the decision requires comparison of long-term outcomes, rather than tracking the evolution of the policy and managing the transitional phase

When the policy options are sufficiently distinct to be represented within the structure of the model.

Unfortunately, the decisions required with respect to climate change policy do not fall into these circumstances. First, the report tries to compare the economy-wide effects of various policy options. In effect, this means betting on the model’s weakness, rather than its strength.

Second, the key policy choices which the report is intended to inform relate to transition, not a choice between long-term equilibria. There is an almost universal consensus that a global emission trading scheme, in which all countries and firms participate, would be the least cost way of achieving global emission reduction targets. However, this is clearly a banal proposition from a policy point of view: there is no global carbon market in sight. While countries such as the US and Australia have announced an intention to introduce some form of carbon pricing, both are having difficulties in getting sufficient political support to implement the schemes. Perhaps even more importantly, New Zealand’s key emerging competitors in Asia are not yet even discussing pricing carbon.

The real question is how the New Zealand policy should evolve to move in line with the evolution of the global policy, and to manage the transition to a new equilibrium. Again, the CGE models are not particularly well suited to that task.

Finally, the CGE models are a coarse tool, which often does not allow decision-makers to differentiate between the real world choices they have to make. The problem is that it is quite difficult to represent subtly different policies within the inevitably simplified world of the model. In other words, the model is useful when the relevant policy choices can be coded into the model, but is not useful when they can not.

The most striking example of this is the acknowledgement in the report that the model can not tell a carbon tax and a cap-and-trade apart. The report relies on the theoretical equivalence between the two—the equivalence which would hold with perfect information and perfect foresight. As far as we are aware, there is a fair degree of consensus among economists that the assumptions required for the two to be equivalent are highly unlikely to hold in real life, and hence that the two policies would have quite different effects and pose different risks.

Overall, we believe that the analytical tool used in the report is not suited to the policy questions which the decision-makers are grappling with. Taken by itself, the use of the model masks, rather than illuminates the real policy choices.

We emphasise that our criticism relates to sole reliance on the CGE model to address complex policy questions. CGE model runs may well have been useful within a broader suite of analytical approaches.

3 Are the policy options appropriately represented within the model?

We understand that NZIER and Infometrics use different models. For the purposes of this peer review, we refer to a single “model”. This is because the issues which we address are handled similarly in both models.

We are concerned that the model used in the report does not appropriately represent the relevant policy choices. We have already highlighted the inability to differentiate between the carbon tax and the cap-and-trade. However, even where the model is able to differentiate between policy choices, it either does not, or it does not describe the policy options realistically.

For example, the report acknowledges that the model does not address the possibility that the price of international emission permits which the government is able to access may be lower than the price which would be available to the private sector. Yet, this is a critical issue for differentiating between policy options. Clearly, if the international markets for carbon permits have limited arbitrage, the ability of different market participants to access the lowest cost permits would make a critical difference to choosing who should face the carbon liability. In simple terms, if the New Zealand Government could buy AAUs in government-to-government transactions at substantially lower prices than private firms, such ability should be taken into account.

The structure of the CGE models allows different sectors to face different prices for the same input. Hence, the fact that this issue is not addressed in the modelling is a deliberate choice, not a simplification required of the model. This is an important oversight, which is likely to have an effect on model results

We are also concerned that the way the model represents free allocation of permits does not capture the economic essence of the policy. As the report explains, the model treats free allocation as a production subsidy. In other words, firms are assumed to take the lump sum value of free allocation and spread it across their entire output. The report acknowledges that this may not be a very realistic representation of firm behaviour, and suggests that an alternative modelling approach is possible. However, the conclusions in the report rely on modelling free allocation as an output subsidy.

We believe this is highly problematic, and is likely to understate leakage caused by the ETS substantially. There are two reasons for this:

The model assumes a binary choice for firms: to produce or not to produce in New Zealand. However, consider a more realistic possibility: that firms can manufacture partly in New Zealand and partly overseas. Under the simplifying “only in New Zealand assumption”, firms may indeed have an incentive to use the free allocation to reduce prices of all output, including of future growth, in order to stay competitive. By contrast, in the world of locational choice, a profit-maximising firm would re-locate future production overseas, where that is possible, while keeping the free allocation to compensate for the emissions from the existing production.

To put it another way, the free allocation of emission permits creates a strong incentive for firms to differentiate between production based on existing assets and levels of output, and production to meet any growth in demand. A profit maximising firm would not use free allocation of permits to under-write new investment, when such investment can be transferred to a location without the carbon tax or emission trading.

Another way of looking at the same issue is that the model, in essence, assumes that the same incumbent firms receive free allocation and undertake new investments which drive growth. If firms enter and exit the market, and if growth tends to come from new entrants, rather than incumbents, the assumptions in the model would not hold.

We believe that a more realistic representation of firm response to free allocation is essential in order to be able to judge the economic effects of an ETS scheme with free allocation. As currently specified, the model sheds little light on the essential features of ETS design.

Finally, but perhaps most importantly, model scenarios do not reflect the actual policy pathways available to New Zealand. In essence, the way the scenarios are defined in the model implicitly assumes that New Zealand has an once-in-a-lifetime opportunity to introduce a climate change policy, and that such policy should remain robust through the evolution of the global policy response. Hence, the model defines mutually exclusive scenarios (which include the Government purchasing permits, a narrow carbon tax and a broad-based ETS with free allocation).

This is a poor description of the actual options available to policy makers. The model does not provide any information on how the choice of one instrument or a set of instruments during the early years of climate change policy affects the options available during subsequent years.

Similarly, the model provides little information on how the choice of domestic policy in New Zealand would be affected by the policies adopted by different trading partners. The way the model is currently set up, it essentially assumes a single rest-of-the-world sector, with the same price elasticity relevant for all trade in each sector.

In reality, different countries will adopt different climate change policies at different times. It is difficult to see how policy makers can make a sensible choice between policy options without understanding how the New Zealand economy would respond to a variety of possible actions by our trading partners.

Overall, it is critically important not to confuse the scenarios for different model runs with actual policy options. Rather, some scenarios represent elements of wider policy options, while other scenarios may actually misrepresent the key economic features of the policy it is intended to describe. This in particular applies to the effects of the ETS with free allocation.

4 Are model results reliable and robust?

The results from the CGE model runs make empirical claims about:

The size of the total economic impact

The ranking of scenarios by their relative economic impact.

As the report itself acknowledges, there is no way to test the robustness of the empirical results against historical evidence. However, considerable amount of sector-specific analysis and modelling has been done for various sectors. It would have been very instructive to compare the sectoral results generated by the model with the results of such sector-specific studies. We are concerned that the report does not present sectoral results nor does it attempt to test the robustness of the model against what is known about the individual sectors of the New Zealand economy. This is a major weakness of the study, as considerable information exists about individual sectors.

A good example of this is forestry. Forestry is the only sector where the report provides some hints about the sectoral results. After acknowledging the difficulty of modelling forestry within the CGE framework, the report notes that at a carbon price of $25 per ton of CO2 (modelled as a negative tax for the sector) the additional area of Kyoto plantation forests is 90,000 ha. This is not an annual planting number—this is the total increase in the area under plantation forestry between the two scenarios.

This clearly understates the potential for additional plantation forestry. A number of submissions to the Emission Trading Scheme Select Committee have indicated that if sensibly structured, relatively modest expenditures per ton of CO2 could induce over 1 million hectares of additional planting. Since carbon capture is a critical component of any New Zealand climate change policy response, such a large discrepancy between the model and sector-based analyses needs to be carefully examined and explained.

In the absence of such empirical reconciliation, the model’s empirical claims about the size of the economic impact and the ranking of the options can not be treated as robust.

5 Do conclusions flow from the results?

The report’s key recommendation is that in the short run an ETS should be introduced with free allocation to competitiveness at risk sectors. Agriculture should be excluded if measurement of its emissions is prohibitively expensive, but included once measurement becomes economic.

Since the analysis in the model:

Does not consider measurement and transaction costs

Indicates that an ETS with free allocation is not the least cost solution in the short term

it appears that the recommendation bears no relationship to the analysis. Indeed, the recommendation seems to be derived from two observations made in passing on page 45, which are expressions of opinion, rather than results of the analysis:

While a “Government pays” model is the least cost in the short-run, there are benefits from giving firms clear and consistent signals around carbon pricing

New Zealand would lose credibility in international negotiations if it did not introduce carbon pricing.

The issue is not whether these observations are right or wrong, or whether we agree or disagree with them. The key point is that there is nothing in the report on which to base these observations. Hence, the conclusions do not flow from the results of the analysis.

6 Conclusion

The NZIER and Infometrics report represents a useful contribution to the debate. It clarifies some issue arising from past CGE modelling attempts. The report would have been useful as an input into an analytical exercise to define policy options and to consider their relative economic effects over time. However, it would be dangerous to rely on the report as a stand-alone review of climate change policy options. In its own right, the report does not represent a useful Regulatory Impact Statement.

We are particularly concerned that the report makes policy recommendations which are not based on the analysis provided in the report.

ENDS

© Scoop Media