Gordon Campbell | Parliament TV | Parliament Today | News Video | Crime | Employers | Housing | Immigration | Legal | Local Govt. | Maori | Welfare | Unions | Youth | Search

 

ACC’s computer-aided decision-making questioned


ACC’s computer-aided decision-making questioned by Otago experts

University of Otago researchers are warning of the potential pitfalls in government departments using computer-based risk prediction models, as has been recently revealed through a controversial new tool used by the Accident Compensation Corporation (ACC) to profile and target clients.

The ACC uses a computer model to assist staff managing claims, however, details of what the model does, and how it is used, are somewhat sketchy, says spokesperson for the University’s Artificial Intelligence and Law in New Zealand Project, Associate Professor James Maclaurin.

The most concrete information comes from a short press release issued by ACC indicating that the tool is used to make three types of prediction:

• which clients are likely to need help and should be called proactively,
• which type of case owner should assist the client, and
• how long we should expect a claim to be managed.

“This somewhat vague description leaves open the possibility that ACC uses these predictions to minimise treatment times, either by intervening in patients’ treatment, or (more seriously) by declining applicants with long predicted treatment times,” says Associate Professor Maclaurin.

“The tool makes predictions about future ACC cases using a database of information about 364,000 past claims that were lodged between 2007 and 2013. ACC stresses that details about individual cases are kept private both from ACC staff and from other agencies,” says Department of Computer Science’s Professor Alistair Knott, another researcher on the project.

Management still appears to be ultimately under human control: an ACC ‘case owner’ makes the final decision about each case.

“But ACC workers find themselves in a situation increasingly common in our society: their decisions are guided by advice generated automatically by a machine, based on a large set of data extending far beyond their own experience.

“We are in the same position when we use Google’s navigation system in our cars, or choose a book based on Amazon’s recommendations. In these cases, having a computer in the decision-making loop seems innocuous enough. It seems less innocuous when it guides the agencies whose decisions have serious consequences for people’s lives. We set out six questions that such agencies must be able to answer about the algorithms they use.

“Of course it is a fundamentally a good thing for people’s decision-making to be informed by statistics. Systems like the ACC tool can be quite accurate, but because they don’t reason in the way humans do, it is essential that Governments and companies relying on them are able to answer the following questions:

1. How accurate is the tool, in fact? Predictive tools are easy to evaluate: if the public is to have confidence in the ACC tool, and the courts are to evaluate its use, the agency should give a public account of how it was evaluated. Without divulging personal details, this should include a thorough description of the data set on which it was assessed.
2. Can the agency explain the way this tool works so that clients could appeal particular decisions?
3. Does the use of this tool distort the way the agency pursues its stated policy objectives?
4. By ‘passing the buck’ to the machine, is the agency to ducking its responsibility to make fair and humane decisions about treatment of New Zealanders in need?
5. Does this tool implicitly discriminate against individuals on problematic grounds such as age, ethnicity or gender? Importantly, there is simply no way to remove this bias without compromising the accuracy of its predictions. There is a real risk that the ACC tool unfairly discriminates against some clients. This possibility needs to be explored in an evaluation of the system.
6. Are employees effectively trained in the use of the system? While the predictive system is probably intended to be used as a guide, to supplement the case owner’s own knowledge and judgements, it is easy to fall into ‘autopilot’ mode when guided by a system—especially if it is fairly accurate. (The dangers of autopilot mode are particularly clear if the system’s errors are prone to bias, of the kind just mentioned.) The charge of ‘falling into autopilot mode’ is often levelled at judges using the US predictive system.

“Predictive analytics technologies show great potential in informing public decision-making, but it is important for these technologies to be evaluated and scrutinised when used in the public domain. It may be that ACC has addressed the issues we raise in its own internal training and evaluation processes. But we are calling for ACC to provide a public account of how it uses its predictive tool, so as to maintain the integrity of its decision-making,” says Faculty of Law Professor Colin Gavaghan.

© Scoop Media

 
 
 
Parliament Headlines | Politics Headlines | Regional Headlines

Gordon Campbell: On The Peters/Ardern Triumph

There are a lot of good reasons to feel joyful about this outcome. It is what so many young voters – the best hope for the country’s future – wanted.

Far more important than the implications for the Economy Gods ( is the dollar up or down? ) last night’s outcome will also mean many, many vulnerable New Zealanders will have a better life over the next three years at least.

Yet the desire for change was in the majority, across the country..>>>More


Reaction

Labour on its agreement |Peters: Post-Election Announcement Speech | Greenpeace “cautiously hopeful” about new Government | ACT - Madman on the loose | E tū ecstatic | Chamber welcomes the outcome | Greens on their joining Govt | EDS welcomes new govt | Immigrant groups worry | Feds ready to engage new coalition government | Labour Ministers of the Crown announced

 

Climate: Increasing Greenhouse Emissions Hit NZ

New Zealand is seeing impacts of excess greenhouse gas emissions in our climate and oceans, according to the latest national report from the Ministry for the Environment and Stats NZ about the state of the atmosphere and climate…More>>

ALSO:


Wellington.Scoop: Arrests At Blockade Of "Weapons Expo"

“We encourage people in Wellington to get down to the Westpac Stadium now for a day of awesome peace action. There will be plenty of food, music and activities to keep us sustained through the day.” More>>

ALSO:

Rorschach Restructuring: PSA Taking Inland Revenue To Court Over Psychometrics

The Public Service Association will be seeing Inland Revenue in Employment Court over its intention to psychometrically test employees reapplying for their roles at the department as part of its controversial Business Transformation restructuring plan. More>>

ALSO:

Nuclear Disarmament: Nobel Peace Prize 2017 Awarded To ICAN

Congratulations from iCAN Aotearoa New Zealand to international iCAN, the other iCAN national campaigns and partner organisations, and the countless organisations and individuals who have worked so hard for a nuclear weapons-free world since 1945. More>>

ALSO:

Expenses: Waikato DHB CEO Resigns

An independent inquiry has identified that Dr Murray had spent more than the agreed $25K allocated for relocation costs, and other unauthorized expenses involving potential financial breaches of the chief executive’s obligations. More>>

ALSO:

Wellington.Scoop: Sad About The Trolley Buses?

The Regional Council’s MetLink is today spending money to tell us that it really loves Wellington’s trolley buses, even though they’re all being taken off our roads by the end of this month. More>>

ALSO:

Post-Election: Preliminary Coalition Talks Begin

New Zealand First will hold post-election preliminary discussions in Wellington with the National Party tomorrow morning and the Labour Party tomorrow afternoon. More>>

 
 
 
 
 

LATEST HEADLINES

  • PARLIAMENT
  • POLITICS
  • REGIONAL
 
 

Featured InfoPages

Opening the Election