Pilot Overview

As part of the development of the first national assessment of research engagement and impact, in 2017 the Australian Research Council (ARC) conducted a pilot exercise. The pilot is essential for informing the development and implementation of the first full engagement and impact assessment in 2018, which will run alongside ERA 2018 as a companion exercise.

The aim of the pilot was to test the proposed approach (that is the methodology and processes) for the full engagement and impact assessment. The pilot was not designed to be an end-to-end assessment exercise but instead was used to test the robustness of a wide range of indicators and methods of assessment of engagement and impact.

A selection of 10 broad discipline groups (Fields of Research) were tested in the pilot enabling a spread of disciplines across STEM and HASS to be involved with a minimum of burden. Discipline groups that were tested by the pilot included: Chemical Sciences, Medical and Health Sciences, History and Archaeology, and Philosophy and Religious Studies, Environmental Sciences, Agricultural and Veterinary Sciences, Engineering, Education, Studies in Creative Arts and Writing, Language Communication and Culture, and Indigenous research.

Engagement and Impact were assessed separately in the pilot. Engagement assessment involved metric indicators and a narrative statement. Impact assessment involved qualitative information (impact studies) supplemented with quantitative information if any was available. Engagement and Impact data and information was assessed by panels comprising a mix of academics with discipline expertise and industry representatives and other end-users of research. A review panel then considered the overall methodology.

All Australian universities were eligible to participate and participation was voluntary. 

Senator the Hon. Simon Birmingham, Minister of Education and Training, and ARC Chief Executive Officer, Professor Sue Thomas, has announced the release of the Engagement and Impact Assessment Pilot Report. The report summarises the findings from the Engagement and Impact Pilot (EI Pilot) and its review, and proposes modifications to the methodology for Engagement and Impact Assessment in 2018 (EI 2018).

• Engagement and Impact Assessment Pilot 2017 Report –  PDF Format (2MB) –  Word Format (2MB)

The assessments were undertaken by five panels each comprised of academic researchers and research end-users.

A Review Panel was also established to examine pilot material and provide advice to the ARC on the methodology, assessed by the five associated EIAP panels.

The members of the panels were:

Panel A—Chemical Sciences and Medical and Health Sciences (Engagement)

• Professor Neil Barnett, Deakin University
• Associate Professor Fiona Bogossian, University of Queensland
• Professor Nicholas Dixon, University of Wollongong
• Professor Helen Edwards, Queensland University of Technology
• Professor Richard Lindley, University of Sydney
• Professor Paul Low, University of Western Australia
• Professor Terry Nolan, University of Melbourne
• Professor William Price, University of Wollongong
• Professor Brian Smith, La Trobe University
• Dr Katherine Woodthorpe, Bioplantforms Australian
• Mr Kevin Cullen, BresMedical
• Ms Catherine Skidmore, OneSteel Manufacturing
• Professor Joanne Aitken, Cancer Council QLD
• Professor Daryl Higgins, Australian Catholic University

Panel B—History and Archaeology and Philosophy and Religious Studies (Engagement)

• Professor Alastair Blanshard, University of Queensland
• Professor Jo McDonald, University of Western Australia
• Professor Catriona Mackenzie, Macquarie University
• Professor Keir Reeves, Federation University Australian
• Professor Mary Spongberg, University of Technology, Sydney
• Professor Nicholas Brown, Australian National University
• Associate Professor Noah Riseman, Australian Catholic University
• Professor Paul Patton, University of New South Wales
• Professor Rachel Ankeny, University of Adelaide
• Emeritus Professor Terence Lovat, University of Newcastle
• Ms Gina Pickering, National Trust of Australia (WA)
• Mr Richard Neville, State Library of NSW
• Dr Richard Gillespie, Museums Victoria
• Ms Jillian Comber, Comber Consultants
• Professor Colin Thomson, Australasian Human Research Ethics Consultancy Services Pty Ltd

Panel C—Environmental Sciences, Agricultural and Veterinary Sciences, and Engineering (Impact)

• Professor James Pratley, Charles Sturt University
• Professor Bronwyn Gillanders, University of Adelaide
• Professor Nigel Tapper, Monash University
• Professor Colin Simpfendorfer, James Cook University
• Professor John Cole OAM, University of Southern Queensland
• Associate Professor Jonathan Binns, University of Tasmania
• Professor Kevin Galvin, University of Newcastle
• Professor Peter Nelson, Macquarie University
• Professor Zed Rengel, University of Western Australia
• Professor Sally McArthur, Swinburne
• Ms Karen Rouse, Water Research Australia
• Mr David Hudson, SGA Solutions
• Dr Julie Morgan, Woodside Energy Limited
• Mr David Shelmerdine, ClimateWorks Australia
• Dr Brad Dunstan, Multimatic Engineering Australia

Panel D—Education, Studies in Creative Arts and Writing, and Language, Communication and Culture (Impact)

• Professor Anne Castles, Macquarie University
• Professor Annemaree Carroll, University of Queensland
• Professor Barbara Comber, University of South Australia
• Professor Catherine Stevens, Western Sydney University
• Professor John Hajek, University of Melbourne
• Professor Larissa Hjorth, RMIT
• Professor Mark Carroll, University of Adelaide
• Professor Jen Webb, University of Canberra
• Professor Will Christie, Australian National University
• Scientia Professor, Dennis Del Favero, University of New South Wales
• Ms Kim Bertino, KU Children’s Services
• Mr Tim Matthies, Musica Viva Australia
• Mr Rob Nairn, Australian Secondary Principals Association
• Mr Pino Migliorino, Cultural Perspectives
• Dr Judith Bishop, Appen Butler Hill Inc.

Panel E—Indigenous Research (Impact)

• Professor Maggie Walter, University of Tasmania
• Ms Lauren Ganley, Desert Knowledge
• Mr Ned David, Torres Strait Islanders Regional Education Council
• Professor Irene Watson, University of South Australia
• Professor Shane Houston, University of Sydney

Review Panel

• Professor John Lynch, University of Adelaide
• Professor Nancy Longnecker, University of Otago – New Zealand
• Dr Michael Wykes, University of Exeter – London
• Professor Maria Delgado, Royal Central School of Speech and Drama – London
• Professor Daryl Higgins, Australian Catholic University
• Mr Rob Nairn, Australian Secondary Principals Association
• Professor Veena Sahajwalla, University of New South Wales
• Dr Joanne Luck, Horticulture Australia
• Professor Catriona Mackenzie, Macquarie University
• Professor Larissa Hjorth, RMIT
• Dr Phil Kearney, MSD Australia