Research Administrators’ Seminar Day 2—ERA 2018—Engagement and Impact assessment
15 November 2016
Dr Marcus Nicol and Therdpun Klinjun

 ""

Presentation slide header

State of Australian University Research 2015–2016
Volume 1, ERA National Report

ERA 2015 National Report cover

ERA 2015 outcomes show Australia’s universities continue to produce high-quality research across a wide range of disciplines.

Overall research quality is improving and reflects ERA’s role in focussing universities on research quality. 

Of the UoEs assessed in both ERA 2015 and the previous round, ERA 2012, 56 per cent of these maintained the same rating, while 29 per cent improved their rating by one. 

The ERA results show an extraordinary breadth of outstanding research performance in Australia.

Volume 1, ERA National Report.

 

 

 

 

[top]

Presentation slide header

State of Australian University Research 2015–2016
Volume 2, Institutional Insights

ERA 2015 Volume 2 cover

  • With three rounds of Excellence in Research for Australia (ERA) now complete, the ERA dataset covers all Australian university research outputs, staffing and activity from 2003 to 2013, and research income and research application data from 2006 to 2013. 
  • The State of Australian University Research 2015–16: Volume 2 Institutional Insights looks at some selected topics to provide further understanding of the state of Australian university research. 
  • This new report covers the following topics: ERA Ratings, Gender and the Research Workforce, Open Access, Volume of Submitted Outputs in ERA 2015 and Percentage Contribution to the National Landscape

Volume 2, Institutional Insights

 

 

 

 

 [top]

Presentation slide header

ERA

  • The next round of ERA is expected to take place in 2018
  • Draft ERA 2018 submission documentation will be released for consultation (by mid 2017)
  • It is currently expected that changes to the ERA process for 2018 will be minimal………

 [top]

Presentation slide header

Updating the ERA Journal List

In preparation for the ERA 2018 evaluation, the ARC will review the ERA Journal List. This process occurs in three phases:

  1. Consultation with universities on changes to the existing journal list and addition of new journals which meet the journal eligibility criteria
  2. Comments will be considered and where applicable incorporated into the ERA 2018 Journal List for the consultation with discipline experts
  3. Peak bodies and academic groups make recommendations then the ARC will review comments and finalises the journal list.

[top]

Presentation slide header

ANZSRC review

  • ANZSRC first published by the ABS in 2008
  • ABS review of ANZSRC due in 2018
  • ARC working with ABS to review the FoR codes
  • No code changes for ERA 2018

[top]

Presentation slide header

Engagement and Impact Assessment
National Innovation and Science Agenda

  • National Innovation and Science Agenda (NISA)—7 December 2015—$1.1 billion over 4 years
  • A range of new initiatives to support research, encourage innovation and entrepreneurship, reward risk taking, and promote science, maths and computing in schools

[top]

Presentation slide header

NISA—ARC measures

  • Continuous linkage
    • Will allow researchers and industry partners to take advantage of opportunities for collaboration faster, and encourage more partnerships between universities and business
  • Engagement and impact assessment
    • NISA includes $11.2 million over 5 years ($2.2 million on average) to conduct a national assessment measuring impact and engagement in university research (EI assessment)
  • Government invests $3.5 billion a year in university research

[top]

Presentation slide header

Engagement and Impact assessment

For 2018

  • all 42 Australian universities (defined by the HESA 2003) will be eligible to participate
  • all research disciplines will be involved
  • the methodology will not advantage one discipline over another
  • evaluations will be conducted by committees of experts from the university sector and industry

[top]

Presentation slide header

Development

  • Assessment is being developed by ARC and Department of Education and Training as a companion to ERA
    • Steering Committee—which includes higher education and industry leaders—overseeing the development of the framework for the assessment. Committee supported by two working groups:
    • Technical Working Group provides expert advice on the development of indicators that will support the engagement and impact assessment 
    • Performance and Incentives Working Group provides advice to the ARC about the potential incentive effects of the preferred model
  • The methodology is being developed with extensive consultation with the university sector, industry and other research end-users

[top]

Presentation slide header

Timeline

  • Methodology being developed in 2016, including extensive sector consultation
  • Framework and guidelines to be developed for 2017
  • Pilot assessments in 2017
  • Full assessment in 2018 as a companion to ERA

[top]

Presentation slide header

Consultation

  • Two formal consultations have been undertaken:
    • Engagement and Impact Assessment Consultation Paper, public consultation, 2 May – 24 June 2016
    • End-user survey, 2 May – 1 June 2016, industry and other end-users or beneficiaries of university research were surveyed
  • A series of face-to-face meetings between the ARC CEO and industry leaders

[top]

Presentation slide header

ERA/EI

  • ERA and EI will run as companion exercises
  • ERA will continue to assess research quality
  • ERA acknowledges and encourages ‘blue sky’ research
  • Engagement and impact assessment will consider:
    • research interactions with industry, Government, non-governmental organisations, communities and community organisations, and
    • research contributions to economy, society and environment

[top]

Presentation slide header

Engagement and Impact Assessment Pilot 

Development is in progress:

  • Expected to run mid-late 2017
  • Two separate pilot studies - engagement and impact
  • all 42 Australian universities (defined by the HESA 2003) will be eligible to participate – participation is voluntary
  • evaluations will be conducted by committees of experts from the university sector and industry/end users of research

[top]

Presentation slide header

Pilot assessment framework?

  • The Unit of Assessment (UoA)—the 2-digit ANZSRC Field of Research (FoR) at the institution
  • Engagement and impact assessed and rated separately
  • Engagement assessed primarily using quantitative information comprising a matrix of metrics. A narrative statement will provide context and supplement the quantitative information
  • Impact assessed primarily using qualitative information (impact study) which can be supplemented by quantitative information if any

[top]

Presentation slide header

Pilot assessment framework?

Unit of Assessment

Engagement

Impact

Matrix of Metrics/

Indicators

Narrative

 

Impact Study

 

Rating for Engagement

Rating for Impact

 [top]