Excellence in Research for Australia 2018 Update

What is ERA?

  • ERA evaluates the quality of the research undertaken in Australian universities against national and international benchmarks
  • The outcomes (ratings) are determined and moderated by committees of distinguished researchers from Australia and overseas. 
  • The unit of evaluation is broadly defined as the Fields of Research (FoR) within an institution based on the Australia and New Zealand Standard Classification (ANZSRC). 
  • The indicators used in ERA include a range of metrics such as citation profiles which are common to disciplines in the natural sciences, and peer review of a sample of research outputs which is more broadly common in the humanities and social sciences. 
  • ERA is a comprehensive collection. The data submitted by universities covers all eligible researchers and their research outputs.

ERA objectives are unchanged

  1. An evaluation framework that gives assurance of the excellence of research conducted in Australian higher education institutions
  2. National stocktake of research discipline strengths and development opportunities
  3. Identify excellence across full spectrum of research performance
  4. Identify emerging research areas and opportunities
  5. Research discipline comparisons nationally and internationally

Reference periods

Data type

Reference period

Years

Research outputs

1 Jan 2011-31 Dec 2016

6

Research income

1 Jan 2014-31 Dec 2016

3

Applied measures

1 Jan 2014-31 Dec 2016

3

Staff census date

As at 31 March 2017

 

Key documents

  • ERA 2018 Submission Guidelines
  • ERA-SEER 2018 Business Rules and Verification
  • ERA-SEER 2018 Technical Specifications
  • ERA 2018 Discipline Matrix
  • ERA-SEER 2018 Technology Pack

ERA 2018

The overall approach of the ARC is to keep ERA relatively unchanged for 2018.

But we are making a few small changes….

Submission process changes

  • Restructuring of the submission stages
  • No hard copy certification statement 
  • Requirement for links to outputs to be direct, not to landing pages 
  • Removal of commercial sensitivity

Publication of ERA data

  • ARC is committed to ERA evaluations being open and transparent
  • Selected submitted data from ERA 2018 may be published such as
    • Research output metadata
    • FoR codes
    • Aggregated institution data

2018 submission stages

Stage

Start

Finish

Submission

19 March 2018

17 April 2018

Data integrity checks

18 April 2018

10 May 2018

Certification

11 May 2018

16 May 2018

ERA-SEER 2018 submission process changes

  • Redevelopment of SEER to adopt the same framework as RMS 
  • Single user profile for ARC (RMS or SEER)
  • Nominations are now done through the SEER system
  • Streamline submission processes
  • Research repository domain and authentication details are not part of the XML submission
  • No metadata pages available for peer review outputs
  • Explanatory Statements will be machine readable PDF upload
  • Clear check list of tasks to be completed before submission can be finalised
  • Institution cannot un-finalise without ARC permission.

Data collection clarifications and changes

  • ORCID—must include if you have it
  • Definition of gender—updated
  • DOI—must include if you have it
  • No esteem measures
  • Staff data
    • FoR assignment—must reflect the activities of the researchers
    • Reporting of levels—inline with HESDC, more granular
    • Employment status—‘other employed’ instead of ‘other’
    • Updated LWOP—more inclusive

Textual clarification in submission guidelines

  • Eligible staff 
  • Byline
  • Open access
  • Revisions across research output types
  • NTRO portfolios
  • Assignment and apportioning of FoR codes—coding must reflect content
  • Action on incorrect coding or false/misleading data

 

 

Engagement and Impact 2018

Submission journal list

  • Three consultation phases 
    • Phase 1 University consultation
    • Phase 2 Peak body consultation
    • Phase 3 Consolidation of feedback
  • Final list released to institutions 23 August 2017
  • 25,017 journals on the list

Submission conference list
Submission publisher list

  • ERA 2015 submission lists used as a basis
  • No Fields of Research (FoR) codes
  • ‘Other’ category available

Citation provider

  • Same methodology
  • Data based on indexed journal articles
  • Panel uses the suite of indicators for a Unit of Evaluation (UoE). Not assessed on just citation analysis

Article tagging by citation provider

For disciplines where citation analysis is used, the ERA 2018 submission process requires a unique article identifier (electronic identifier) for each indexed journal article. 

  • The ARC facilitates the tagging for each of the submitting institutions and bears the cost of this tagging. 
  • A portal is set up allow documents to be uploaded for tagging. 
  • Institutions communicate directly with provider.
  • The citation provider needs to work with eligible institutions to tag outputs with unique identifiers. 
  • The institution is responsible for ensuring that all articles published in indexed journals during the reference period are tagg

Note: The degree of quality and accuracy of the tagging service provided by the citation provider depends on the quality of data provided by the institutions

Research outputs to be tagged

Only research outputs that meet the following criteria require tagging:

  • Disciplines subject to citation analysis as indicated in the ERA discipline matrix
  • Published in journals on the ERA 2018 Journal List and indexed by the citation provider
  • Published during the reference period (1 January 2011–31 December 2016*)

All articles meeting the above must be tagged.

Research Evaluation Committees

  • Over 600 nominations for ERA 2018 Research Evaluation Committees
  • Members bring considerable experience and ability which will ensure that ERA evaluations are of the highest standard
  • Australia and overseas with broad range of relevant expertise

ERA/EI

  • ERA and EI will run as companion exercises
  • ERA will continue to assess research quality
  • ERA acknowledges and encourages ‘blue sky’ research
  • Engagement and impact assessment will encourage universities to look beyond the academic sector.

 

 

 

Engagement and Impact 2018

A national engagement and impact assessment—what is it?

New national assessment:

  • how well researchers in Australian universities engage with end users beyond academia
  • what kinds of impacts are occurring outside of academia as result of research undertaken by Australian universities
  • how well Australian universities support their researchers to deliver research which has an impact beyond academia


Why?

Encouraging innovation and entrepreneurship by:

  • encouraging researchers to better engage with users of their research
  • encouraging greater focus on delivering research impacts
  • encouraging Australian universities to support their researchers in these activities

Definition of engagement

The interaction between researchers and research end-users outside of academia, for the mutually beneficial transfer of knowledge, technologies, methods or resources.

End-user definition

An individual, community or organisation external to academia that will directly use or directly benefit from the output, outcome or result of the research

Definition of impact

The contribution that research makes to the economy, society, environment and culture beyond the contribution to academic research.

Aboriginal and Torres Strait Islander research

Aboriginal and Torres Strait Islander research means that the research significantly:

  • relates to Aboriginal and Torres Strait Islander peoples, nations, communities, language, place, culture or knowledges and/or 
  • is undertaken with Aboriginal and Torres Strait Islander peoples, nations, or communities.

2018 assessment framework

Unit of Assessment

Engagement

Impact

Matrix of indicators

Indicator Explanatory Statement

Engagement Narrative

Impact (and associated research)

 Approach to impact


 

Rating for Engagement 

Rating for impact

Rating for approach to impact

UoA 11

Biomedical and Clinical Sciences

Public and Allied Health Sciences

1101 Medical Biochemistry and Metabolomics

1104 Complementary and Alternative Medicine                                                          

1102 Cardiovascular Medicine and Haematology

1106 Human Movement and Sports Science

1103 Clinical Sciences

1110 Nursing

1105 Dentistry

1111 Nutrition and Dietetics

1107 Immunology

1117 Public Health and Health Services

1108 Medical Microbiology

1199 Other Medical and Health Sciences

1109 Neurosciences

 

1112 Oncology and Carcinogenesis

 

1113 Ophthalmology and Optometry

 

1114 Paediatrics and Reproductive Medicine

 

1115 Pharmacology and Pharmaceutical Sciences

 

1116 Medical Physiology

 

Reference periods

Stage

Reference period

Years

Impact study

1 January 2011 – 31 December 2016

   6

Associated research

1 January 2002 – 31 December 2016

  15

Engagement

1 January 2014 – 31 December 2016

   3

Low volume threshold

  • 150 weighted outputs 
  • Based on relevant two-digit FoR submission to ERA 2018. 
  • Institutions that meet the low volume threshold must submit both engagement and impact* data for that UoA
  •  Can opt-in to either engagement or impact assessments if don’t meet threshold.
  • Does not apply to the interdisciplinary and Aboriginal and Torres Strait Islander research impact studies.

Engagement narratives

Engagement Narrative

  • 7000 characters
  • Describing engagement activities, strategy and/or objective

Indicator Explanatory Statement

  • 4500 characters
  • Adding context/further explanation to indicator data

2018 Engagement Indicators        

Cash support from end-users

Total HERDC income per FTE

Proportion of Specified Cat 1 grants to all HERDC Cat 1 grants

Research commercialisation income              

Additional data collection

Co-supervision of Higher Degree by Research (HDR) students

Impact studies

  • Impact is assessed using qualitative information in the form of impact studies
  • Impact studies include
    • Approach to impact
    • Details of the impact
    • Associated research
  • Impact studies will allow for the inclusion of measures or indicators of impact

EI 2018 rating scale

The ratings are established by a panel of experts based on the indicators and the narratives provided. One of three ratings will be given

Low

Medium

High

Submission components—Engagement

  • Engagement narrative per UoA
    • 7000 characters
  • Engagement indicators per UoA (from ERA data)
    • HERDC research income per FTE
    • Specified HERDC Category 1 grants
    • Research commercialisation income
  • Engagement indicators per UoA (Data required)
    • Cash support from research end-users
    • Co-supervision of HDR students
  • Engagement indicator explanatory statement per UoA
    • 4500 characters

Submission components—Impact

Impact study

  • Impact (per UoA)
  • Approach to impact (per UoA)
  • Aboriginal and Torres Strait Islander research (one optional)
  • Interdisciplinary (one optional)

Templates available

To assist with submission the following templates will be available in the submission guidelines

  • Engagement narrative
  • Engagement indicator explanatory statement
  • Impact study UoA
  • Impact study—Aboriginal and Torres Strait Islander research
  • Impact study interdisciplinary

Expected timeline—2018 Submission  Stages

 Stage  Start  Finish

Draft Guidelines consultation

06 November 2017

17 November 2017

EI Submission document release

early December 2017

-

Submission

16 May 2018

27 June 2018

Submission will be by web forms not XML.