Standard: Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 Outcomes

Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 Outcomes

ERA and EI outcomes published

The Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 national reports, outcomes and other data are available on the Australian Research Council (ARC) website via the ARC Data Portal.

• ARC Data Portal
• ERA
  • ERA National Report—State of Australian University Research 2018-19
  • ERA Research Outputs 
  • ERA Outcomes
• EI
  • EI National Report—Engagement and Impact Assessment 2018-19
  • EI Impact Studies 
  • EI Outcomes

[back to top]

ERA and EI Key Differences

ERA: 

• Evaluates research quality 
• Comprehensive data collection 
• Fourth round in 2018

EI:

• Assesses the engagement and impact of research beyond academia
• Selective narrative and indicator based assessment 
• Inaugural round in 2018

[back to top]

Useful background and terminology

FoR stands for Fields of Research from the Australian and New Zealand Standard Research Classification (ANZSRC). The FoR classification has three hierarchical levels of increasing specificity. The three levels are Divisions (broad disciplines), Groups (collections of related Fields) and Fields, respectively indicated by two, four or six digit classification numbers.

[back to top]

ERA 2018 Framework and Outcomes

How do we evaluate research quality in ERA?

• The unit of evaluation (UoE) is broadly defined as the Fields of Research (FoR) within an institution based on the Australia and New Zealand Standard Classification (ANZSRC).
• UoEs are 2-digit FoRs (22 broad disciplines) and 4-digit FoRs (157 specific disciplines).
• The indicators used in ERA include a range of metrics such as citation profiles which are common to disciplines in the natural sciences, and peer review of a sample of research outputs which is more broadly common in the humanities and social sciences.
• The outcomes (ratings) are determined and moderated by committees of distinguished researchers from Australia and overseas.

[back to top]

ERA Outcomes

Data (longitudinal across four ERA rounds)

• Discipline strengths
• Research output and indicator data
• University performance by discipline
• Workforce data–including  gender

[back to top]

National UoE ratings ERA 2018 (aggregated four-digit results, grouped by two-digit FoR code)

[back to top]

Distribution of ERA 2018 ratings for all UoEs across all Australian universities

[back to top]

Distribution of rating results across ERA rounds

[back to top]

Change in UoE ratings between ERA 2015 and ERA 2018

[back to top]

HERDC research income by category and two-digit FoR code ERA 2018
All universities

[back to top]

HERDC research income by category across four ERA Rounds
All universities

[back to top]

Staffing profile by FTE, employment level and two-digit code ERA 2018
All universities

[back to top]

Staffing profile by headcount, employment level and two-digit code ERA 2018
All universities

[back to top]

Staffing profile by headcount and employment level across four ERA rounds
All universities

[back to top]

FTE staff by gender and employment level ERA 2018
All universities

[back to top]

EI 2018 Framework and Outcomes

EI 2018 Framework

[back to top]

Definitions

• Research is the creation of new knowledge and/or the use of existing knowledge in a new and creative way to generate new concepts, methodologies, inventions, and understandings. This could include the synthesis and analysis of previous research to the extent that it is new and creative. 
• Aboriginal and Torres Strait Islander Research means that the research (as defined above) significantly: 
  • relates to Aboriginal and Torres Strait Islander peoples, nations, communities, language, place, culture or knowledges, and/or
  • is undertaken with Aboriginal and Torres Strait Islander peoples, nations, or communities.
• Research impact is the contribution that research makes to the economy, society, environment or culture, beyond the contribution to academic research.
• Research engagement is the interaction between researchers and research end-users outside of academia, for the mutually beneficial transfer of knowledge, technologies, methods or resources.
• A research end-user is an individual, community or organisation external to academia that will directly use or directly benefit from the output, outcome or result of the research.

[back to top]

EI findings

EI 2018 provides a range of data. This data:

• is a rich new national resource
• includes examples of enabling institutional processes and infrastructure
• provides a benchmark of performance and
• includes best practice examples. 

[back to top]

Engagement, Approach to Impact and Impact Ratings EI 2018
All universities

^{Percentages may not total 100 due to rounding}

Charts showing national results for Engagement, Approach to Impact and Impact ratings for EI 2018

[back to top]

Additional data collected EI 2018 

Additional data collected for EI 2018 includes:

• Additional Fields of Research - impact 
• Additional Fields of Research - associated research
• Socio-Economic Objective (SEO) codes
• Australian and New Zealand Standard Industrial Classification (ANZSIC) codes
• Science and Research Priorities
• Keywords
• Beneficiaries
• Countries where impact is occurring

[back to top]

Keyword analyses 
All universities

Word cloud showing some of the more common keywords that appeared in EI submissions nationally.

[back to top]

Countries of impact 
All universities

Word cloud showing countries where university impacts are occurring, based on EI impact studies.

[back to top]

FoR mapping—Impact
All universities

^{FoRs are listed in numerical order. Some FoRs may not be labelled due to space limitations.}

Sanke chart showing associated research FoRs mapped to impact FoRs.

[back to top]

Insights from EI 2018

Insights from EI 2018:

• Many pathways to impact 
• Impact occurring in many research areas and in different ways 
• Many types of collaborations

[back to top]

High rated engagement narratives

Common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20)

High rated engagement narratives – common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20):

• Engagement embedded in strategic plan
• Target the ‘right’ partner
• Engagement and impact linked to staff performance and promotion
• Dedicated business units – the ‘front door’
• Importance of ARC Linkage, ITRP and CRCs
• Embedded staff (between university and industry)
• Staff on advisory boards, committees etc. (between university and industry)
• Consultancies
• Specialist, end-user focused conferences and workshops
• Multi-media platforms
• Co-supervision and mentoring of HDRs
• Engagement with schools and educators

[back to top]

Engagement—discipline deep dives

STEM—06 Biological Sciences, 09 Engineering

Engagement discipline deep dives for STEM disciplines of Biological Sciences and Engineering. Common themes included:

• Co-location of partner industries to campus sites
• Provision of specialist services through unique infrastructure and equipment 
• Reviews of national and international standards 
• Citizen science

[back to top]

Engagement—discipline deep dives

HASS—18 Law, 20 Language, Communication and Culture

Engagement discipline deep dives for HASS disciplines of Law and Language, Communication and Culture. Common themes included:

• Working with policy makers and public interest groups – influence policy
• Involvement in developing/reviewing policy and professional standards
• Invitations from industry and governments to provide expert review/advice
• Invitations to provide submissions to inquiries
• Involvement in international agencies – WHO, UNICEF, UN
• Provision of access to specialist databases/repositories
• Provision of training for professionals

[back to top]

High rated approach to impact

Common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20)

Common themes from high rated ‘approach to impact’ submissions:

• Multi-disciplinary research centres
  • important for creating larger research/end-user networks
• Formal partnerships
• Strong institutional, administrative and legal support to staff
  • flexibility within staff contracts to support engagement – reduce teaching hours, providing time for collaboration and support for travel
• Co-location of research facilities
• Knowledge transfer experts
• Embedded/co-located staff and HDR students
• Multi-media platforms
• Competitive grants (supporting indirect costs)

[back to top]

Approach to impact—discipline deep dives

STEM—06 Biological Sciences, 09 Engineering

Approach to impact discipline deep dives for STEM disciplines of Biological Sciences and Engineering. A common theme was:

• Support for multi-disciplinary research centres, which was important for creating larger research/end-user networks.

[back to top]

Approach to impact—discipline deep dives

HASS—18 Law, 20 Language, Communication and Culture

Approach to impact discipline deep dives for HASS disciplines of Law and Language, Communication and Culture. Common themes included:

• Support for workshops, education and training services for training professionals and local communities
• Public seminars
• Websites and other social media
• Hiring across a broad range of expertise – including industry professionals
• Focus on providing early career researchers with engagement experience

[back to top]

Low rated engagement narratives—common themes

Common themes from low rated ‘engagement’ narratives:

• Engagement activities and research not clearly linked or well integrated
• Limited evidence of overarching engagement strategy and/or supporting processes
• Engagement indicators were comparatively low for the discipline with little explanation
• Little to no engagement activity in the reference period
• Limited evidence of mutually beneficial engagement

[back to top]

Low rated impact studies—common themes

Common themes from low rated ‘impact’ studies:

• Insufficient evidence to support the claims regarding research impact
• Identified potential impacts not actual impacts
• End-user impacts beyond academia not clearly articulated or of limited significance
• Impact stemmed from a non-research activity (e.g. training) or activity outside the reference period
• Link between research activity and claimed impact unclear.

[back to top]

Low rated approach to impact—common themes

Common themes from low rated ‘approach to impact’ submissions:

• Impact driven by individual(s) not the university
• Little evidence presented on university’s role
• University support considered business as usual
• University support not demonstrated to be strategic, well integrated or with the intention of facilitating impact

[back to top]

Aboriginal and Torres Strait Islander Research Results and Insights

Ratings for Aboriginal and Torres Strait Islander research

Charts showing the distribution of 'impact' and 'approach to impact' ratings for Aboriginal and Torres Strait Islander research.

[back to top]

High rated impact studies—who benefited?

High rated Aboriginal and Torres Strait Islanders studies—who benefited?

• Aboriginal and Torres Strait Islander people and communities across Australia
• professionals and industry groups
• general public

Examples:

• health care workers
• teachers and students
• artists and performers
• rangers
• journalists and publishers
• curators
• police officers and court officials
• business owners
• policy makers

[back to top]

Types of impacts

Types of impact (high rated Aboriginal and Torres Strait Islander impact studies):

• Improved physical and mental health
• Preserving and sharing of languages and culture
• Improved educational outcomes – school to tertiary
• Environmental preservation and management
• System wide legal changes
• Increased knowledge of First Nations history
• Economic savings in health and education

[back to top]

Additional impacts

Additional impacts:

• employment and upskilling of community members
• capability developed in industry professionals, and
• resources created and shared with sectors and public.

Mutual knowledge exchange was key.

[back to top]

Key themes of approaches to support impactful outcomes

Key themes of approaches to support impactful outcomes:

• Leadership and governance
• University investment, infrastructure and support
• Partnerships and communication

[back to top]

Examples of leadership and governance arrangements

Approach to impact – examples of leadership and governance arrangements:

• Frameworks
• Senior appointments 
• Board appointments
• Advisory councils
• Strategic Plans
• Capability building next generation of leadership

​​​​​[back to top]

Examples of university investment, infrastructure and support

Approach to impact – examples of university investment, infrastructure and support:

• Financial support
• Infrastructure 
• Administrative support
• Build next generation of researchers

​​​​​[back to top]

Examples of partnerships and communication approaches

Approach to impact – examples of partnerships and communication approaches:

• Formal agreements
• Long-term partnerships
• Appropriate communication and media
• Training and capability

​[back to top]

ERA and EI next steps

ERA and EI next steps include the Review of Research Evaluation in Australia, which will consider:

• Key learnings ERA and EI
• Frequency of evaluations
• Reporting burden
• Sector feedback
• Various research reviews/reports-recommendations 

​[back to top]

Review of ANZSRC

Review of the Australian and New Zealand Standard Research Classification (ANZSRC):

• Joint review by the ARC, ABS, Statistics New Zealand, and the New Zealand Ministry of Business, Innovation and Employment
• Review will ensure research classifications reflect current practice and remain responsive to change in the sector
• Discussion Paper published earlier this year and public consultation closed on 7 June 2019
• Feedback is currently being examined and proposed revisions to ANZSRC will be published for comment later in 2019.
• Review anticipated to be completed mid-2020

​[back to top]