Linkage Projects 2020 Round 3 Announcement Banner

Considering the quality of research and its translation to engagement and impact

Considering the quality of research and its translation to engagement and impact

ERA EI logo

ERA EI logo

It’s full steam ahead here at the ARC to deliver this year’s Excellence in Research for Australia (ERA) and Engagement and Impact Assessment (EI). With submissions now open for ERA 2018, and EI submissions to follow shortly, Australia’s universities are working hard on their input to these two important processes.

2018 will mark the fourth round of ERA since its inception in 2010. ERA identifies and promotes excellence across the full spectrum of research activity in Australia’s universities. It is a wide-ranging and comprehensive evaluation—in 2015, ERA assessed 2,460 units of evaluation involving over 430,000 unique research outputs produced by more than 67,000 researchers. By comparing Australia’s research effort against international benchmarks, ERA provides a national stocktake of areas of research strength, and, in turn, creates incentives for universities to improve the quality of their research.

Indeed, past ERA outcomes have shown that Australia’s universities produce high-quality research across a wide range of disciplines. For example, the results of ERA 2015 highlighted Australia’s national research strengths[1] in 43 different fields as diverse as pure mathematics, immunology, business and management, and cultural studies. Overall, the first three rounds of ERA (in 2010, 2012 and 2015) have shown steady improvements in performance across the university sector and across research disciplines.

EI will run for the first time in 2018 as a companion exercise to ERA 2018. This assessment will examine how universities are translating their research into economic, environmental, social and other benefits. EI aims to create incentives for greater collaboration between universities and industry, as well as other research end-users. EI has a focus on the wider benefits of research, while ERA continues to recognise and reward excellence in fundamental research. EI will also identify institutional processes and infrastructure that enable research engagement.

EI 2018 follows a successful pilot of the methodology in 2017. The development of the methodology was based on extensive consultation with universities, industry groups and leaders, and experts in research evaluation.

The methodology minimises burden on universities by using a small set of key quantitative engagement indicators that are, as far as possible, aligned with existing data sets. For both engagement and impact, the number of submissions is limited to one per broad field of research.

All assessments will be made by panels of experts from the university and industry sectors. Importantly, the assessment methodology provides flexibility through the collection of discipline-relevant qualitative and quantitative indicators. The methodology recognises and rewards the broad range of research engagement and impact in all disciplines, from the sciences and engineering, through to the social sciences and humanities. EI is also designed to recognise performance across the full range of Australian universities (whether they have a specialised or regional focus, or are major comprehensive universities).

For engagement, the methodology uses a combination of narratives and key quantitative engagement indicators—based on research income—to form a single rating for each discipline assessed. The quantitative indicators will be accompanied by an explanatory statement that allows the university to provide further explanation of the indicators. In addition, the university will provide an overarching engagement narrative, which will describe the university’s engagement activities for the unit of assessment. As part of this narrative, universities can include any other quantitative indicators that demonstrate the university’s research engagement for the discipline. To determine the engagement ratings, the assessment panels will consider all the quantitative and qualitative information submitted holistically and in the context of the discipline.  

For the assessment of impact, qualitative studies will be used to derive two ratings for each discipline—one rating for approach to impact (i.e. the university’s role in the translation of research into impact) and one for the impact that occurred. In addition to the discipline-based impact assessments, each university may submit an impact study for interdisciplinary research and an impact study for Aboriginal and Torres Strait Islander research:

  • The interdisciplinary impact study provides an opportunity to capture broader impact unable to be described by a single primary discipline.
  • The Aboriginal and Torres Strait Islander research impact study provides an opportunity to highlight impact with a specific focus on Aboriginal and Torres Strait Islander research[2]. This study may demonstrate the ways in which Aboriginal and Torres Strait Islander leadership and governance arrangements were integrated into the activities and processes throughout all stages; from the initial research, through translation, to the impact itself.

Both ERA and EI will play important roles in improving Australia’s research performance in the future. In 2013, an independent review by ACIL Allen Consulting, Benefits Realisation Review of Excellence in Research for Australia, found that ERA led to improvements in research performance, university planning, strategy and operation, and better accountability, transparency and policy-making. It helped identify cost savings for universities as they improved their management and planning, and provided opportunities for universities to promote their disciplinary strengths and gain greater international recognition.

The ARC expects similar benefits from EI. The transparent reporting of university performance will be helpful for university decision-making in areas of researcher/end-user collaboration. Publication of EI impact studies will allow universities to identify and implement sector best practices. It is likely that universities will see reputational benefits and it may help high-performers raise their profile with possible industry partners.

Finally, the increased accountability and transparency provided by both of these evaluations will allow the community to better understand the benefits of Australian public research funding. They will provide clarity to the Australian Government about how public investment in university research translates into international standing and tangible benefits. ERA and EI will support better-informed government policy-making and provide a framework to drive future policy discussions about research quality, engagement and impact.

More information on ERA and EI can be found on their respective pages on the ARC website: Excellence in Research for Australia and Engagement and Impact Assessment.

[1] Defined as specific disciplines where ten or more Australian universities were rated at above world standard or higher, including four or more Australian universities rated at well above world standard.

[2] The EI assessment defines Aboriginal and Torres Strait Islander research to mean that the research significantly:

    • relates to Aboriginal and Torres Strait Islander peoples, nations, communities, language, place, culture or knowledges and/or 
    • is undertaken with Aboriginal and Torres Strait Islander peoples, nations, or communities.
Back to top