Engagement and Impact Assessment (EI)

Background

On 7 December 2015 the Australian Government launched its National Innovation and Science Agenda (NISA). One of the measures within the agenda is for Australia to introduce a national impact and engagement assessment, which will examine how universities are translating their research into economic, social and other benefits and encourage greater collaboration between universities, industries and other end-users of research. 

Working closely with university and industry stakeholders and through a Steering Committee and working groups, the Australian Research Council (ARC) and the Department of Education and Training have been developing the framework for the assessment.  

In line with the NISA announcement in December 2015, a pilot was undertaken in 2017, providing a basis for a national roll-out in 2018. The national assessment will be undertaken as a companion to Excellence in Research for Australia (ERA).

The ARC reviewed the pilot and analysed the engagement and impact assessment methodologies, incorporating feedback from university, industry and end-user participants. This review will inform the development of the full assessment. Senator the Hon. Simon Birmingham, Minister of Education and Training, and ARC Chief Executive Officer, Professor Sue Thomas, have announced the release of the Engagement and Impact Assessment Pilot Report.  The release of the Pilot Report marks the end of the pilot and the start of preparations for the first full assessment launching in 2018.

Following consultation, the final key documents for the 2018 Engagement and Impact assessment (EI 2018) are now available.

EI Key documents

The key documents required for the 2018 Engagement and Impact Assessment (EI 2018) are:

If you have questions or require any of these documents in any other format, please email ARC-EI@arc.gov.au or call the EI Helpdesk on 02 6287 6755 (between 9.00am and 5.00pm, Canberra time, Monday to Friday).

Publication of EI Submission Information

Following the completion of EI assessments, the ARC will be releasing information submitted by universities. Full details are outlined in Publication of EI submission information page (also available in PDF format). 

If you have questions or require any of these documents in any other format, please email ARC-EI@arc.gov.au or call the ERA Helpdesk on 02 6287 6755 (between 9.00am and 5.00pm AEDT Monday to Friday).”

EI 2018 FAQs

General questions about submission

Is the EI low volume threshold based on apportioned indexed journal articles as for ERA citation analysis? Or on the equivalent of 150 weighted apportioned outputs as per ERA?

The low volume threshold for EI (as stated in the EI 2018 Submission Guidelines, Section 2.1) is 150 weighted outputs. This includes all outputs in the two-digit FoR, not just indexed journal articles. This will apply across all disciplines.

How will the ARC calculate the low volume threshold for FoR 11 now that it has split into two FoRs?

The threshold amount is for each part of FoR 11—Biomedical and Clinical Sciences and Public and Allied Health Services. For example, if a university met the threshold for Biomedical and Clinical Sciences but not Public and Allied Health Sciences then the university would only submit for Biomedical and Clinical Sciences. Where the university meets low volume threshold in both units of assessment, then the university will need to submit to EI in both.

Will a university be able to see the information already submitted to ARC as part of ERA 2018 during EI 2018 submission?

Yes, the information already submitted and certified by an institution will be made available for the institution to view in the System to Evaluate the Excellence of Research (SEER) web interface from the start of the EI submission phase, which is 17 May 2018.

How do universities submit engagement data for EI engagement?

Institutions are required to make submissions through SEER. In SEER, institutions will populate and then submit online web forms.

For the cash support from end-users indicator, the institution will use a spreadsheet template, which will be made available in SEER when EI 2018 submissions open on 17 May 2018.

Engagement submission

What weighting does the EI 2018 assessment give to the engagement narrative and the indicators?

Assessment panels make a holistic judgement about the performance of a UoA. There are no weightings applied to the individual components of the engagement submission and panels may focus on aspects of the qualitative statements or indicators that are particularly relevant for different disciplines.

Does an institution have to address all the additional indicators listed in Appendix F?

No, an institution does not have to address the additional indicators listed in Appendix F. Appendix F provides a non-exhaustive list of possible additional indicators. Institutions may or may not choose to include information about any or all of the indicators listed. Institutions may also choose to provide other additional quantitative information, where relevant. It is not compulsory to include additional quantitative information in the engagement narrative.

How is FTE calculated and used for EI indicators?

As stated in the EI 2018 Framework, FTE will be calculated using the academic staff at levels A to E with employment function of ‘Research only’, or ‘Research and Teaching’ that was submitted by institutions in the relevant part of their ERA 2018 submissions. The FTE amount is used in the total HERDC income per FTE indicator.

What happens if an institution doesn’t submit data for co-supervision of Higher Degree by Research (HDR) students?

Data on co-supervision of HDR students will not form part of the assessment for EI 2018. Universities will not be penalised if they do not submit this data. However, universities are encouraged to supply this data for EI 2018 as the ARC will look to develop this into an indicator for subsequent EI rounds.

Are voluntary members of staff who work for end-user organisations and are included in supervisory panels eligible as co-supervisors of HDR students?

If the person is in an employment relationship with the end user and included in the supervisory panel due to their expertise as an end-user, then the institution can count them as a co-supervisor even if they are volunteering at the university provided that there is at least one other supervisor from the submitting university.

Impact submission

Can information that is outside the reference period be included in the impact study?

As outlined EI 2018 Submission Guidelines (Section 2.2), the reference period for the impact study is 1 January 2011–31 December 2016 (6 years). The reference period for the associated research is 1 January 2002–31 December 2016 (15 years). While a reference period is not specified for approach to impact, the approach must be retrospective and within the context of the impact study.

Information that relates to activity or outcomes outside of the reference period should not be included. Impact studies may, however, refer to external evidence that verifies the claims being made, for example an auditor’s report on return on investment, even if this was published after the impact reference period.

Can an institution request not to be assessed for impact in a discipline?

If the institution meets the low volume threshold for impact and considers that this discipline in the institution either:

  • mainly focuses on fundamental research and not impact beyond academia or
  • is new to the institution and researchers in this discipline have not had sufficient time to have an impact beyond academia

then, the institution may submit a request not to be assessed for impact for that discipline, noting that requests not to be assessed will be published following completion of the EI process.

For further details please refer to the EI 2018 Submission Guidelines.

When should an institution submit a request to not be assessed for impact?

During the submission stage, institutions will be able to select ‘request not to be assessed’ in SEER and will then complete the request form instead of the impact study form for that UoA. Please note—requests not to be assessed will be published following the completion of the EI process.

Should research impact case studies be submitted to the FoR code based on the associated research or the impact?

The FoR code(s) should describe the overall content of the impact study. However, institutions should note that impact assessment is about the impact and the approach to impact, and not the assessment of the research associated to the impact.

Institutions can assign up to three additional FoR codes for the associated research. However, these codes provide context only and the impact study will only be assessed in the primary FoR code assigned to the overall impact study.

Can institutions attach supporting evidence and testimonials to impact studies?

No, institutions must not attach separate supporting evidence and testimonials to their impact studies. All information relevant to the impact study must be contained within the impact study itself.

Can institutions include URLs in the impact studies as supporting evidence?

No, institutions must not include URLs. The impact study should stand on its own merits without the need for assessors to access additional information. Institutions can reference evidence in the impact study to support claims but must not include any links or URLs.

What should the reference section of impact studies include?

The associated research sections of the impact template (Appendix G1 and G2 s5, 6 and 7, and Appendix G3 s4, 5 and 6) describe the research associated with the impact.

The references section relates to the associated research. Any references listed must be within the reference period for associated research, 1 January 2002–31 December 2016.

If required, references describing the impact that occurred may be included in the details of the impact section (Appendix G1 and G2 s4, and Appendix G3 s3).

EI Notifications

EI 2018 notification 002

The list of Specified HERDC Category 1 grants in Appendix E of the EI Submission Guidelines has been updated. 

EI 2018 notification 001

Section 1.8 of the EI Submission Guidelines has now changed. The section should now read

1.8 Timeline

Submission of EI 2018 narrative and data components occurs after ERA 2018 submission. 

Phase

Stage

Start Date

Deadline

Responsible

Submission

Submission

17 May 2018

6 July 2018

Institutions

Data integrity checking

9 July 2018

13 July 2018

ARC, with institutions

Certification

16 July 2018

18 July 2018

Institutions

[top]