The Australian Research Council and Research Impact
AIATSIS National Indigenous Research Conference
21 March 2017

Dr Fiona Cameron—Executive Director for Biological Sciences and Biotechnology 
Kylie Emery—Acting Executive General Manager 
Sarah Howard—Acting Branch Manager Research Excellence

Presentation slide header

Outline

  • ARC National Competitive Grants Program
    • Kylie Emery
  • Quick overview of ARC schemes, DI and ROPE
    • Sarah Howard
  • Engagement & Impact Pilot
    • Dr Fiona Cameron 
  • Pathways to research impact
    • Writing an impact statement

[top]

Presentation slide header

National Competitive Grants Program

description below

Graphical representation of schemes in the ARC's National Competitive Grants Program. Each scheme is a rectangle with the area of the rectangle representing ARC funding (new and ongoing projects) for 2016.

Area of box represents ARC funding by scheme (new and ongoing projects) for 2016*. 

N.B.

  • *LP16 figures not including Continuous Linkage

[top]

Presentation slide header

Discovery Projects—return and success rates

Discovery Projects—return and success rates

Chart showing Discovery Projects scheme return and success rates 2009–2017.

[top]

Presentation slide header

Discovery Indigenous—return and success rates

Discovery Indigenous—success rates

Chart showing Discovery Indigenous scheme success rates 2009–2017

[top]

Presentation slide header

ARC Policy Statement: Eligibility and Career Interruptions

  • The ARC promotes and encourages, within the NCGP, approaches to both eligibility and assessment, which take into account the diversity of career and life experiences of individual researchers.
    • In relation to eligibility for a scheme, the eligibility period for early-career awards and mid- career fellowships may be extended to take into account career interruptions experienced due to specified career and life experiences.
    • In relation to assessment within a scheme, the ARC uses the selection criterion Research Opportunity and Performance Evidence (ROPE) to assess the quality of individual researchers
  • ARC Policy Statement: Eligibility and Career Interruptions

[top]

Presentation slide header

Aboriginal and Torres Strait Islander researchers and research

 

National Competitive Grants Program

Excellence in Research for Australia / Engagement and Impact Assessment

 

Policy development and implementation

 

[top]

Presentation slide header

Current elements of support under the NCGP

  • Aboriginal and Torres Strait Islander researchers
    • NCGP—all schemes
    • Discovery Indigenous
    • NIRAKN
    • AIATSIS—eligible organisation
  • Aboriginal and Torres Strait Islander research
    • NCGP—all schemes
    • Codes of research
    • Australian Government Science and Research Priorities

[top]

Presentation slide header

ARC Research Workforce Statement (December 2015)

  • Complete—ARC statement of support and expectations for gender equality (December 2015)
  • Complete—ARC policy statement: Eligibility and career interruptions (September 2016)
  • Complete—ARC statement of support for Aboriginal and Torres Strait Islander Researchers
  • Planned—ARC statement of support for early- and mid-career researchers

[top]

Presentation slide header

Statement of support―related activities

Flow chart of important dates in Statement of Support for Aboriginal and Torres Strait Islander Researchers and Research:

October 2016: Targeted consultation—draft Statement of Support

February 2017: Public release-Statement of Support for Aboriginal and Torres Strait Islander researchers

March 2017: Review—ARC support for Aboriginal and Torres Strait Islander researchers and research

March 2017: Consultation—ARC support for Aboriginal and Torres Strait Islander researchers and research

July 2017: Public release—funding rules

[top]

Presentation slide header

ARC Action Plan 2017

  • provide information on its website about ARC initiatives and data
  • review NCGP support for Aboriginal and Torres Strait Islander researchers and research
  • ARC committees include Aboriginal and Torres Strait Islander members
  • Aboriginal and Torres Strait Islander researchers participate as ARC assessors
  • highlight Aboriginal and Torres Strait Islander researchers’ achievements and research outcomes
  • continue to liaise with relevant groups 
  • monitor the engagement of Aboriginal and Torres Strait Islander researchers under NCGP schemes, including the Discovery Indigenous scheme

[top]

Presentation slide header

What our data reveals

  • Discovery Indigenous scheme
  • Aboriginal and Torres Strait Islander research

[top]

Presentation slide header

Discovery Indigenous, number of individual awards or fellowships (2002–2017)

51 researchers have been the recipient of 71 awards / fellowships

34 researchers have been awarded ONE award or fellowship

14 researchers have been awarded TWO awards or fellowships

3 researchers have been awarded THREE awards or fellowships

[top]

Presentation slide header

Discovery Indigenous, funding by 2-digit FOR (2012–17)

Discovery Indigenous, funding by 2-digit FOR (2012–17)

Bubble chart showing Discovery Indigenous, funding by 2-digit FOR (2012–17). Size of bubble indicates funding amount.

[top]

Presentation slide header

Discovery Indigenous, number of research projects by 2-digit FoR and Administering Organisation (2012–2017)

 Discovery Indigenous, number of research projects by 2-digit FoR and Administering Organisation (2012–2017)

Circular relationship graph showing Discovery Indigenous projects by 2-digit FoR and Administering Organisation (2012–2017).

[top]

Presentation slide header

Discovery Indigenous, number of projects funded by 2-digit FoR (2012–2017)

 Discovery Indigenous, number of projects funded by 2-digit FoR (2012–2017)

Discovery Indigenous Fields of Research, 2-digit FoR codes, funded projects 2012 to 2017 IN 2012 to 2017

Notes

  • The graphs show the relationship between Fields of Research at the 2-digit level.
  • Projects are linked to a 2-digit FoR if they have at least one 6-digit code belonging to that classification.
  • Projects are linked to a 2-digit FoR if they have at least one 6-digit code belonging to that classification. In the example below the project would have one link to Chemical Sciences and one link to History and Archaeology.
  • DP130100077 030505 Chemical Sciences
  • DP130100077 030501 Chemical Sciences
  • DP130100077 210203 History and Archaeology 

[top]

Presentation slide header

Aboriginal and Torres Strait Islander research, number of projects funded by 2-digit FoR (2012–2017)

Aboriginal and Torres Strait Islander research, number of projects funded by 2-digit FoR (2012–2017)

Discipline map of Indigenous related research, 2-digit FoR, funded projects 2012 to 2017 Note: Green/yellow = HASS; Blue = STEM

Method

Indigenous related research has been identified using three criteria:

  1. Scheme name for 'Discovery Indigenous';
  2. 6-digit level FoR and SEO code;
  3. National Research Priority for Indigenous research including Future Fellowship priorities (see 'Criteria' sheet)

Between 2012 and 2017, 1,085 projects were identified using this method.

These projects have been mapped in relation to 2-digit FoR.

Projects are linked to a 2-digit FoR if they have at least one 6-digit code belonging to that classification. In the example below the project would have one link to Chemical Sciences and one link to History and Archaeology.

  • DP130100077 030505 Chemical Sciences
  • DP130100077 030501 Chemical Sciences
  • DP130100077 210203 History and Archaeology 

 

[top]

Presentation slide header

Aboriginal and Torres Strait Islander research, funding by scheme (2002–2017)

Scheme

Funding

Special Research Initiatives

 $           3,198,392

Linkage Infrastructure, Equipment and Facilities

 $           6,100,000

Australian Laureate Fellowships

 $           8,883,151

Discovery Early Career Researcher Award

 $          10,281,362

Discovery Indigenous

 $          24,506,646

ARC Future Fellowships

 $          25,008,561

Linkage Projects

 $          33,863,579

Discovery Projects

 $          34,594,552

ARC Centres of Excellence

 $          61,750,000

Grand Total

 $        208,186,243

 

[top]

Presentation slide header

Aboriginal and Torres Strait Islander research, Linkage Projects partner organisations (2012–2016)

 Aboriginal and Torres Strait Islander research,Linkage Projects partner organisations (2012–2016)

Circular relationship graph showing Aboriginal and Torres Strait Islander research, Linkage Projects partner organisations (2012-2016).

[top]

Presentation slide header

The consultation

  • To look at
    • current support under NCGP
    • possible future funding models
  • Including
    • barriers to participation?
    • key characteristics of DI scheme?

[top]

Presentation slide header

 

A National Engagement and Impact Assessment—What is it?

A new national assessment which looks at:

  • how well researchers in Australian universities engage with the people who use their research—outside of academia
  • what kinds of impacts research undertaken in Australian universities is having—beyond academia
  • how well Australian universities are supporting their researchers to deliver research which has an impact—beyond academia

 [top]

Presentation slide header

Why are we doing it?

  • National Innovation and Science Agenda (NISA)
    • announced 7 December 2015
    • $1.1 billion over 4 years
  • A range of new initiatives to
    • support research
    • encourage innovation and entrepreneurship
    • reward risk taking
    • promote science, maths and computing in schools

 [top]

Presentation slide header

Why…?

Encouraging innovation and entrepreneurship by…

  • Encouraging researchers in Australian universities to
    • better engage with users of their research
    • better focus on delivering research impacts
  • Encouraging Australian universities to support their researchers in these activities

 

 [top]

Presentation slide header

When?

2016

  • sector consultation—universities, research end-users, broader public
  • development of pilot methodology 

2017

  • Development of pilot framework and guidelines 
  • Pilot assessments—mid 2017
  • Panel nominations for 2018—later in 2017

2018

  • Full assessment—as a companion to ERA

 

 [top

Presentation slide header

What is ERA ?

ERA evaluates the quality of the research undertaken in Australian universities.

 [top

Presentation slide header

Key features of ERA

  • Every 3 years—2009 (trial), 2010, 2012, 2015, 2018…
  • Comprehensive—covers all eligible researchers and their research outputs (and other data) 
  • Uses metric indicators or peer review—depending on the discipline
  • Evaluation against national and international benchmarks 
  • Ratings determined by committees of distinguished researchers from Australia and overseas
  • The unit of evaluation—the Field of Research (FoR) at the university (ANZSRC)
    • Ratings at 2-digit (broad field) and 4-digit (specific discipline) levels
    • 16—Studies in Human Society
      • 1601—Anthropology
        • 1607—Social work
  • Rating scale
    • 5 point scale against world standard
      • 5—the quality of the research is well above world standard
      • 3—the quality of the research is at world standard
      • 1—the quality of the research is well below world standard

 [top

Presentation slide header

How does ERA measure research quality?

Quality assessments focus on the outputs:

  • A range of metrics—citation profiles—mostly for the natural sciences; or
  • peer review of a sample of research outputs—more broadly common in the humanities and social sciences

Other supplementary indicators

  • Research income—HERDC cats 1–4
  • Applied measures—patents, plant breeder’s rights, registered designs, NHMRC-endorsed guidelines, research commercialisation income
  • Esteem measures—e.g. membership of a Learned Academy or AIATSIS etc. not for 2018….

[top

Presentation slide header

 Key principles

For 2018

  • all 42 Australian universities (defined by the HESA 2003) will be eligible to participate
  • all research disciplines will be involved
  • the methodology will not advantage one discipline over another
  • evaluations will be conducted by committees of experts from the university sector and industry
  • Unit of Assessment (UoA)—2-digit FoR at the university

[top

Presentation slide header

ERA/EI

  • ERA and EI will run as companion exercises
  • ERA will continue to assess research quality
  • ERA acknowledges and encourages ‘blue sky’ research
  • Engagement and impact assessment will consider:
    • research interactions with industry, Government, non-governmental organisations, communities and community organisations, and
    • research contributions to economy, society and environment

[top

Presentation slide header

Definitions

Engagement
Research engagement is…

  • the interaction between researchers and research end-users (including industry, government, non-governmental organisations, communities and community organisations) 
  • for the mutually beneficial exchange of knowledge, technologies and methods, and resources 
  • in a context of partnership and reciprocity.

Impact
Research impact is…

  • the contribution that research makes to the
    • Economy
    • Society
    • Environment
  • beyond the contribution to academic research.

[top

Presentation slide header

Engagement and Impact Assessment Pilot 

  • Two separate pilot studies—engagement and impact
  • All Australian universities are eligible to participate—participation is voluntary
  • 40 Australian universities (defined by the HESA 2003) are participating
  • The pilot will be reviewed—outcomes inform the full assessment in 2018
  • Institutions will submit information by Fields of Research—2-digt and 4-digit
  • The pilot will test a sample of those fields across engagement and impact
  • Panels of experts—academic and research end-user expertise
  • Indigenous research is being treated as its own discipline in the impact pilot

[top

Presentation slide header

Pilot assessment framework

 

Engagement

Impact

GROUP A

03 Chemical Sciences

11 Medical and Health Sciences

GROUP C

05 Environmental Sciences

07 Agricultural and Veterinary Sciences

09 Engineering

 

GROUP B

21 History and Archaeology

22 Philosophy and Religious Studies

GROUP D

13 Education

19 Studies in Creative Arts and Writing

20 Language, Communication and Culture

 

Indigenous Research Impact Studies

 [top]

Presentation slide header

Engagement—a narrative

  • Provides context for the engagement indicators
  • Allows institutions to describe engagement activities where the engagement indicators are not sufficient
  • Allows institutions to provide additional quantitative information, where relevant, which may be used by the ARC to develop indicators for subsequent rounds

 [top]

Presentation slide header

Impact studies

  • Impact will be assessed primarily using qualitative information in the form of impact studies
  • Impact studies typically detail
    • Research underpinning the impact
    • Approach to impact
    • Details of the impact
  • Impact studies will allow for the inclusion of measures or indicators of impact

[top]

Presentation slide header

 
 

Questions

[top]

Presentation slide header

Acknowledgement of Research Impact PFRA working group

  • AIATSIS: Lisa Strelein, Dylan Marsh, 
  • AIMS: Sue English
  • ANSTO: Herma Buttner, Mike Siers
  • ARC: Fiona Cameron (chair), Liz Visher, Tim Cahill, Alistair Gibson, Amy Phillips, John Murray, Sarah Howard
  • BOM: Peter May
  • CSIRO: Mark Bazzacco, Renate Hays, Mark Johnson, Jennifer Kelly
  • DSTO: Mark Heinrich
  • GA: Rhonda Henry
  • NHMRC: Marcus Nicol, Henadeera Kumara
  • NMI: Bruce Warrington

 

[top]

Presentation slide header

Research Impact Pathway 

Examples only:

Inputs

Activities

Outputs

Outcomes

Benefits

  • Research income
  • Staff
  • Background IP
  • Infrastructure
  • Collections
  • Research work and training
  • Workshop/Conference organising
  • Facility use
  • Membership of Learned Societies and Academies
  • Community and stakeholder engagement
  • Publications including e-publications
  • Additions to national collections
  • New IP: Patents and Inventions
  • Policy briefings
  • Media
  • Commercial products, licences and revenue
  • New companies—spin offs, start ups or joint ventures
  • Job creation
  • Implementation of programs and policy
  • Citations
  • Integration into policy
  • Economic, Health, Social, Cultural, Environmental, National Security, Quality of Life, Public Policy or Services
  • Higher quality workforce
  • Job creation
  • Risk reduction in decision making

 [top]

Presentation slide header

Impact: Realised or Aspirational?

Impact Statement

Project Description

ROPE

Best publications (30-word)

Aspirational

Realised and Aspirational

Realised

Both realised and on the pathway to research impact

[top]

Presentation slide header

Tips for writing a good impact statement

Keep it simple—clear, concise language
Answer 6 basic questions:

  • Who is the intermediary user or customer being targeted?
  • What is to be improved / changed?
  • What does the ultimate outcome and impact look like?
    • Outcomes—e.g. contributing to national/global knowledge, national preparedness to respond
    • Economic impact—e.g. employment, new industries, increased profit & productivity
    • Social impact—e.g. improved health & wellbeing, safety, increased social cohesion
    • Environmental impact—e.g. reducing pollution, maintaining biodiversity
  • What is the new technology or solution or service?
  • What is your target(s) (e.g. end date and/or monetary value)?
  • Is it a cohesive and compelling case for investment?

[top]

Presentation slide header

Impact statement: Aspirational

 please see text below

Impact statement showing activity, output, outcome and benefit.

Example: 

"The candidate will work and collaborate with local councils on this project which is focused on the palaeoenvironmental development of National Parks in Victoria. This will produce publications in high ranking journals, which will generate broad discussions in the literature. The work will inform the design of new parks and particularly the way visitors can interact with a park. It is expected that new designs will allow enhanced educational experiences without damage to important heritage sites."

[top]

Presentation slide header

Research Impact: ROPE realised

please see text below

Research impact statement showing activity, output, outcome and benefit.

Example: 

"The candidate has successfully worked and collaborated on a number of projects with the most recent ones focusing on the palaeoenvironmental development of National Parks in Victoria. In addition to 3 publications in high ranking journals, which are gathering citations and generating broader discussions in the literature, the candidate has worked closely with local councils. The work has made a major impact in informing the design of new parks and particularly the way visitors can interact with the park. One new area has already been designed to allow educational experiences without damage to important heritage sites."

[top]

Presentation slide header

Research Impact

Example A

Examples of research impact statements showing activity, output, outcome and benefit.

Example:

"The candidate has collaborated on a number of full book length studies into the relationship between the theory of consciousness and recent discoveries in neuroscience and is working closely with the journal Philosophy in developing a series specifically dedicated to exploring this relationship. The studies have been cited 35 times in articles in highly ranked journals for illuminating new interdisciplinary approaches in understanding the meaning of consciousness."

Example B

Examples of research impact statements showing activity, output, outcome and benefit.

Example:

"The candidate has undertaken a number of innovative database projects recently for Museum Victoria. In addition to 4 exhibitions in first tier museums, the candidate has worked closely with peak industry bodies. Evaluation of the work has been cited in Leonardo as being used by museums to design audio-visual technology to enhance visitors on-screen and off-screen experience of their physical collections. Museum Victoria’s Board has commissioned a review to access the pilot implementation of the key outcomes of the work."

[top]

Presentation slide header

Best publications; 30–word statements realised and on track to benefit

Text:

  • With 48 citations, including Cambridge Companion to Australian Art, noted as “a benchmark for the interactive exploration of 3D multi-modal data”, uptake by broadcasting organisation NDR, (North German Broadcasting).
  • Cited in 30 sources including The Historical Journal, it demonstrates effective use of social media data in exploring contemporary refugee histories; impact noted in EU policy debates, The Times (03.01.15.p35).

[top]

Presentation slide header

Who are the statements & descriptions for?

  • Peers (College of Experts, other academics)
  • Unlikely to be your direct peers (COI)
  • Broad understanding of the area
  • Write to this audience

[top]