Introduction

Hello everyone and welcome to the ARC Centres of Excellence commencing in 2023 or CE23 assessor information video.
I begin by acknowledging the traditional owners of the country throughout Australia and pay my respects to elders, past, present and emerging. I extend that respect to Aboriginal and Torres Strait Islander people viewing this video today.
This video provides information, instructions and advice for both detailed and general assessors on the assessment process with the ARC Centres of Excellence all applications.

National Competitive Grants Program (NCGP)

Through the National Competitive Grants Program, or NCGP, the ARC supports the highest quality, fundamental, and applied research and research training across all disciplines except clinical medical research.
The ARC encourages partnerships between researchers and industry, government, community or community organisations and the international community.
The NCGP comprises 2 main elements – Discovery and Linkage –  under which the ARC funds a range of complementary schemes to:

  • Support researchers at different stages of their careers
  • Build Australia's research capability
  • Expand and enhance research networks and collaborations
  • Develop centres of research excellence

This slide provides a graphical representation of schemes in the ARC NCGP. Each scheme is a rectangle with the area of the rectangle representing the proportionate scale of ARC funding for 2020.

The ARC Centres of Excellence scheme is part of Linkage Program and is considered one of the ARC major investments.

Linkage Program

The ARC's Linkage Program funding schemes aim to encourage and extend cooperative approaches to research and improve the use of research outcomes by strengthening links within Australia's innovation system and with innovation systems internationally.

The Linkage Program schemes are: 

  • Linkage Projects
  • Industrial Transformation Research Program (ITRP)
  • ARC Centres of Excellence
  • Linkage Infrastructure, Equipment and Facilities
  • Special Research Initiatives
  • Linkage Learned Academies Special Projects

With previously mentioned the ARC Centres of Excellence scheme sits within this Linkage Program.

ARC Centre of Excellence – Overview

The ARC Centre of Excellence scheme is designed to:

  • Create a national research entity
  • Build research capability and capacity in areas of national importance
  • Develop scale and focus necessary for Australia to achieve international standing in those areas
  • Scheme funds, world class, internationally competitive research teams investigating and finding solutions to important and challenging Australia international problems with a clear focus on achieving outcomes by 2030.
  • Research undertaken in the ARC Centres of Excellence, is expected to be interdisciplinary, innovative and transformational.
  • Funding of up to $5 million per year for up to seven years is available from the ARC for each funded ARC Centre of Excellence.

ARC Centre of Excellence – Objectives

When assessing full applications for ARC Centres of Excellence scheme, assessors must consider both the objectives of the scheme and the assessment criteria for full applications and is outlined in the grant guidelines for the ARC Centres of Excellence funding commencing in 2023.

Grant guidelines can be accessed on Grant Connect website www.grants.gov.au

ARC Centre of Excellence – Assessment Stages

The ARC Centres of Excellence is selected through a competitive three stage process. Expressions of interest (EOI) stage was conducted during 2021 and is now complete.

The scheme is currently at the full application stage where shortlisted expressions of interest or EOIs have been indicted by the ARC to submit full applications for the ARC Centres of Excellence for funding commencing in 2023.

Full applications are assessed by both Detailed and General Assessors.

  • Detailed Assessors complete in depth assessments of full applications providing comments and scores against each of the 5 assessment criteria separately.
  • Applicants are able to read comments only (not the scores) from detailed assessors and submitted rejoinder.
  • General assessors, the Selection Advisory Committee (SAC) will consider each full application against the assessment criteria, taking into consideration information from the full application, the detailed assessments, and the rejoinder.

SAC Members will meet before the interview stage to decide whether all full applications should proceed to the interview.

A High Quality Full Application – Key Points It Should Address

The assessment criteria for full applications under the ARC Centres of Excellence grant opportunity are:

  • Researched program
  • Quality innovation
  • Investigators
  • Institutional support
  • Governance, leadership and measuring
  • Outcomes and linkages

All equally weighted.

Only those applications that are truly excellent across all five assessment criteria are recommended for funding.

As a guide those in the top scoring band A would be assessed as flawless or near flawless in each assessment criteria.

All Detailed Assessors are typically assigned one to two full applications to assess.

When assessing a full application for the ARC Centres of Excellence scheme, an assessor should consider a number of questions.

  • Will it create a national entity?
  • Is it interdisciplinary in it’s approach?
  • Is it proposing something new, transformative and innovative?
  • It is an ARC Centre of Excellence, it should not be a Centre of research.
  • Why is it important?
  • Why does it have to be done now?
  • Why is it particularly beneficial to Australia?
  • Does it have a vision of success over 7 to 10 years?
  • Does it meet the ARC scheme objectives and explain how these are being met?
  • Is the research program at the right scale? Are they not too narrow or too broad?
  • How does a Centre fit within the international research landscape?
  • Does the Centre have the best and most appropriate and relevant team?
  • Are there any critical personnel or groups missing?
  • Are named participants demonstrating their commitments to the research program?
  • Are the participants, creating collaborative and integrated team?
  • How well does the Centre align with the strategic objectives of the participating organisations?
  • Is there appropriate and sufficient support and commitment from the participating organisations?
  • Is the management and government best practice?
  • Are performance measures, milestones or KPIs appropriate and realistic?
  • How does the sender intend to deliver research outcomes and translate these to users and others?
  • Does the research leadership have the right skills to manage and lead a Centre?
  • Is it offering innovative training programs, mentoring a professional development for all staff students, postgraduate and postdoctoral researchers?
  • How well does it support and develop innovative programs to enable gender equality, diversity and inclusion?
  • How appropriate or novel are the education engagement programs?
  • What will be the impact of the research over the Centres duration and beyond?
  • Does the is Centre have robust, feasible, inclusive strategies application and transfer of knowledge and outcome?
  • and ultimately, is it value for money?

Detailed Assessments

  • Detailed Assessors may receive applications to assess at any stage of the assessment process.
  • This is due to late conflicts of interest being declared by other assessors.
  • As a Detailed Assessor you'll be asked to review ARC Centres of Excellence applications within your broad discipline field (noting that these types of grants maybe disciplinary)
  • Detailed Assessments must meet the minimum character count 200 characters per assessment criteria, and minimum a 3,500 characters for the overall assessment.
  • If you have any concerns regarding the eligibility or integrity of an application, you should raise them with the ARC separately and not include these issues in the assessment.
  • Detailed assessors must assess using the information contained in the application. You must not seek information from other sources, and this includes the expression of interest application if you assessed it in 2021.

High Quality Detailed Assessments

Details Assessors are asked to provide high quality constructive assessments with the following elements:

  • Clearly address the scheme, objectives and criteria
  • Thoughtful, meaningful and balanced assessments with objective, fair and meaningful and professional comments
  • Detailed comments on the merits of the application, which provides sufficient information to allow applicants to provide an informative and reasonable rejoinder
  • Comments and questions and constructive criticism, so they're justified and align closely to the scores
  • Comments free from exaggeration and understatement
  • The comments are designed to enable the Selection Advisory Committee (SAC) to make informed decisions about funding recommendations
  • The assessment should not include any inappropriate elements as outlined in the Assessor Handbook.
  • and do not use outside sources of information beyond what the ARC provides in Research Management System (RMS).

Then please submit by RMS by the ARC deadline, with observations of our Conflict of Interest and Confidentiality Policy.

Please refer to our ARC peer review webpage for examples of good, detailed assessments.

If necessary, the ARC may contact you to request clarification of your comments.

Scoring and Ranking Assessments

The Scoring Matrix table on the screen shows an example of the scores applicable to the ARC Centres of Excellence grant opportunity.

It is important that Detailed Assessors follow the Scoring Matrix.

The assessment comments must fit the scores you give. For example, if you give a score of D, your comment should draw attention to the possible weaknesses or areas of concern which have led you to give that score.

Assessments Screen 1 for Detailed Assessors

This busy slide provides a screenshot of an assessment form for Detailed Assessors. In this screen, Detailed Assessors are able to open the application and enter the assessment screen, access assessor material and submit scores and ranking.

Assessments Screen 2 for Detailed Assessors

This slide and the following slide provide screenshots of the assessment screen where Detailed Assessors are required to input their written comments and scores against the application to which they are assigned.

Please note, that RMS does not autosave and the save button on the top of the screen can be used if the assessment is not complete or ready for submission.

The Role of an ARC General Assessors

  • General Assessors are selected to form an ARC Selection Advisory Committee or SAC, to conduct the peer review process and provide recommendations to the ARC.
  • The Selection Advisory Committee for the ARC Centres of Excellence scheme, has eminent members of the wider national and international academic community and/or key industry groups on it.
  • General Assessors provides scores only in RMS.
  • The SAC attends applicant interviews and the selection meeting to discuss and recommend applications.

General Assessments

General Assessors are asked to review the full application to identify the merits or otherwise of the application and provide a score against the assessment criteria.

General Assessors may also receive applications to assess at any stage of the assessment process due to late conflicts of interest being declared by other General Assessors.

Each application will have four General Assessors assigned to provide an assessment.

Prior to the interviews in the selection meeting all General Assessors who are not conflicted with an application should review the assessments and scores and rejoinder each submitted application.

More information on the applicant interviews and selection meeting will be provided to the SAC members as the time for this process approaches.

Assessments Screen for General Assessors

This slide provides a screenshot of the assessment screen for General Assessors. From this screen, General Assessors are able to open the applications, enter the assessment screen, access the assessor material and submit their scores and rankings.

Applications Outside Your Broad Area of Expertise

The ARC Centres of Excellence are inherently interdisciplinary research entities.

Assessors may be assigned to applications outside their field of expertise.

If you have any concerns, please contact the ARC before rejecting the assignment.

Please review your RMS profile, expertise text and Field of Research (FoR) codes regularly to ensure that your profile remains accurate and up to date.

Please ensure that both 2008 and 2020 FoR codes are listed in your profile as these will be used in the full application into assign assessors.

Assessors are required to assess all research on a fair and equal basis, including applications and outputs involving interdisciplinary and collaborative research.

Avoiding Inappropriate Assessments

Here are some tips in preparing an assessment and common issues that can arise.

  • Assessments must only be based on the information contained in the applications.
  • Ensure that text meets the minimum count of 200 characters per criterion, with a minimum of 3,500 characters to the overall assessment
  • Please contact ARC directly if you have any concerns regarding the eligibility or integrity of an application
  • Ensure that you do not provide information in your assessment text that could identify you
  • Provide comments that are specific to each assessment and do not compare one application to another
  • Do not include comments that could be perceived to be discriminatory, defamatory or distastefully irrelevant
  • Do not restate or rephrase excessive parts of the application
  • Do not use the scores or acronyms within the assessment text
  • Do not simply quote the rubric or restate the metrics provided by the applicant

Treatment of Inappropriate Assessments

  • The ARC reviews all detailed assessments before applications for the ARC Centres of Excellence, including those that have been identified to contain inappropriate comments.
  • Inappropriate assessments can compromise the integrity of the peer review process. To be fair to all applicants, ARC will reject assessments with inappropriate or highly subjective comments from individual assessors at any aspect of the application.
  • The ARC has absolute discretion to decide what action will be taken. This can include requesting a Detailed Assessors to amend their assessment text or removing the assessment from the peer review process altogether.
  • The ARC website also contains information for applicants advising how to request the ARC reviewing assessment if they believe it contains inappropriate elements during the rejoinder period.

The Research Management System (RMS)

  • RMS is a web-based computer system for the preparation and submission of each research application, and assessments and rejoinders to the ARC.
  • All assessors use RMS to submit their assessments.
  • To assist General and Detailed Assessors navigate to the assessment functionality in RMS there is a RMS Handbook for assist is available on the ARC website.
  • As Detailed Assessors, you should ensure that:
    • Your RMS profile is kept up to date
    • Your expertise text should highlight your discipline specific and/or interdisciplinary knowledge including areas of your specialty and it techniques. It is not a career summary, but an expertise summary
    • Please select your most relevant fields of research FoR codes
    • An accurate, up-to-date employment history helps minimize the chances that you will be assigned to an application with which you have an organizational conflict of interest

COVID-19

  • Assessors should assess each application based on the content of that application only and without making assumptions about the impact of COVID-19.
  • Assessments must not make any assumptions about the viability of a proposed ARC Centre of Excellence due to the potential impacts of COVID-19.
  • An assessment that includes assumptions about the impact of COVID-19 would be considered inappropriate assessment and will be handled accordingly.
  • Assessors should not make assumptions about the administering organisation, other eligible organisations, level of commitments and support of an application solely based on lower levels of pledged additional cash and/or in-kind support than provider historically for previous Centres of Excellence rounds.

Confidentiality and Conflict of Interest (COI)

The ARC Conflict of Interest and Confidentiality Policy is designed to ensure that all Conflicts of Interest (COI) are managed in a rigorous and transparent way. They are to prevent individuals from influencing the decisions unfairly and to maintain public confidence in the integrity, legitimacy, impartiality and fairness of the peer review process.

Any individual who is reviewing material from the ARC must agree to COI statement and must clearly disclose any material, personal interests that may affect or might be perceived to affect their ability to perform their role.

The information in your RMS profile will assist the ARC with the identification and management of organizational COI.

Failing to declare a COI may result in a Detailed Assessor being investigated for breaching the ARC research integrity policy.

Any COI queries, please contact the ARC using the peer review email address if you're a Detailed Assessor or the college email address if you're a General Assessor.

What to disclose?

In reviewing an application prior to accepting it for assessment, Detailed Assessors must disclose any close personal relationships with named participants on a research application, as well as any professional relationships with that named participant, including the following being some of the common examples:

  • Conjointly holding funding within the past two years
  • Having a current application for funding
  • Being a collaborator or co-author on a research output within the past four years
  • Being a co-editor of a book, journal, compendium, or conference proceedings within the past two years
  • Being a postgraduate student or supervisor of that named participant within the past five years
  • Benefiting materially from the awarding of funding to the application involving that named participant
  • and other situations as outlined in our COI policy on the ARC website

Unconscious Bias

As assessors, you should also be aware of how our unconscious bias could affect the peer review process. Unconscious biases are pervasive and may relate to the perceptions about a range of attributes including:

  • Gender and/or sexuality
  • Social/cultural background
  • Career path
  • Institutional employer
  • or even discipline

The ARC encourages you to recognise your own biases and be aware of them in your assessment.

All Detailed Assessors are encouraged to complete the Implicit Bias training offered by Harvard.

A Word on ARC Eligibility

If you have any concerns about the eligibility, ethics or other issues associated with an application, you must not include this information in your assessment.

Please send an email highlighting your concerns to the peer review mailbox as soon as possible.

Please complete your assessment based on the merits of the application without giving consideration to the potential eligibility issue.

The ARC NCGP eligibility committee is responsible for reviewing the eligibility of applications and will be advised of your concerns.

Key Contacts

We'd like to thank you for your contribution to the peer review process for full applications for the ARC Centres of Excellence commencing in 2023.

We greatly value your time and expertise and contribution to the assessment of the full applications.

Should you have any questions or require assistance, please contact any other people listed on this slide.

Conclusion

Thank you for watching this video on the assessment processes for the Detailed and General assessors for the full applications for the ARC Centres of Excellence commencing in 2023.

We thank you for your valuable contribution, the Australian research, through your participation in the ARC peer review process. Again, we can't thank you enough. Thank you.