Peer Review

This page provides an overview of the peer review process used for ARC funding schemes.

In addition:

 

Conflict of Interest

The ARC Conflict of Interest and Confidentiality Policy is designed to ensure that all conflicts of interest are managed in a rigorous and transparent way. It aims to prevent individuals from influencing decisions unfairly and to maintain public confidence in the integrity, legitimacy, impartiality and fairness of the peer review process.

Any individual reviewing material for the ARC must agree to a confidentiality and conflict of interest statement, and must clearly disclose any conflicts of interest which may affect their ability to perform their role.

Examples of conflicts of interest for an assessor include holding funding with a named participant within the past two years, or having been a collaborator or co-author with a named participant on a research output within the last four years. For more information on the timeframes that apply for common conflicts of interest, please refer to the Identifying and Handling a Conflict of Interest in NCGP processes page.

[top]

Assignment

For large schemes, the Selection Advisory Committee may be comprised of multiple discipline Panels. The Field of Research (FoR) codes nominated in the proposal will normally determine the Panel in which the proposal will be considered. ARC Executive Directors review the appropriateness of the proposals in each Panel and may transfer proposals to different Panels. They also identify interdisciplinary proposals that need General Assessors from more than one Panel.

The assessor’s role varies depending on the objectives of the scheme, the scale of funding support requested and the timing of the overall process.

The responsible Executive Director assigns at least two General Assessors to each proposal. One of these General Assessors will be designated as Carriage 1 and will be responsible for coordinating and leading discussion regarding the proposal at the selection meeting.

For some schemes, including Discovery Projects and Linkage Projects, General Assessors assign Detailed Assessors to each of the proposals for which they are Carriage 1. For other schemes, Executive Directors assign proposals to an appropriate number of Detailed Assessors.

FoR codes and expertise text on assessor profiles stored in RMS and the information provided in the proposal (FoR and Socio-economic Objective (SEO) codes, project summary and impact statement) are used to match Detailed Assessors with proposals. It is therefore important that Detailed Assessors update their RMS profile regularly, including the Fields of Research codes and expertise text, to ensure that proposals are assessed by the most relevant assessors. Detailed Assessors may decline an assignment where there is a significant mismatch of expertise. In that case the ARC will arrange a re-assignment where it is feasible to do so. 

[top]

Assessment

Selection criteria

The scheme selection criteria are provided in the Assessor Handbook – PDF format (1MB) – Word format (183KB) and in the scheme Funding Rules. Selection criteria and weightings vary from scheme to scheme depending on the nature and objectives of the scheme. Most scheme selection criteria include ‘investigator’ and ‘proposed program of research’ components. Assessors must assess proposals against the relevant scheme specific selection criteria. 

Assessment against the ‘investigator’ criterion must include consideration of the opportunities the researcher has had to build their research profile. For example, researchers who are early in their research career or have had an interrupted research career, including employment outside academia, unemployment, child birth, carers’ responsibilities and other personal circumstances, will have this taken into account. Similarly, researchers whose current or previous conditions of employment have allowed them a relatively large amount of research time  will have this considered when assessors are examining opportunity to build a research profile. This provides assessors with information to determine whether a CI has had extensive opportunity for research compared to a CI who may have had more limited time for research. In this way, the quality and benefit of achievements are given more weight than the quantity or rate of particular achievements enabling excellent researchers to be competitive regardless of their career path. Please see the ARC Research Opportunity and Performance Evidence (ROPE) Statement for more information.

Rating Scale

An A–E rating scale is used by both Detailed and General Assessors to rate proposals against the selection criteria. Please note that the wording and percentages used in the rating scale vary between schemes. The Assessor Handbook includes detailed descriptions of the criteria for each scoring band in the rating scale. Please consult the Assessor Handbook – PDF format (1MB) – Word format (183KB)  to ensure you use the correct rating scale for the scheme round under assessment. 

Detailed Assessments

Detailed Assessors submit online assessments to the ARC through the Research Management System (RMS). The merits of each proposal must be assessed against the selection criteria set out in the scheme Funding Rules.

The structure of the assessment form varies from scheme to scheme, but typically Detailed Assessors provide scores and written comments for each selection criterion against the rating scale provided in the Assessor Handbook – PDF format (1MB) – Word format (183KB)The comments given by the Detailed Assessors should correlate to the ratings given and also provide constructive details that justify the ratings and enable an appropriate rejoinder.

For most schemes, the ARC aims to have a minimum of two Detailed Assessors assigned to each proposal.

Rejoinder

In most ARC schemes, once Detailed Assessors submit their assessments to the ARC, applicants are invited to respond to the assessor comments via a rejoinder process. Scores and assessor details are not released to applicants.

The rejoinder process allows applicants to clarify any misunderstanding or difference of opinion about perceived weaknesses in the proposal. This process also provides an important feedback mechanism that can help researchers in future proposals. The Detailed Assessments (ratings and comments) and the rejoinders are provided to the General Assessors assigned to that proposal.

General Assessments

General Assessors use the rating scale in the Assessor Handbook – PDF format (1MB) – Word format (183KB) to assess a proposal’s merits against selection criteria set out in scheme Funding Rules.

General Assessors are asked to undertake a preliminary assessment and scoring of their Carriage proposals when they are first given access to their assignments. Once the rejoinder period has closed, General Assessors are given access to Detailed Assessor comments and ratings, and the applicant’s rejoinder. General Assessors consider these when providing proposal ratings. 

For most schemes, General Assessors are expected to discuss their assessment with the other General Assessor/s assigned to a given proposal before they finalise and submit ratings to the ARC. 

General Assessors submit online assessments to the ARC through the Research Management System (RMS).

Initial Ranking of proposals

The assessor assesses each selection criterion separately using the A–E scoring bands. A ‘Proposal Score’ which takes into account the criterion weightings for the specific scheme is automatically calculated when an assessor enters all the ratings and saves the proposal assessment.

Proposal Scores from Detailed and General Assessors are then combined to provide a single value (the overall Proposal score) for each proposal. This is performed by calculating the median of the average of the General ratings and the average of the Detailed ratings. For some schemes, the General Assessor’s raw ratings are normalised to redistribute all of the assessments provided by a General Assessor across the full spectrum of ratings.

The creation of an overall Proposal score allow proposals to be sorted into an initial ranked list to be used as the basis of selection panel discussions.

[top]

Selection Meetings

Selection Advisory Committee (SAC) meetings for ARC schemes are held throughout the year. The largest selection meeting is in August.

SAC members (General Assessors) attend selection meetings to identify and recommend the highest quality proposals and to recommend how much funding these proposals should receive. 

Selection of proposals to recommend for funding

The SAC considers all proposal information, assessor reports and rejoinder statements in determining the final list for recommendation.

Budget recommendations

SAC members consider each budget request in detail and make a recommendation as to the level of funding to be provided. Considerations include value for money, the justification for the requested budget items, and whether the items are allowable under the scheme Funding Rules. 

Approval

The SAC’s funding recommendations are reviewed by the ARC CEO and then the Minister. The Minister gives the final approval of the funding outcomes. 

[top]