ARC response to the Evaluation of the Discovery Projects Scheme
ARC response to the Evaluation of the Discovery Projects Scheme – PDF Format (205KB) – Word Format (40KB)
The Discovery Projects scheme evaluation was completed in November 2020 by the ARC Program Evaluation Section, drawing on analysis of stakeholder feedback and ARC data. The evaluation found that, overall, the Discovery Projects scheme is effective and appropriate in achieving its objectives and intended outcomes, and is regarded as a unique and important component of the Australian research funding landscape.
The evaluation identified opportunities to address stakeholder feedback and support improvements in the scheme’s performance:
- Targeted advice, engagement and outreach
- Improved facilitation of national and international collaboration
- Improved Discovery Projects data collection
- Potential options to provide aggregated feedback on Selection Advisory Committee assessments
- Development of a Discovery Projects program logic
- Ongoing monitoring and evaluation
To address all of these opportunities, the ARC will seek to make the following improvements:
Improvement 1. Ongoing engagement with stakeholders to communicate the intent of the scheme
The ARC will continue to strengthen its communication with stakeholders, including exploring opportunities for targeted engagement and outreach to provide advice to the sector. The ARC is developing targeted social media content and ARC website material, including research highlights and articles through our ARChway newsletter, to improve stakeholder understanding of the Discovery Projects scheme and the National Competitive Grants Program (NCGP) more broadly.
The ARC is improving the way we communicate insights and data analysis, while upholding high standards of privacy and security of data. Currently ARC data is available in a number of ways, for example:
- the recently launched data visualisations is an exciting new interactive way of viewing and exploring NCGP data through trend visualisations powered by Microsoft Power BI;
- Grants Search is designed to make it easier to find details of ARC-funded research projects and collate data about the research projects funded by the ARC since 2001 for statistical analysis;
- ARC research data for projects funded since 2002 is published annually on the Grants Dataset | Australian Research Council webpage in Microsoft Excel format; and
- when funding from a grant opportunity is announced, the ARC publishes Grant Outcomes, which include both a Funding Announcements web page and detailed Selection Reports with outcome information like discipline, career age and gender, as well as international collaboration statistics.
Improvement 2. Streamlining to facilitate research collaboration
The ARC is committed to continuous improvement and streamlining of grant application and administration processes while balancing assessment and legislative requirements. The ARC will consider appropriate streamlining activities to improve facilitation of national and international collaboration supported by the Discovery Projects scheme.
It is noted that in 2018-19, the ARC engaged with representatives of the Australasian Research Management Society (ARMS) in implementing streamlining activities across the application and post award processes of the ARC’s National Competitive Grants Program (NCGP). Several improvements to the application process were implemented in 2019, including allowing non-research Partner Investigators to provide a two-page CV rather than completing the entire Research Opportunity and Performance Evidence (ROPE) section of Discovery Projects application forms.
Informed by ongoing consultation with ARMS representatives and feedback from Research Office staff, researchers and peer reviewers in the ARC’s 2020 streamlining survey, the application form, in particular the Research Opportunity and Performance Evidence section, is being investigated to identify and implement further streamlining of questions and enhancements to functionality.
Improvement 3. Improving mechanisms to monitor and evaluate benefits of the scheme, including research collaboration
The ARC will undertake work to improve consistency in the collection of data through applications, final reports, and variation requests to more comprehensively understand the benefits of the scheme.
The ARC will develop a program logic to support enhanced implementation, monitoring and evaluation of the scheme. The program logic will clearly outline the scheme’s policy intent, and how its implementation will continue to support the achievement of objectives and intended outcomes. The program logic will also outline ongoing mechanisms to monitor, assess and report on the scheme’s performance, including by informing benchmarks and supporting improvement in the consistency of data collection.
The ARC will also develop a set of metrics to specifically monitor the scale of research collaboration supported through Discovery Projects. This will allow the ARC to understand the benefits of collaboration more comprehensively. These metrics may be reported on publicly, for example through new trend visualisations, to bolster the overall effectiveness of Discovery Projects.
Improvement 4. Improving feedback to applicants
An internal ARC working group has been formed to explore the possibility of providing general feedback to applicants following Selection Advisory Committee meetings summarising overall issues or considerations emerging in a Discovery Projects round. The key challenge identified is how to provide useful feedback to applicants, with consideration to managing the workload of Selection Advisory Committee members and the ARC.