Michelle Barker1, Ingrid Mason2, Peter Elford2
1Nectar, Parkville, Australia
This BoF adds to the expanding discussion on how to demonstrate the value of research infrastructures. A wide range of approaches to demonstrating the outcomes and/or impact of research infrastructures already exist, alongside exemplars of different methods focused on a wide range of national and international infrastructures, including virtual infrastructures. This BoF will introduce some of the major types of impact categories – scientific, economic, workforce and social/cultural – and some examples of their use. Participants will then engage in small-group discussion on the most appropriate metrics for the type of value they are trying to evidence, taking into consideration the stakeholders to whom this is being targeted. Participants will have the opportunity to reflect on their current approach to metrics to demonstrate the value of physical and/or virtual facilities they are involved with, and consider alternatives. The BoF will end with a group feedback session to enable participants to widen their perspectives by hearing how others are approaching similar issues.
This BoF is targeted at eResearch professionals involved in developing and managing research infrastructure, on behalf of specific research communities or providing services for all research domains. Participants may include: research managers and infrastructure managers, eResearch directors, academic librarians, eResearch analysts, communication and service managers.
60 mins duration.
Research infrastructures continue to play a key role in enabling increased research impacts. There is an increasing demand for methodologies and tools for assessing the impact of research infrastructures. The role of research infrastructures in facilitating different parts of the research lifecycle can cause challenges for detailing their importance in impact pathways, resulting in increasing national and international conversations on how to best demonstrate their value.
A range of quantitative and qualitative methodologies currently exist to demonstrating the value of research infrastructures in terms of outcomes and/or impacts, including:
- Return on investment
- Cost-benefit ratio
- Contingent valuation
- Policy change
- On-ground intervention and actions change
- Amplification and improvement in the creation, delivery and competitive advantage of new products and services and enterprises by others
- Efficiency gains: productivity and efficiency gains in information management and delivery of services
- Cultural change in the behaviours relating to open sharing of data and integration and interoperability between platforms and services and data
- Extent of help in the application of knowledge and activities in key sectors including through the creation of “information derivatives”
- Instrument and use value/Uptake and usage of services/accessibility
- Platform and facility attribution tracking
- Production of reports, papers and publications
- Data sharing, data re-use
- Software sharing, software re-use
- Software Citation and Attribution
- Data Citation and Attribution
- Facilitation of collaboration
- Skills development
Australian exemplars include an evaluation of AuScope impacts , Atlas of Living Australia  and CSIRO . International exemplars include studies on the European Bioinformatics Institute and CERN. Examples of methodological approaches include tools from CSIRO , the Council of Rural Research and Development Corporations  and Beagrie Cost Benefit Toolkit .
This BoF will frame discussion around four major types of impact categories: scientific, economic, workforce and social/cultural. After discussion of these approaches and their use, participants will have the opportunity to reflect on their own approaches to evidencing value and discuss this with their peers, to encourage critical engagement on effective demonstration of value. These methodologies can be applied to both physical and virtual infrastructures.
- Lateral Economics. AuScope Infrastructure Program – evaluation of impacts. August 2016. http://www.auscope.org.au/wp-content/uploads/2017/02/Lateral-Economics-report.pdf, accessed on 3 June, 2017.
- Alluvium Consulting. Assessment of the Atlas of Living Australia’s Impact and Value. 2016. https://docs.google.com/document/d/1QQLG-wivYDwDMNXIMsKpRIydnwcqPJe4LDCBWloB5Hk/edit, accessed on 12 June, 2017.
- ACIL Allen Consulting. CSIRO’s Impact and Value. December 2014. http://www.acilallen.com.au/cms_files/ACILAllen _CSIROAssessment_2014.pdf, accessed on 12 June, 2017.
- Beagrie, Neil, and Houghon, John. 2016. The Value and Impact of the European Bioinformatics Institute. 2016. https://beagrie.com/static/resource/EBI-impact-report.pdf, accessed on 10 June, 2017.
- Organisation for Economic Co-operation and Development. The Impacts of Large Research Infrastructures on Economic Innovation and on Society: Case Studies at CERN. 2014. https://www.oecd.org/sti/sci-tech/CERN-case-studies.pdf,, accessed on 5 March, 2017.
- Evaluating Our Impact. http://www.csiro.au/en/About/Our-impact/Evaluating-our-impact, accessed on 7 April 2017.
- Council of Rural Research and Development Corporations. Impact Assessment Guidelines. 2014. http://www.ruralrdc.com.au/wp-content/uploads/2016/04/CRRDC-Impact-Assessment-Guidelines-V.1-070514.pdf, accessed on 12 June, 2017.
- Consortium of European Social Science Data Archives. Cost-Benefit Advocacy Toolkit. https://cessda.net/CESSDA-Services/Projects/Current-projects/CESSDA-SaW/Work-Packages/WP4/Cost-Benefit-Advocacy-Toolkit, accessed on 12 June, 2017.
Michelle Barker is Deputy Director (Research Software Infrastructure) at National eResearch Collaborative Tools and Resources (Nectar), a National Collaborative Research Infrastructure Strategy (NCRIS) funded program. She is one of the convenors of the annual International Workshop on Science Gateways – Australia, and the International Coalition on Science Gateways. She also facilitates national conversations around common challenges such as research impact metrics, software sustainability and reproducibility.
Ingrid Mason, Deployment Strategist with AARNet, provides support for engagement and the uptake of national research and education network (NREN) and services with AARNet members across the research, cultural and collections sectors. Ingrid has worked on several NCRIS programs: Australian National Data Service, National eResearch Collaborative Tools and Resources, and Research Data Services.
As the Director, Government Relations and eResearch, Peter Elford manages AARNet’s relationships across a broad range of Federal and state government agencies, and AARNet’s engagement with the Australian (e)Research community. He is a strong and passionate advocate for the role Information and Communications Technology (ICT) plays in enabling globally collaborative and competitive research through ultra-high speed broadband connectivity. Peter is an ICT professional with over 30 years’ experience within the government, education, research and industry sectors having worked at the Australian National University, AARNet (twice) and Cisco. In his first stint at AARNet (in 1990) he engineered much of the original Internet in Australia.