Managing the unmanageable – strategies for thriving in a world where the research data explosion threatens to overwhelm us with complexity, scale and risk
Benjamin Wu1 1Netapp, North Sydney, NSW, Australia
Abstract
The rapid changes in our local and global environments require us to do more research. This results in a data explosion, which can lead to unsustainable data management challenges at scale. As the diversity of formats, volume and richness of research data increases dramatically, along with longer retention periods, so the risk also increases of data becoming too costly to sustain or falling into the wrong hands and being mis-used.
Three major challenges exist in managing research data: complexity, scale and risk. They show no signs of abating; in fact, the issues are accelerating due to conflicting needs: cross-domain and cross-border sharing and security.
Complexity creates issues for versioning, indexing, finding and recalling data in a usable and timely manner. Historical research with great value also can be overlooked.
Scale creates issues for logistics; no researcher would waste their time managing data rather than spending it on research activities.
Data risk is potentially the biggest challenge. Risk is subjective and diverse stakeholders will have different views on the associated values, attributes and budgets.
Most organisations are data survivors, unprepared to manage the impact of fundamental changes. We set out to develop innovative ideas to help researchers and eResearch organisations successfully address these challenges.
We researched relevant papers, gathered experiences from relevant stakeholders and combined our technology knowledge. We then correlated this information and formulated functional models which have been validated at a peer level.
Our presentation describes practical, coherent and sustainable ways for eResearch organisations to become research data thrivers.
Biography
Ben Wu is an Executive Architect in information systems and technology with over 30 years of experience working with global and commercial, government and non-profit organisations. Ben specialises in digital strategy, and educates clients, partners and communities on changing the world with data. His work at the forefront of data fabrics includes sharing expertise to maximise efficiency, security and agility across the eResearch landscape. Ben is an influential advisor in the data industry and his positive attitude and tireless energy encourages others to work smartly with integrity to achieve genuine success. He is inspired daily by his faith and family.