Harnessing The Wisdom of Crowds: The SWARM Project

Prof. Richard O. Sinnott1 & the SWARM Team1

1University of Melbourne, rsinnott@unimelb.edu.au


The Smartly-assembled,  Wiki-style Argument Marshalling (SWARM) project was funded by the   US   Intelligence   Advanced   Research   Projects   Activity   (IARPA)   as   part   of   the Crowdsourcing Evidence, Argumentation, Thinking and Evaluation (CREATE) program. The project  formally  commenced  in January  2018 and has been awarded  (up to $19m).  The SWARM is one of 4-projects funded globally through the IARPA CREATE program. These projects are tasked with suporting improved reasoning to aid the intelligence community by leveraging the wisdom of crowds. Whilst previous IARPA programs have demonstrated the benefits in leveraging the wisdom of crowds to get improved answers, the actual reasoning and deliberation in what makes a good answer remains unclear. This is the primary goal of SWARM.

The evaluation of the SWARM platform and the other platfoms is currently being undertaken by an independent crowd managed by IARPA and their Test & Evaluation team. This crowd will be organised into separate teams. Each team will be assigned a set of questions that require reasoning and evaluation to come up with the most highly regarded answers over several months, i.e. those answers (hypotheses) with the best reasoning and presentation.

This presentation will cover the overarching goals of SWARM and the underpinning technical solutions that have been developed. This includes the mobile applications that have been developed  to encourage  crowd  participation.  The talk will also briefly  include  early (non- funded SWARM work) exploring the extent that deep learning approaches can be used for automation of the assessment of collective reasoning.


Professor Richard O. Sinnott is the Director of eResearch at the University of Melbourne and Chair of Applied Computing Systems. In these roles he is responsible for all aspects of eResearch (research-oriented IT development) at the University. He has been lead software engineer/architect  on  an  extensive  portfolio  of  national  and  international  projects,  with specific focus on those research domains requiring finer-grained access control (security). He has over 300 peer reviewed publications across a range of applied computing research areas.