Launched September 2020, Completed June 2021
From fall 2020 through spring 2021, the LC Labs team collaborated with data management solutions provider AVP to explore approaches with the Humans in the Loop: Accelerating access and discovery for digital collections initiative.
The Humans in the Loop initiative builds on the foundation of the Library’s success with crowdsourcing—including establishing Flickr Commons, LC Labs initiatives like Beyond Words, and the Library’s By the People program—to explore deepened engagement with collections, while foregrounding the role human expertise plays in machine learning. Furthermore, the experiment builds on the expertise LC Labs has built in machine learning by convening expert summits, sponsoring state of the field research, supporting Innovators in Residence, and investigating models and resources for computational research in the cloud.
The HITL initiative resulted in a human-in-the-loop framework co-developed by AVP for designing approaches to combining machine learning and crowdsourcing. AVP has over a decade of experience working with the Library of Congress on projects focused on software, digital preservation, and best practices. The HITL initiative outcomes focus on the opportunities and challenges of creating future access to the U.S. Telephone Directory collection as well as prototyping human-in-the-loop workflows, proof of concepts for training data, associated code, and experimental interfaces for engaging users with collections and crowdsourced data. The final recommendations circle on how to combine crowdsourcing and machine learning with an emphasis on the requirements for designing and supporting human-in-the-loop approaches that are ethical, useful, and engaging.
The major takeaways from the Humans in the Loop initiative are:
- Human-in-the-loop initiatives have the potential to be extremely powerful for maximizing access to LC’s content at scale.
- Human-in-the-loop projects will require significant investment in staffing and resources.
- There are ways to generalize human-in-the-loop approaches, however, there will not be a one-size-fits-all approach.
In the recommendations, AVP suggests the following areas for further exploration:
- Ongoing user testing and iterative development of crowdsourcing and end-user platforms will significantly improve overall user engagement with and access to LC collections and content.
- Investigation of other methods of sharing human-in-the-loop data will benefit collection end users.
- Broader representation across staff will better surface and address potential risks and biases.
Read more about the research from these blog posts, project code and documentation: