Bowers CIS Research: Teaching Novices How to Leverage Machine Learning

By Louis DiPietro

As machine learning has entered fields and industries traditionally outside of computing, the need for research and effective, accessible tools to enable new users in leveraging artificial intelligence has never been more necessary. 

With the rise of new AI algorithms and increasing computing power, research into interactive machine learning – a subfield of artificial intelligence that explores machine-learning accessibility and education among new users – has proliferated recently, as scholars in human computer interaction and machine learning converge to improve tools for non-experts.

New, award-winning research from the Cornell Ann S. Bowers College of Computing and Information Science is helping inform and improve future design and development of interactive machine learning tools, bringing us closer to harnessing artificial intelligence in our individual lives.  

“We don’t know much about how non-experts in machine learning come to learn algorithmic tools,” said Swati Mishra, a doctoral student in the Department of Information Science and lead author of “Designing Interactive Transfer Learning Tools for ML Non-Experts,” which received a Best Paper Award at the annual ACM CHI Virtual Conference on Human Factors in Computing Systems held in May. “The reason is because there’s a hype that’s developed that suggests machine learning is for the ordained.”

Trained on a steady diet of data, machine-learning algorithms are used to find patterns in data that humans wouldn’t otherwise notice and are being deployed to help inform decisions big and small, from Covid vaccination development to Netflix recommendations. Bringing everyday professionals up to speed on how to effectively, efficiently and ethically use machine-learning algorithms will take training, and scholars of late have honed in on researching and developing better interactive tools and platforms for non-experts to harness the power of AI. 

Existing research into these interactive machine learning systems has mostly focused on understanding the user and the challenges they face when navigating the tools. Mishra’s latest research – including the development of her own interactive machine-learning platform – breaks fresh ground by investigating the inverse: how to better design the system so that users with limited algorithmic expertise but vast domain expertise can learn to integrate pre-existing models into their own work.

“When you do a task, you know what parts need manual fixing and what needs automation,” said Mishra, a 2021-2022 Bloomberg PhD Data Science fellow. “If we design machine-learning tools correctly and give enough agency to people to use them, we can ensure their knowledge gets integrated into the machine-learning model.” 

[T]here’s a hype that’s developed that suggests machine learning is for the ordained. We as researchers and designers have to mitigate user perceptions of what machine learning is. Any interactive tool must help us manage our expectations.

Mishra takes an unconventional approach with this research by turning to a complex process called “transfer learning” as a jumping-off point to initiate non-experts into machine learning. Transfer learning is a high-level and powerful machine-learning technique typically reserved for experts, wherein users repurpose and tweak existing machine-learning models for new tasks. 

Why begin here? To Mishra, domain experts – those who know their areas of expertise but may not have any experience with machine learning – bring extraordinary knowledge to the process of designing, developing, and evaluating machine-learning models. However, teaching domain experts how to build algorithms from scratch is a heavy lift and an unnecessary one. Built models already exist, the authors state. Entire online repositories called “model zoos” are filled with tested, domain-specific models for users to integrate.  

“By intentionally focusing on appropriating existing models into new tasks, Swati’s work helps novices not only use machine learning to solve complex tasks, but also take advantage of machine-learning experts’ continuing developments,” said Jeff Rzeszotarski, assistant professor in the Department of Information Science and the paper’s senior author. “While our eventual goal is to help novices become advanced machine-learning users, providing some training wheels through transfer learning can help novices immediately employ machine learning for their own tasks.”

Mishra’s award-winning research exposes transfer learning’s inner computational workings through an interactive platform so non-experts can better understand how machines crunch data sets and make decisions. Through a corresponding lab study with people with no background in machine learning development, Mishra was also able to pinpoint precisely where beginners were pulled off track, what their rationales were for making certain tweaks to the model, and what approaches were most successful or unsuccessful.

In the end, the duo found participating non-experts were able to successfully use transfer learning and alter existing models for their own purposes. However, researchers discovered that inaccurate perceptions of machine intelligence frequently slowed learning among non-experts. Machines don’t learn like we do, Mishra said.

“We’re used to a human-like learning style, and intuitively, we tend to employ strategies that are familiar to us. If the tools do not explicitly convey this difference, the machines may never really learn.” she said. “We as researchers and designers have to mitigate user perceptions of what machine learning is. Any interactive tool must help us manage our expectations.”