Teaching machines to learn how to learn

Joaquin Vanschoren is assistant professor Machine Learning at the department of Mathematics and Computer Science, where he is part of the Data Mining group. He is also involved in the work of the Eindhoven AI Systems Institute (EAISI). Artificial Intelligence plays a crucial role in Joaquin’s research. A short introduction in five questions and answers.

What is your key research question?

My team and I aim to build AI systems that learn how to learn. Today, many AI learning systems are built from scratch, by hand. That takes a lot of expertise, a lot of data, and a lot of time. Humans operate very differently. We don't read all of Wikipedia every morning, and we don't need 10,000 tries before we learn a new skill. From newborn to adult, we experiment and learn how to do simple tasks first, and then leverage what we have learned to learn new, more complex tasks very efficiently. Think about walking, reading, and basically any task that we do. Likewise, we want AI systems to never stop learning, experiment across lots of different tasks, learn what works, and leverage that to learn new tasks more effectively.
 

What is the key challenge in your work?

There is an infinite number of ways to build a machine learning system (or 'model’). That’s why you need so much expertise to build one that works well. We are building systems that automatically acquire this expertise by doing lots of clever experiments on lots of different tasks, and by learning from humans. To facilitate this, we created an open science platform called OpenML.org (for Open Machine Learning). There, anybody in the world can easily share their machine learning experiments, and reuse the best models shared by others.

It also serves as a global memory of machine learning experiments, and we can train algorithms on top of it to learn what works well. This is called meta-learning. Anyone can then use these algorithms to predict what will likely work well on new problems. Still, there are hard challenges both in terms of basic research and engineering. We need to efficiently run, represent and store large numbers of machine learning experiments while focusing on the most promising ones.
 

What are the practical applications of your research? How does it benefit society?

We want to replace the current manual, error-prone, trial-and-error approach to building machine learning models with a more systematic, robust, and data-driven approach based on prior results. This is called automatic machine learning (AutoML). To make an analogy with self-driving cars, we aim to automate all the tedious and error-prone aspects to minimize human error and to let people focus on the more important aspects of their applications.

This democratizes machine learning, since many more people (with fewer resources) can leverage machine learning in innovative new ways. Also, when a new task is similar to previously seen tasks, we can learn those tasks with much less data, which enables practical applications where only very little data is available. For instance, we have worked with bio-informaticians to learn how to build better models for drug discovery, especially to fight rare diseases for which less data is available.
 

How do your see the development of AI in the future?

If you compare AI systems with humans, we still have a long way to go in terms of systematically analyzing problems, recognizing similarities with previous problems, and reusing previously learned sub-solutions (e.g. learned features and neural architectures) in a modular way. Also, AI systems should be able to explain how they solved the task and what was actually learned. Another challenge is to let AI systems collect and select their own input data to avoid biases that may have detrimental outcomes. Just like humans, AI systems should be critical about their input data and look for additional input where possible. In general, we need to think a lot more about the relationship between human and AI, and take a human-centric approach that makes sure that AI developments empower humans rather than replace them.
 

Why should any AI researcher want to work at the TU/e?

­­

The TU/e offers a very interesting interplay between basic research and practical validation, especially through collaborations with high tech companies in the region and beyond. It also provides a quiet and supportive environment to focus on research, and the students are self-driven and pleasant to work with. The founding of EAISI will enable us to take our AI research to the next level by providing the necessary environment and resources (e.g. compute) to tackle more ambitious problems in AI and using them in society.