Why ethics should be integral part of Artificial Intelligence

March 16, 2021

Mykola Pechenizkiy develops data mining and machine learning tools to prevent undesirable algorithmic biases in AI.

Image: Shutterstock (metamorworks)
Image: Shutterstock (metamorworks)

Much of current AI is about finding statistical patterns in historical data to make valuable predictions for the future. However, there is increasing evidence that the use of algorithms, including racial profiling, can severely disadvantage vulnerable sections of society, even when there is no explicit intention of discrimination. We ask Mykola Pechenizkiy, professor Data Mining at Eindhoven University of Technology and one of the pioneers in ethical AI, about his efforts to make AI fair and responsible. “We should stop blaming the data and make AI trustworthy by design”.

On January 15th 2021, the Dutch government resigned, just two months ahead of the general elections. The reason was not, as you might expect, its handling of the Corona pandemic, but a scandal in which thousands of Dutch families, many of them from ethnic minorities, were wrongly accused of child welfare fraud.

The scandal is just one example of the increasing evidence that AI and machine learning have an impact on society and humans that is less beneficial. Other examples include facial-recognition software that discriminates against darker-skinned faces and the faces of women, image-generation algorithms that autocomplete cropped images of women with a low-cut top or bikini, or a state-of-the art language model (GPT-3) that has an explicit tendency to link muslims to violence.

“In other words: AI has a bias problem”, says Pechenizkiy. “Its predictions necessarily reflect the data they are based on, which, as we know, are often skewed in ways that reinforce existing inequalities in society. Or, as data analysts like to put it: garbage in, garbage out.”

Ethics takes center stage.

AI has an increasing impact on our everyday world - affecting millions of people not just in industry, but also in healthcare, education, government and many other areas. Take predictive analytics, for example, which can screen patients for potential corona infections based on data from routine blood tests, or predict hospital readmissions of heart failure patients using an attention-based neural network.

Therefore, bias in data is a big problem, which forces computer scientists to take a stand. “I strongly believe that ethics should be an integral part of AI. It can no longer be considered as a mere afterthought to an optimization problem, as traditionally has been the case,” says Pechenizkiy.

“These days, AI and machine learning are tightly connected to fairness and non-discrimination. If you build an algorithm that helps employers decide who should be invited for a job interview, it should not only be accurate, but also respect different aspects of diversity, and be legally and ethically compliant. Or if you design an AI system to diagnose skin cancer, it should work for both men and women, and for people of brighter as well as darker skins.”

Pioneers

Pechenizkiy and his colleagues at TU/e have been pioneering the modern field of ethical AI. “The first PhD thesis on fairness-aware machine learning was written here at TU/e by Faisal Kamiran, under the supervision of Toon Calders. And back in 2010, when few people could see the relevance of it, we wrote a number of ground-breaking papers on this topic, for instance on discrimination aware decision tree learning. We designed a new classification technique based on decision tree learning, which generates accurate predictions for future decision making, yet does not discriminate against vulnerable groups,” says Pechenizkiy.

“The technique takes into account discrimination both in the construction of the decision tree, and in the labeling of the leaves, without losing in accuracy. This works better than just ‘cleaning away’ the bias before applying traditional classification algorithms. Our model also works in cases of indirect discrimination. For example when the ethnicity of a person is strongly linked with the area they live in, leading to a classifier that discriminates based on postal code.”

Fairmatch

Much more recent work, from 2020, addresses bias in recommender systems. Recommenders are used by companies like Amazon and Spotify to predict user preferences and give them personalized recommendations.

Research has shown that these systems, while very effective, suffer from a number of problems, because of feedback loops that introduce and amplify stereotypical behavior. As a result, they tend to favour a limited number of very popular content, and they can be biased against certain groups, such as women.

PhD candidate Masoud Mansoury has designed a graph-based algorithm, Fairmatch, that significantly improves diversity in recommendations. It works as a post-processing tool on top of any existing standard recommendation algorithm. “The idea is to is to improve the visibility of high-quality items that have a low visibility in the original set of recommendations”, says Pechenizkiy.

“The beautiful thing is that Fairmatch can also be used in other contexts, for instance in recommenders that currently generate less accurate predictions for women or to fairly distribute high-paying jobs among men and women.”

 Making AI trustworthy by design

“This is truly an exciting time for computer scientists”, says Pechenizkiy. “We have to start thinking about how important properties like fairness and privacy can be formalized, how we can integrate them in research and development life cycles, and propagate them into machine learning pipelines. Sure, data is always biased in some ways, and messy in many ways. But by developing models that are explainable and certifiable, we can ensure that AI is trustworthy by design, reflecting both its added value and its vulnerabilities.”

“At the same time, it is important that computer scientists set up collaborations with researchers from other disciplines, including the ethical or legal domains”, says Pechenizkiy. He points to some interesting paradoxes that he and his colleagues have come across in their work.

“Take for instance the fact that you need to have access to sensitive information to be able to learn unbiased models. Or the situation where different notions of fairness conflict, or need to be balanced. While these problems can partly be solved by computer scientists, they often touch on issues that go beyond the exact sciences, and enter into the realms of philosophy and law.”

Finding a Common language

While there has been a lot of progress in this field over the last five years – also at TU/e, where computer scientists have been exploring collaborations with philosophers and ethicists -, Pechenizkiy still sees a big divide between the domains.

“Computer scientists have a tendency to oversimplify real-world problems, in order to be able to optimize them in their models, whereas social scientists often don’t have a clear view of what can be made operational. I do hope we can move further, once we are able to find a common vocabulary, but it won’t happen automatically. It requires commitment from all parties and also extra funding.”

At the same time, the Ukrainian-born researcher sees certain ethical challenges around AI that computer scientists cannot solve (and should not solve), even if they work together with ethicists. “Take for instance the use of facial or speech recognition to predict certain properties, like sex or sexual orientiation. That would plainly be unethical. Or trying to predict IQ based on facial features, which would not only be unethical but also unscientific!”

Plain science

In this context, it is interesting to see that a growing number of AI conferences (e.g. NeurIPS 2020) require researchers to provide an ethical impact statement, to make sure the research is responsible. “I think this is a good thing. It is common practice to perform an ethical review in the social sciences, so why not in computer science?”.

Still, Pechenizkiy thinks we shouldn’t overdo it. “As there is a hype in AI, we also run the risk of creating a hype in AI ethics. Much research in computer science is just that: plain science, with probably no ethical impact at all. Ethics is important, but remember: if you don’t have AI, you don’t have ethics in AI. Sometimes people tend to forget that”.

Mykola Pechenizkiy and trustworthy AI

 

Mykola Pechenizkiy is the chair of the Data Mining Group at TU/e. He is also the lead of the Data & AI cluster, which combines three research groups within the department of Mathematics and Computer Science involved in AI research, and member of the scientific board of EAISI, the AI institute of TU/e.

Pechenizkiy’s work in responsible AI is part of a much broader effort at TU/e to create Artificial Intelligence systems that are trustworthy, now that AI is penetrating more and more into our everyday lives. This implies that for AI to become widely adopted and used, people should be able to trust it.

All across TU/e, researchers are working hard on achieving this goal: they are ensuring that the data they exploit is privacy-friendly and readily available; that the algorithms and software models they design are fair, robust and accountable; and that the autonomous systems they develop are safe, reliable and resilient.

A leading principle in all research on trustworthy AI at TU/e is interdepartmental and interdisciplinary collaboration, as evidenced by the Certifiable, Robust, and Explainable AI research line within EAISI.

For a full overview of all research groups at TU/e working on trustworthy AI, check out this list. You can find the latest news on our AI research here.

Media contact

Henk van Appeven
(Communications Adviser)

Latest news

Keep following us