Automated decision-making challenges trust in authorities

Welfare services are becoming increasingly digital. Researcher Anne Kaun is looking at what happens from a citizen perspective when algorithms start to control decision making. A development that could undermine people’s trust in authorities and the perception of democracy.

Anne Kaun

Professor of Media and Communication Studies

Wallenberg Academy Fellow 2021

Institution:
Södertörn University

Research fields:
Media theory, mediated temporalities, algorithmic culture, automation and artificial intelligence from humanistic social sciences perspectives.

Critical data research has long drawn attention to the fact that our lives are increasingly “datafied”. The vast amount of data produced is financially significant to businesses that use it to map our purchasing behavior and interests. In certain contexts, big data is described as the new oil. 

Digitalization is not only underway in the business world, however, but in the welfare sector as well. Authorities have started to reorganize their systems and use algorithms to control and process information, says Professor Anne Kaun.

“From the outset, digitalization emerged as a way for authorities to simplify administrative routines. In step with technological development, the digitalization of the welfare sector has expanded into new areas that include artificial intelligence and automated decision making.”

A new role for social services

Since 2017, social services at the Swedish municipality of Trelleborg have used automated decision making to process benefit applications. But the fact that responsibilities of social services employees have been usurped by what are sometimes referred to as “smart computers” is a point of contention. What has become known as the Trelleborg model has been met with skepticism from several quarters. Automated decision support is a challenge to conventional practice and partly replaces personal contact.

Another example is when municipalities use machine learning to process reports of concern made by members of the public and analyze language used in such reports to assess how quickly they should be processed.

“What’s new is that these automated decisions are taken at the very heart of the welfare system,” says Kaun.

The pace of digitalization is so rapid, it is important we have research that sheds light on these changes.

“We see how digitalization could result in new opinions about how welfare should be organized and distributed in society. The technology also provides entirely new ways to collect information about citizens.”

Smart cities

Investment is also being made in smart cities in various ways, for example sensors that measure environmental data or information about traffic flow, or sensors that are given to municipal housing providers to collect data with the aim of improving security. 

“These kinds of projects are often conducted as pilot studies and there’s a lot of interest, especially from commercial businesses that see significant opportunities in the public sector,” says Kaun.

The computerization and thus digitalization of authorities goes back to the 1970s, such as in the Swedish Social Insurance Agency, the Swedish Prison and Probation Service and the Swedish Tax Agency. It is still the larger national authorities that have the resources to implement new tech solutions.

“Some even have their own AI departments that develop in-house pilot projects. At the municipal level, collaborations are more often with smaller start-up companies, and we try to map how that market looks there as well.”

One goal is to develop guidelines for decision makers, so that they have a better understanding of which areas digital solutions can make the work of authorities more efficient and in which areas they risk damaging trust in democracy.

Monitoring trust

Kaun entered this field of research through her work on social media. The public tends to think of social platforms as places where algorithms are used to control information flows, which can create fertile ground for criticism and distrust. What happens when authorities start using similar tools?

“Trust in authorities in Sweden is strong compared to other countries. The question is whether that relationship will change when contact with authorities becomes more digital, when we no longer meet case managers and instead interact with chatbots.”

Kaun captures citizens’ attitudes towards digitalization in surveys, in-depth interviews, and with the help of creative methods such as experience-based workshops. The digitalization of some areas is seen as being more problematic than others.

“For example, when it comes to police work and surveillance, citizens are more critical.”

Technological development has already come a long way and there are a large number of questions that remain unanswered, Kaun says.

“Algorithms are complicated systems, and we often don’t know which third-party actors are involved in them. What actually happens when we interact with a chatbot, what data is generated and is it shared with commercial actors or not? And what does the interaction with artificial intelligence mean for my perception of authorities?”

Kaun’s research will include a survey of actors present in schools’ digital platforms, healthcare, and social services. She also wants to expand on certain themes that she has worked on previously, for example how algorithms control decision making in welfare institutions in a comparative study between Estonia, Germany, and Sweden.

“One goal is to develop guidelines for decision makers, so that they have a better understanding of which areas digital solutions can make the work of authorities more efficient and in which areas they risk damaging trust in democracy.”

The Wallenberg Academy Fellow grant is also being used to build a research hub on digital welfare research. Kaun hopes to bring together researchers who work on issues related to digital welfare from Sweden and abroad.

“We hope to contribute to a citizens’ perspective and examine the consequences of the digital welfare state in terms of citizens’ own expectations and needs.”

Text Nils Johan Tjärnlund
Translation Nick Chipperfield
Photo Magnus Bergström