“The general understanding of artificial intelligence is strongly influenced by Hollywood”
Dr. Wolfgang Hildesheim (IBM) on the advance of AI in art and culture

Oliver Koerner von Gustorf: Artificial intelligence is already known from all kinds of areas, including business, science, and the service sector. Why do we need AI in art education? What are the advantages of MIA and how does it differ from conventional guided tours?

Dr. Wolfgang Hildesheim: Artificial intelligence is a worldwide megatrend that will fundamentally change the way we live and work, in all areas of daily life, in all industries, in the public sector, as well as in products and services. Everywhere. And, of course, museums and museum visitors will be affected by it. MIA belongs to the application class of smart assistants, which are very popular at the moment because the new available skills can be put to very good use. It’s a certain type of software that analyzes spoken words, puts them into the right context, and reacts to them. MIA, the PalaisPopulaire application, is a so-called smart assistant for a museum.

OK: What exactly is MIA’s job?

WH: This happens every day. You go to a museum, view a picture, and say to yourself: “Wow, I like that, it’s great. But what exactly is it? Did I really understand it? How can it be interpreted? What is the object down there in the lower left-hand corner? Everyone has experienced this in a museum; you want answers to specific questions. The classic audio guides used in museums today don’t really do this. Many questions that museum visitors have about works aren’t answered. This is unsatisfactory. And this is where we want to take the next logical step with the museum organizers. The idea is that you can talk to artworks to a certain extent and ask any question you like about a given work. If a smart assistant, like MIA, is well made, it can answer essential questions about the artworks.

OK: Smart assistants answering questions is already part of everyday life. For example, when you go to an insurance company’s website and a smart assistant gives you information about services and rates. This works very well when it comes to very tangible facts and figures. MIA provides assistance in a completely different situation. Of course many people ask: “What is that in the picture?” But what happens when this is not clear or it is abstract? As you just said, it is also about interpretation and very subjective impressions, and perhaps also emotions or personal memories. What challenge did that pose for you?

WH: That’s a very good question, because there really are differences. A chatbot for supplementary dental insurance answers questions about additional dental insurance, about how to get it, and what it covers. This is very goal oriented. Or, for example, when you’re dealing with a car insurance claim settlement. The aim is to find out whether the dent is large or small, whether the repair will cost a lot or a little. The smart assistants in this area are very goal oriented. And the user knows this goal too. He or she wants to obtain certain, relatively narrowly defined information. The situation is different with a picture in a museum. It has a much broader and more open context. It can be philosophical, aesthetic, social, or related to the artist’s biography. Naturally, it’s a great challenge to be able to cover all that. Nevertheless, one thing is the same: the viewers of the picture, the users of the smart assistant MIA, want to learn something about the work.

OK: But unlike with a fender bender, where the questions are predictable, MIA first needs to be asked something in order to give an answer.

WH: That’s right. And that’s why at the beginning we collected questions from many people who go to museums, questions about what interests them about certain works, to which MIA now answers live in the exhibition. If hundreds, thousands, or tens of thousands of questions are asked, the questions repeat themselves increasingly over time. And then you can—and this is the typical procedure with this kind of AI project—first answer and automate the frequent and simple questions, and then address the more complicated and rare ones with texts or sections of text that the artist herself has written. In this combination, together with speech and word processing, a smart assistant can be built and improved, so that over time more and more questions can be answered.

OK: A normal guided tour, when you walk through an exhibition with a museum educator, is a dialog. A human guide does not explain the picture frontally or only answer questions, but engages the visitor in a conversation. Is it conceivable that MIA could become more dialog-oriented in the future?

WH: The answer is yes. The development of the technology we are working on at IBM is moving in that direction. Our new IBM Project Debater demonstrates this. Have you heard of it?

OK: No.

WH: IBM regularly defines so-called "grand challenges," major tasks that initially would be considered almost impossible to solve. In these tasks, technologies developed by IBM in our research laboratories compete against human intelligence, against experts in a particular discipline. These challenges always involve attempting something that previously seemed impossible. In 1997, for example, we took part in a grand challenge with Big Blue, a chess computer we developed that beat the world chess champion Garry Kasparov. The next grand challenge, in 2011, was IBM Watson’s appearance on the popular American game show Jeopardy! Here, too, it was not clear whether such complicated questions could actually be answered with software. But Watson won against two of Jeopardy’s greatest champions.

OK: But that is also fact based. Quite logical trains of thought and knowledge is tested.

WH: Yes, but that was only the beginning. That was in 2012, eight years ago. And the next step is exactly what you’re talking about, that the technology is moving in the direction of dialog. You’re right. MIA is currently fact based and not dialog based. It simply spits out facts or prepared questions. There is no real dialog with visitors yet. This is the next step, and it is being demonstrated now in the third grand challenge. In 2019, the IBM Project Debater competed against world champion debater Harish Natarajan in a public debate in San Francisco. Although Natarajan won in the audience’s opinion, the Debater’s performance and the arguments it put forward were impressive.

OK: What benefits does this debating program have in day-to-day life?

WH: It's a text-based decision-making aid. It collects arguments and helps with potentially polarizing issues in collective decision-making, whether in communities or in companies.

OK: Can AI find connections that humans cannot?

WH: In technology and research, definitely yes. One of the main artificial intelligence applications is pattern recognition. And finding ways to recognize patterns is a core area of human research. Take, for example, the analysis of the human genome, consisting of millions of amino acids chained together in sequence. Deep learning algorithms, that is artificial intelligence, are used for this purpose, for example to find a link between certain diseases and their cure with certain drugs. Deep learning algorithms are surely now being used in various fields of research in the search for drugs to combat the coronavirus.

OK: But what about the debates on the virus?

WH: When thousands of people are giving their opinions about the coronavirus on social media, then such techniques can be used to find out how many people have certain feelings, whether they’re afraid, what they’re afraid of, and whether they know what they can do. You can get an overview much faster and possibly also find doctors’ opinions that you have overlooked so far. This is AI’s special ability: to create summaries from large amounts of text, to classify sentences in millions of texts based on search terms. And so you can find new facts and other opinions.

OK: This is really fascinating. On the other hand, many people are afraid that artificial intelligence will destroy jobs, that people will soon only talk to chatbots on the phone, that the human aspect will be lost. What would you say to people who are afraid of being overwhelmed by artificial intelligence?

WH: First of all, it will be a very long time before machines can conduct psychological, emotional, and highly complex dialogs. Technology is infinitely far from being able to do that, it’s far too stupid. The general understanding of artificial intelligence is strongly influenced by Hollywood, by films like Terminator and Independence Day, or by HAL in 2001: A Space Odyssey, where there are always certain computers, robots, or cyborgs that are infinitely more intelligent than humans. This is called “super intelligence.” Great movies can be made about this in Hollywood and people can be inspired. In reality, however, our technology today is light years away from being so intelligent. What we currently have, so-called artificial neural networks, are “narrow artificial intelligence,” that is a very limited form of artificial intelligence that can only perform small, narrowly defined tasks. An example is image recognition software that can recognize a cat. This is incredibly little compared to what humans can do. In this respect, the fears fueled by Hollywood are exaggerated, and many people worry unjustifiably because the technology itself is much stupider.

OK: But then why is AI so important?

WH: The opportunities provided by these small, stupid steps are underestimated. More can be done with them than people think. Especially for Germany as a highly developed country, this is extremely important. Which brings me to the answer to your question. You just suggested that AI can lead to job losses. In fact, the opposite is true. Jobs will be lost if we don't use AI and miss the AI trend. This has already been shown by the Internet trend, which was long neglected here in Germany. If German companies now help shape the next trend, AI, the intelligent layer on the Internet, and determine the competition, then I am optimistic that there will be no negative effects on the labor market. We want to support people with our technologies, not replace them. People and intelligent systems can coexist.