“How can Mathematics help us to fool Artificial Intelligence?”, that was the question we intended to answer at the Taugenichts Bar on the 2nd of May, for the 10th edition of “Wissen vom Fass” (or, “Knowledge from the Barrel”). This event is organized annually by DESY together with the University of Hamburg. The goal is to provide insights into ongoing research in the comfortable atmosphere of Hamburg’s bars and pubs. Anyone interested is invited to sip at their favorite drink while attending an entertaining scientific talk, followed by an open discussion.
 
Choosing an interesting topic for this occasion was easy. The reliability and vulnerability of artificially intelligent systems are of great interest to society. Self-driving cars seem practical, but only if they don’t mistake pedestrians for a green light. Large language models can be a great source of inspiration, but ideally, we would like to avoid that they plagiarize to create their output.
Having decided on a topic, we had to solve a way more complicated problem: How can we explain the complex mathematical foundations of so-called adversarial attacks without boring our audience to tears?
 
The answer we came up with eventually? Existential acting, mathematically interpreted by our group leader, Martin Burger! To visualize our play, we grabbed some old cardboard boxes and hand-crafted a “neural network” inspired by shape sorters for children. This neural network (convincingly impersonated by Tim Roith), had the task of identifying triangles among different kinds of shapes. During the training phase, which was greatly supported by the audience, Tim was allowed to adapt the network parameters with the help of the so-called gradient (i.e., scissors, tape, and more cardboard). Once the network performed well on the provided training data, the attacker (Samira Kabri) took the stage. Although she was not allowed to change the network parameters themselves, she could fool the network by slightly perturbing a test shape. A successful adversarial attack!
 
After this artistic introduction to the topic, the audience got the chance to consolidate their newly acquired knowledge and to perform an adversarial attack themselves. Using a pre-computed adversarial patch, they became invisible! (At least, through the eyes of a person detection network.)
 
Our presentation was followed by lots of questions and interesting discussions, ranging from rather philosophical topics to plain mathematics. We enjoyed our time at “Wissen vom Fass” a lot and are already looking forward to participating again next year!