Hybrid International Conference, 1 & 2 December 2022
Badisches Landesmuseum, Karlsruhe, Germany
Please visit our project website Creative User Empowerment
Thursday, 1 December 2022
Schloss Karlsruhe, Gartensaal
Badisches Landesmuseum, Gartensaal
Session 1: Reflect
Session 2: Empower
End of Day 1
Get Together in museum x
|Poster & Demo Space|
Art Intervention: Frank Bierlein, Moph Zielke, Karlsruhe: Artificial Artist Assistant by intelligentartsolutions.com
Friday, 2 December 2022
Schloss Karlsruhe, Gartensaal
Summary & Perspectives
End of Day 2
The Role of Culture in the Intelligence of AI
— Following the French anthropologist André Leroi-Gaughan, we can understand Artificial Intelligence systems as tools that provide us with new forms of cultural intelligence through novel approaches towards images and language, which scholars describe as a “calculation of meaning”. This talk explores the different cultural dimensions of this intelligence through a critical but curious approach. Given the fact that the adoption of Artificial Intelligence systems is already advanced, it asks: What new and different perspectives do we gain through a calculation of meaning? What has become possible? And how do our societies frame these new possibilities? Who dictates in which direction AI systems are being developed and what can be the role of cultural institutions in that development? By asking these questions, the talk will explore the role of culture in the intelligence of AI from different perspectives.
Dr. Mercedes Bunz is a Professor in Digital Culture and Society at the Department of Digital Humanities, King’s College London. She studied Philosophy, Art History, and Media Studies at the FU Berlin and the Bauhaus University Weimar. Co-lead of the Creative AI Lab. Her last publications are How Not to Be Governed Like That by Our Digital Technologies, in The Ends of Critique: Methods, Institutions, Politics, edited by K. Thiele, B. Kaiser, T. O’Leary (Rowman & Littlefield) and with Claudia Aradau Dismantling the apparatus
of domination? Left critiques of AI, in: Radical Philosophy.
Session 1: Reflect
Artificial Intelligence in the Creative and Cultural Sectors: The Case of Museum and Cultural Heritage
— Artificial intelligence (AI) can be seen as a set of technological solutions allowing content prediction, optimization in resource allocation or personalization of services. Although not new, the considerable improvement in its performance has contributed to the diffusion of this family of technologies to many products and services, especially in the creative and cultural sectors. Visual arts, music, and new media have been areas of great emphasis. In comparison, the museum and cultural heritage sectors may have experienced a slower uptake of this technology, but the transformations brought about by the spread of AI are present and it is important to understand how it will affect these sectors in the long term. In this presentation, I will draw on the results of recent European efforts to understand the role of artificial intelligence in the creative and cultural sectors. I will give a quick overview of the different sectors before focusing on Museums and Cultural Heritage. As a researcher at the interface between AI and creative practices, I will also try to identify relevant perspectives.
Unboxing machine learning: Data handling toolbox and digital capitalism’s go-to machine
— AI-inside applications have become ubiquitous, while remaining mostly invisible. They play an ever-increasing role in digital capitalism’s revenue creation. Although these data-driven micro-machines are no less powerful or frightening as any machine component, in aggregate, they are maturing into a powerful machinery lead by the hand of digital corporations. AI became the “latest machine of capital”, to use a phrase by Ernst Bloch. The debate on AI so far circled mostly around ethics, black boxes or inherent biases. But a “conceptual model” (Don Norman) of the technologies involved might not suffice anymore. The talk pleads for an opening up of “black boxes” and takes up the cudgels for an ever-deeper understanding of machine learning algorithms and their applications.
AI and art. Arguments for practice
— The generation of pictures by computers, from early beginnings to recent successes in AI or ML research, have mostly been discussed in terms of whether the machine can produce art. However, this discussion usually presupposes equally questionable notions of the deployed technologies as of artistic creation. Therefore, I would like to argue that the relationship of AI and art should be discussed with artistic practice in focus and pursue the question of what role concrete AI or ML technologies can play in it.
Oumaima Hajri is a researcher and lectures at the Rotterdam University of Applied Sciences. Her work focuses on the intersection of AI, ethics and society. For the Designing Responsible AI Media Applications project, she is investigating, in collaboration with Dutch media organisations, how AI can be applied in a responsible manner within the media context. Part of that project investigates how ethical guidelines can make an important contribution in translating strategic interventions into societal impact, by focussing on socio-technical principles. Furthermore, she is currently part of the first cohort of the MSt in AI Ethics & Society at the University of Cambridge, wherein she conducts research on the socio-political impact of AI, mainly focusing on decolonisation and demystification. In her latest research she has examined the role of AI within the migration control context and how this is further marginalising asylum seekers’ and refugees’ position. She holds a BSc in Political Science (Vrije Universiteit, Amsterdam) and a MSc in Data Science & Society (Tilburg University)
Session 2: Empower
Power, Data and Control: Critical Technology Discourse in the Museum
— The talk will examine how institutional bias multiplies technological bias to shape how we see and experience the world. The focus of the discussion will be on museums as public institutions and their role in shaping how visitors experience digitally mediated worlds. I will argue that while social and technological changes are not new concepts for cultural organizations, what is new is the depth and reach of these technologies in terms of art form development, data creation, manipulation, and interpretation. In conclusion, the talk will outline how cultural organizations and practitioners can create space for important conversations about power, data, and control.
Dr. Oonagh Murphy is an expert on museums and digital culture, she is Co-Founder of the UK-USA Museums + AI Network, advisor to the Department of Digital Culture Media and Sport (UK Government), and an Open Data Institute Accredited Data Ethics Professional. Lecturer, Institute for Creative and Cultural Entrepreneurship, Goldsmiths, University of London
Digital Curation and Collections for AI: opportunities and risks for cultural heritage institutions
— Numerous use cases, from text recognition to image analysis or classification to contextualization, have already demonstrated how digitization and curation can benefit from the use of AI. But there is also a risk in applying black-box technologies from the private sector to cultural data without fully understanding the implications. This talk will present successful examples for AI in cultural heritage, highlight some of the recent criticisms of data practices in the domain of AI, and raise ideas on the role of digital curators and cultural heritage institutions aiming to get involved in AI.
Clemens Neudecker studied Philosophy, Computer Science and Political Science at LMU Munich and currently works as research advisor in the Directorate General of the Staatsbibliothek zu Berlin – Preußischer Kulturbesitz. His research focuses on computer vision, natural language processing and on the application of machine learning in the context of digital humanities
Archives, Fake News and AI
— Artificial intelligence is often seen as an accelerator of fake news. Increasingly accessible, it puts powerful tools in the hands of everyone to mass-produce misleading content. Texts, images, videos … all the information media we consult online can be the object of these misappropriations. Historical content such as archives are not immune to such manipulation either. However, artificial intelligence has the paradoxical aspect of being able to be both the evil and the solution, and more and more applications for detecting forgeries are emerging in art and culture.
Discovering Culture with AI
— The past few years have seen a rapid development of AI capabilities and applications, including in the fields of art and culture. Machine learning tools now find a variety of uses in cultural institutions, such as improving accessibility, aiding research and providing new forms of audience engagement through roaming robots, deepfake installations, chatbots and interactive image processing applications. At the same time, museums serve as venues for AI art exhibitions and discussions about technology ethics. This talk will give an overview of creative AI practices by cultural institutions, showcase artistic exploration with AI and consider tools for public engagement with museum collections.
Luba Elliott is a curator, writer and researcher specialising in creative AI. Her recent projects include the Feral File exhibition Reflections in the Water and the in Leicester, UK. She is an Honorary Senior Research Fellow at the UCL Centre for Artificial Intelligence.
Session 3: Next Intelligence
Symbolic vs. Subsymbolic Knowledge Representation, an Epic Dilemma?
— Over the last decade, deep learning methods made tremendous progress. Massive parallelization via GPUs, huge training data harvested from the Web, and efficient neural network architectures enable humanlike or even superhuman performance in specific areas. Huge pre-trained language models seem to capture complex semantics of natural languages and obtain outstanding results in classification, prediction, or generation tasks. The same holds for the image generation domain with models like Stable Diffusion or Dall-E2. As a result, do we still need symbolic knowledge representations and logics? Will Deep Learning models take over and will symbolic logic, ontologies, or knowledge graphs become an obsolete niche product? In this talk, we will look at various examples from both worlds and show that each by itself alone might fail. Both sides will have to join forces to succeed and move forward.
Prof. Dr. Harald Sack is Division Manager for Information Service Engineering at FIZ Karlsruhe – Leibniz Institute for Information Infrastructure and Professor at the Karlsruhe Institute of Technology (KIT), Institute for Applied Informatics and Formal Description Methods (AIFB) with the research group Information Service Engineering.
Transparency, Reproducibility and Fairness in Data Science and AI
— For Data Science and AI projects, transparency, reproducibility and fairness are very important aspects. To work towards these goals, making publications, data, models and code available, interlinking them, and offering innovative tools and services can help. The overarching objective of NFDI4DS is the development, establishment, and sustainment of a national research data infrastructure for the Data Science and AI community. NFDI4DS aims to support all steps of the complex and interdisciplinary research data lifecycle, including collecting/creating, processing, analyzing, publishing, archiving, and reusing resources in Data Science and AI. In this talk, we will give an overview of NFDI4DS and its work programme. We will report on how we plan to utilize FAIR Digital Objects and Knowledge Graphs as a basis for the infrastructure envisioned. We will also give an overview of the services planned, and how they are meant to interact.
Dr. Sonja Schimmler is reponsible for the topic Research Data Infrastructures at Fraunhofer FOKUS. She is also an associated researcher at Technical University of Berlin. She leads the research group Digitalisation and Science at the Weizenbaum Institute and is principal investigator of the project A Digital Research Space for the BUA within the Berlin University Alliance. She is also actively involved in the NFDI initiative, as spokesperson of NFDI4DataScience, and as co-spokesperson of NFDI4Cat, Base4NFDI, and Section Common Infrastructures. In her research, she focuses on the digitalisation and opening up of science and puts a special emphasis on research data infrastructures.
Her research interests range from semantic web and linked data over data science and artificial intelligence to software engineering and human-centered computing. She holds a Ph.D. in Computer Science from the University of the Federal Armed Forces Munich. She studied Computer Science at the Technical University of Munich and at the Georgia Institute of Technology (USA).
From Quantity to Quality: Structuring Provenance Data
— With the extensive production of increasingly detailed provenance texts in museums, new possibilities for collaborative research within and across institutions are emerging. This necessitates the transformation of large quantities of unstructured textual information to become fully machine-readable. By addressing natural language processing challenges, AI can assist with the laborious and resource-intensive work of structuring provenance data. However, the research and documentation complexities of provenance records require human intervention. This paper proposes the application of artificial intelligence combined with a human-in-the-loop approach to unlock the potential of provenance data.
Cultures of AI: Perspectives from Digital Anthropology and their Potential for the Field of Museum Studies
— The presentation will introduce an analytical concept of Cultures of AI based on the approaches of cultural and digital anthropology. From this perspective, cultures of AI consist of the routines, relationships, and orders emerging from the assemblages of human actors and AI-based technologies in everyday life. While this general concept is designed for the study of AI in everyday culture, this presentation asks for its potential value in the field of museum studies. Instead of providing research results, it will elaborate on potential research questions that can be posed with the help of this analytical framework. The presentation is intended to spark a discussion about how we can better understand the role of AI in museums and critically evaluate their potential for professional museum work.
Prof. Dr. Christoph Bareither‘s research and teaching focus on the ethnographic study of everyday digital cultures, combining the strengths of cultural and digital anthropology. The aim of his work is to contribute to urgent socio-political debates by shedding light on the transformations of everyday practices and experiences enabled through digital technologies. Professor at Ludwig-Uhland-Institut für Empirische Kulturwissenschaft Universität Tübingen.
Open Session: Lightening Talks, Posters & Demos
Stefan Schaffer, Oliver Gustke
CHIM, Chatbot in Museums goes Open Source
— The project CHIM Chatbot in the Museum explored the use of AI-supported, interactional conversation systems in knowledge transfer in museums. CHIM is a cooperation between Linon Medien and DFKI (Deutsches Forschungszentrum für Künstliche Intelligenz) and was evaluated at the Städel Museum. Functionalities of the software are now made available open source.
Nicole High-Steskal, Rainer Simon
Linking Viennese Art through Artificial Intelligence. Evaluating the Blackbox
— The pilot project LiviaAI examines the use of artificial intelligence to identify connections between objects from three Viennese museums (Wien Museum, Museum für Angewandte Kunst, Belvedere Museum Wien). In the first project phase, collection metadata and their creation were investigated in order to derive specifications for an AI model for similarity determination. In the current project phase, a model is being developed that uses the metadata of a selected collection as input to learn cross-collection visual representations of similarity. The goal is to evaluate the model in terms of its practical utility for curators and museum visitors, and gain insights into AI decision mechanisms.
Dr. Nicole High-Steskal is course director at the University of Continuing Education Krems, her research and teaching focus on topics of digital transformation of museums.
Rainer Simon is a software developer and researcher at the Austrian Institute of Technology. He has been working on the representation, visualization and use of cultural data for over 15 years.
The SociAl BIas Observatory: Algorithmic Perspectives in the Human Search of Biases in Collections
— Given the vastness and complexity of the past, searching for social biases intertwined in collections of cultural heritage is a task that calls for the aid of computation. However, the current mainstream approaches in AI are unfit to support researchers, as they ignore a core tenet from the humanities: the perspectivity of people and biases. Explicitly not automating an inherently human task, SABIO, a visual search interface, and initial investigation equips users with algorithmic and humanly unattainable perspectives on collections. While thus affording users their own gaze, SABIO simultaneously highlights likely places of known biases and patterns that can reveal unknown ones.
Digital Curator: Motifs and Themes in Central European Fine Art Explored by Computer Vision
— How to see a million works of art? And what can this distant viewing bring to us? The lecture will focus on big culture historical data, in particular digitized fine art collections, and the intelligent algorithms that can handle these databases and allow to select (search, discover, recommend or filter) and interpret (classify, describe or link) specific artifacts in them. We will focus on computer vision technology, its principles of operation and the potential it can bring to the exploration of artworks.
Lukáš Pilka is a digital designer and media theorist focusing on interactive and communication design, contemporary technologies, new media and the overlap between these fields and the world of fine art. His research at the Academy of Arts, Architecture and Design in Prague focuses on the use of computer neural networks for the automated classification and quantitative interpretation of works of art, particularly classical paintings, drawings and prints.
Ana Müller, Anke Neumeister
„Talk to me!“ Research and Development of AI-based Conversations in a Museum
— Robots have long been researched in the laboratory and used away from the public eye. Technological progress is paving their way into public spaces and cultural institutions become promising for AI systems’ research and development. At the OZEANEUM Stralsund the cooperating projects (Digital) MEER erleben (BKM) and SKILLED (BMBF) evaluated how visitors interact with a Furhat Robot connected to an AI-based dialog system, which is currently researched with the goal to develop socioempathic human-machine interaction. The results of the sequential mixed method provide insights into user expectations and improvement of service quality. AI technologies are advantageous for the interaction with museum guests as these systems – if well designed – not only offer information and service but a new world of infotainment.
Ana Müller, M.A. Sociology, Research Associate and PhD Student at Cologne Cobots Lab,
TH Köln – University of Applied Science, Germany. Research Project: BMBF (Federal Ministry of Education and Research) funded project
Skilled – Socioempathic AI-Based Dialogues
Anke Neumeister is a research associate and coordinator for the subproject (Digital) MEER erleben of the German Oceanographic Museum
in the network museum4punkt0. From 2018
to 2020, she was a scientific volunteer at the Senckenberg Museum of Natural History Görlitz.
Real-Time Visitor Tracking for the Deutsche Museum Nuremberg
— Advances in AI enable us to process data not only with high accuracy but also in real-time. But these advances are oftentimes misused to excessively analyze data of customers and citizens for various reasons. The goal of our project is to showcase this scenario to willing participants in the Deutsche Museum Nürnberg by introducing an inconspicuous surveillance atmosphere. For this, we are developing a privacy-compliant multi-camera visitor tracking system. The system performs in real-time, estimates trajectories, and – in limited scope – personal parameters of the visitors to generate visitor profiles. The prototype performs satisfactorily under laboratory and study conditions.
Melanie Fahden, Anja Gebauer
With artificial intelligence to art! Chatting with Salomé and Co. through IBM Watson
— Helena in front of the fire, Salomé with John’s head – how do you chat with a femme fatale? What questions do young people ask such a figure? In a participatory project, students develop their own questions and answers, which are fed into the text-based dialogue system IBM Watson. The AI promotes authentic and lively conversations with the works.
Anja Gebauer holds a teaching degree and worked in artistic-pedagogical teaching and research at Ludwig-Maximilians-Universität Munich from 2016–2021. She completed her doctorate with the topic of digital art education in spring 2021. As a freelance speaker and workshop leader, she works for various cultural education institutions (Deutscher Museumsbund, BA Wolfenbüttel, NÖKU Kulturvermittlung, Kulturstiftung des Bundes). Since August 2021 she works as a research assistant for digital education & outreach at the Hamburger Kunsthalle.
Melanie Fahden studied cultural studies and cultural mediation with a focus on visual arts at the University of Hildesheim. In 2017, she trained as a staff member in memorial education and worked freelance at the Neuengamme Concentration Camp Memorial until 2020. Parallel to this, she worked as a teacher for art as well as in the area of politics-society-economy at an elementary school and at a district school. Since 2021, she has been a research assistant for school (secondary I+II), outreach & inclusion in the Education & Outreach department of the Hamburger Kunsthalle.
Michael Zöllner, Markus Bosl, Dirk Widmann, Moritz Krause
Symotiv – Virtual insights into the symphony orchestra
— The Symotiv project, funded as part of the Excellent Orchestral Landscapes program, aimed to analyze the workings of a symphony orchestra using motion analysis through machine learning and explain the aspects from rehearsal to performance to a wide audience via VR.
Further information and documentary film symotiv.de, Hochschule für Angewandte Wissenschaften Hof, Institut für Informationssysteme der Hochschule Hof
Curating Museum Collection Data with the Curator’s Machine
— Multimodal recommendation systems promise an intuitive discovery of museum collection data. The retrieval of objects changes from a narrow search for fixed terms to an interpretive approach toward contexts of meaning. The Curator’s Machine as a possible prototype enables curators to approach museum collections intuitively. By transforming the embedding space of CLIP the tool helps to initiate an explorative human-machine interaction in which the contextual knowledge of the experts is also included in the machine learning.
Dominik Bönisch studied cultural studies at the University of Hildesheim and the Moholy-Nagy University Budapest. He is currently the scientific project manager of the research project Training the Archive at the Ludwig Forum Aachen, where he is investigating the connections between artificial intelligence and museum collections.
Inferring User Interaction from Data. Using Museum Recommender System
— MuseREC is a recommender system for museum online collections that serves recommendations to users based on collection metadata and images. The technology uses various algorithms to create personalised, content-based recommendations for users and therefore establish a tailored pathways through collections that suits the users’ interests. However, applying such collections to museum data has several implications and our latest user study shows that extrinsic perception of such systems can significantly differ from intrinsic interaction. This demonstration sheds light on the those differences and highlights the pros and cons of such powerful artificial agents.
Dr. Lukas Hughes-Noehrer is an interdisciplinary HCI researcher in the Department of Computer Science at the University of Manchester. He is interested in fostering human-computer interaction to investigate the applicability and usability of algorithmic outputs in various settings, addressing trust issues, testing new strategies, exploring content creation and consumption by users, and the policies of their future use in a technically informed society. He is further a group organiser at the Alan Turing Institute – the UK’s National Institute for Data Science and Artificial Intelligence – where he leads the AI, Arts and Creative Industries Interest Group and the Local Network Lead of UKRN for the University of Manchester and other institutions in the North West.
ofxTensorFlow2 – A media arts application for creative empower-ment in the AI field
— The team of intelligent.museum has developed ofxTensorFlow2, an AI extension for the software toolbox openFrameworks. openFrameworks is an op en source software library developed by and for artists for programming creative applications. An openFrameworks add-on is an additional tool that provides specific functionality. ofxTensorFlow2 was developed to bridge the gap between AI developers and artists using openFrameworks. Users can run AI models trained by specialists in TensorFlow. This makes it much easier for artists to access
and use AI.
Sonja Thiel, Etienne Posthumus, Murat Kazokoglu
xCurator – AI Curation Tool for Museum Data & User Empowerment
— The xCurator is a tool that allows users to explore the collections of the Baden State Museum and the Allard Pierson Museum in a completely new way. It allows access to the depth of the content: With innovative AI technologies, the tool supports the user in perceiving the diversity of objects, learn about contexts and to explore topics in greater depth according to their own interests. With xCurator, users can simply slip into the curator role and create their own story from the collection.
Sonja Thiel leads the AI development in the project Creative User Empowerment (CUE) at Badisches Landesmuseum.
Etienne Posthumus is a senior researcher at FIZ Karlsruhe and project leader of CUE at Allard Pierson Amsterdam.
Murat Kazokoglu is a consultant at 3pc and implementation partner for the Proof of Concept of the xCurator tool.