Tackling multimedia data with AI techniques

Research

Multimedia data can be imagery, text, video, graphs, but also other informational context like geocoordinates. The predominant question the group tries to answer: How can you bring together all this information in a way that users get a better understanding of it?

The researchers work with three main themes; Multimedia integration, the interaction of users and multimedia visualization. Applications can be found within the fields of health, forensics, law enforcement, cultural heritage, urban liveability, and social media analysis. For each partner they work with, MultiX tries to extract the richest information possible from large collections of data.

Facts & figures

While other research groups traditionally separate the work on computer vision, text and visualization, MultiX brings the research on multimedia together in a unique way. This distinguishes them in the Netherlands and beyond. About a dozen PhD students and post-docs work in the group and it plans to grow in the coming years.

The group received several European funds in recent years like the Interreg programme funding and the Marie Curie Fellowship. Other prominent funds come from the two ICAI labs that are a part of the research group. ICAI (Innovation Center for Artificial Intelligence) is a national network that stimulates AI technology and talent development between academia, industry and government.

Partnership & collaborations 

Industry partner of the AI for Medical Imaging (AIM) Lab, one of the ICAI labs MultiX participates in, is the Inception Institute of Artificial Intelligence (IIAI) from the United Arab Emirates. This lab looks at medical image recognition using AI, but also at the combination of images and electronic health records. In the second ICAI lab, the Police Lab AI, the group collaborates with the Dutch Police and Utrecht University. The researchers develop state-of-the-art AI techniques to improve the safety in the Netherlands in a socially, legally and ethically responsible way.

Within ASGARD, a large European project on law enforcement, the group worked on new techniques like hypergraphs. Hypergraphs allow you to model group relations instead of only binary relations. This technology, which the group will continue to develop, has a great advantage in gaining insight into such diverse applications as social network analysis or fraud detection.

Future mission 

Due to the AI revival ten years ago, automated techniques have become incredibly good in the last few years. At some point researchers will start to see the limits of what automatic AI techniques can do. Breakthroughs will then occur into integrating information rather than looking at individual data streams. Interactions with humans will also start to play a more important role. How can you combine machine intelligence with humans to create synergy? That way AI can start learning from the user.

MultiX positions itself in the Data Science and AI research themes of the Informatics Institute.