Real-time Multimodal Argument Mining

Overview

Argument Mining (AM) focuses on identifying reasoning patterns and argumentative structures in natural language. To date, most AM systems have relied solely on text sources such as essays or debate transcripts, overlooking the acoustic information present in spoken contexts. In political debates, for example, audio conveys cues that can significantly improve system performance and better reflect how arguments are naturally exchanged.

Another limitation of current AM approaches is that they operate offline, limiting their usefulness in real-time applications. This PhD project seeks to address these challenges by developing real-time multimodal AM systems that integrate both text and audio inputs, enabling discourse analysis as debates or conversations unfold. The outcomes could revolutionise how electoral debates are consumed and inform the development of time-sensitive decision-support systems.

Why It Matters:

• Innovative Approach: Integrates multimodal (text + audio) inputs for richer and more accurate argument analysis.
• Real-Time Impact: Opens new possibilities for live applications in politics, media, and decision support.
• Global Relevance: Advances understanding of spoken argumentation, improving systems used in democratic and societal contexts.s.

Key Objectives:

1. Design a new dataset for real-time multimodal AM including both audio and text with timestamps.
2. Develop algorithms that model acoustic features or combine them with textual features.
3. Create a real-time AM architecture capable of processing discourse without sacrificing performance.

References:

1. Lawrence, J. and Reed, C. (2020). Argument mining: A survey. Computational Linguistics, 45(4), 765–818.
2. Gemechu, D., Ruiz-Dolz, R. and Reed, C. (2024). ARIES: A General Benchmark for Argument Relation Identification. Proc. 11th Workshop on Argument Mining, 1–14.
3. Ruiz-Dolz, R. et al. (2021). Transformer-based models for automatic identification of argument relations: A cross-domain evaluation. IEEE Intelligent Systems, 36(6), 62–70.
4. Lippi, M. and Torroni, P. (2016). Argument mining from speech: Detecting claims in political debates. Proc. AAAI Conf. Artificial Intelligence, 30(1).
5. Mancini, E. et al. (2022). Multimodal argument mining: A case study in political debates. Proc. 9th Workshop on Argument Mining, 158–170.

Diversity statement

Our research community thrives on the diversity of students and staff which helps to make the University of Dundee a UK university of choice for postgraduate research. We welcome applications from all talented individuals and are committed to widening access to those who have the ability and potential to benefit from higher education.