Andy Lücking, Goethe University Frankfurt, March 14, 2024
ABSTRACT: Language use is situated in manifold ways, including the exploitation of the visual context and the use of manual gestures (multimodal communication). I will survey recent theoretical advances concerning the semantics and the semantic contribution of co-verbal deictic and iconic gestures. Multimodal communication challenges traditional notions of reference and meaning developed in formal semantics. Computationally tractable models of deictic and deferred reference and iconic gestures are proposed instead. These models specify language/perception interfaces for two concrete phenomena that are central to situated language. Inasmuch as LLMs lack perception and embodiment, these phenomena are currently, but not in principle, out of reach. I will conclude by pointing out *what* is needed for an LLM to be capable of deferred reference and iconic gestures.
Andy Lücking is Privatdozent at Goethe University Frankfurt. His work contributes to theoretical linguistics and computational semantics, in particular to a linguistic theory of human communication, that is, face-to-face interaction within and beyond single sentences. Besides publishing on deixis and iconicity in manual gesture, Andy is the main author of Referential Transparency Theory, the current semantic theory of plurality and quantification. His work on the perception of iconic gestures received an IEEEbest paper award.
Andy Lücking, Alexander Henlein, and Alexander Mehler (2024). Iconic Gesture Semantics. In review. Manuscript available on request.
Andy Lücking and Jonathan Ginzburg (2023). Leading voices: Dialogue semantics, cognitive science, and the polyphonic structure of multimodal interaction. Language and Cognition, 15(1). 148–172.
Andy Lücking, Thies Pfeiffer and Hannes Rieser (2015). Pointing and Reference Reconsidered. In: Journal of Pragmatics 77: 56–79. DOI: 10.1016/j.pragma.2014.12.013.
Understanding LLM Understanding 