multimodal dimensions of human expression

Arjan van Hessen, Twente University and Stef Scagliola, Erasmus University Rotterdam


January 29, 2015.  15.00- 17.00 hrs. At the eHumanities Group.

Speeding up research and exploiting the multimodal dimensions of human expression: the blessings and perils of HLT for Speech Retrieval

The state of the art within ASR at present allows scholars to search in spoken content. However, this is true for certain languages and only if the audio-recordings do have a certain quality. ASR can be divided into real speech recognition and alignment where existing transcriptions can be aligned with the audio, enabling the search in the spoken content.

Moreover, modern HLT-technology enables the (semi-) automatic creation of additional metadata such as “emotion detection”, “body language extraction”, “audio summarisation” and more. Technology that can be used to enrich the new and existing collections of interviews.

In the presentation a variety of applications will be illustrated that can be used in oral history, psychiatry, language studies, political science and media studies.



Stef Scagliola is a postdoc researcher at the Erasmus University in Rotterdam, specialized in oral history archives and Digital Humanities. Her topical interest focuses on decolonization wars, military cultures and on how societies come to terms with historical taboos and injustices. The method through which these themes are discussed and communicated is by making use of digital technology. She was involved in the creation of several ‘digital born’ oral history collections: on experiences of veterans, of children born of war ( and of witnesses to the Yugoslav wars ( . At present she is responsible for designing a multidisciplinary research agenda for the Post Yugoslav Voices-project and for developing a DH- humanities curriculum. She has published in the European Review of History, the Journal of Genocide Research and in the proceedings of several Digital Humanities conferences.


Arjan van Hessen is a postdoc researcher at the University of Twente, specialized in the use of HLT for the research in spoken content. Moreover, he is member of the executive board of CLARIN-NL (2009-2015) and CLARIAH (2015-2019). Half of the week he works as “Head of Imagination” at Telecats: a software company specialized in the use of HLT in men-machine-interaction (call centre application, searching in Spoken Meetings such as the Dutch Parliament, etc.).



Information  to follow