Transformer Based Multimodal Summarization and Highlight Abstraction Approach for Texts and Speech Audios

Simple item page
dc.contributor.authorAltundogan T.G.
dc.contributor.authorKarakose M.
dc.contributor.authorTanberk S.
dc.date.accessioned2024-07-22T08:01:57Z
dc.date.available2024-07-22T08:01:57Z
dc.date.issued2024
dc.description.abstractMultimodal summarization is a kind of summarization application in which its inputs and/or outputs can be in different data types like text, video, and audio. In this study, a new approach based on fine tuning of different pre-trained transformers was developed for abstractive and extractive summarization of audio and text data. In the proposed method, abstractive and extractive summaries of text data are provided only as text, while extractive summaries of audio data are presented as both text and audio data. Abstractive summaries of the audio data are presented as text only. Transformers with text2text input-output relationship were used in both extractive and abstractive summarization processes of the proposed method. For the training and inference processes of audio this type of data to be handled in transformers, an ASR step was followed before the summarization step. The experimental results obtained were given in detail and compared with similar approaches in the literature. As a result of the comparison, it was seen that the proposed method achieved better performance than similar prior approaches. © 2024 IEEE.
dc.identifier.DOI-ID10.1109/IT61232.2024.10475775
dc.identifier.urihttp://akademikarsiv.cbu.edu.tr:4000/handle/123456789/11666
dc.language.isoEnglish
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.subjectAbstract data types
dc.subjectAbstracting
dc.subjectAudio data
dc.subjectAudio summarization
dc.subjectDatatypes
dc.subjectFine tuning
dc.subjectInput-output
dc.subjectMulti-modal
dc.subjectMultimodal summarization
dc.subjectSpeech audio
dc.subjectText data
dc.subjectTransformer fine-tuning
dc.subjectData mining
dc.titleTransformer Based Multimodal Summarization and Highlight Abstraction Approach for Texts and Speech Audios
dc.typeConference paper

Files

Collections

Scopus Koleksiyonu