Sistema híbrido inteligente para el análisis de contenido multimedia

Abstract

[ES] Los grandes avances en las áreas de las TIC, el IoT y la IA han propiciado una serie de sistemas cuyo uso se ha visto incrementado exponencialmente en los últimos años, fomentando la generación de ingentes cantidades de datos de naturaleza heterogénea. Las propuestas recogidas en la literatura para la explotación de estos datos están enfocadas a la resolución de problemas muy específicos, favoreciendo el desaprovechamiento de la información. Este trabajo plantea una arquitectura modular y flexible para implementar un sistema híbrido inteligente capaz de soportar diferentes procesos de análisis de contenido multimedia gracias a la adaptación del concepto de ETL y la aplicación de tuberías de datos. Con el objetivo de comprobar el potencial de la arquitectura propuesta, se diseñan dos frameworks para la automatización del proceso de composición musical descriptiva a partir de contenido audiovisual y se desarrollan dos casos de estudio bien diferenciados donde se aplican diversas técnicas de extracción de meta-información y algoritmos enmarcados en el área del aprendizaje automático. La discusión de los resultados obtenidos se realiza considerando el rendimiento de los algoritmos y la aceptación social de la música por medio de diferentes test de usuario. En conclusión, la propuesta favorece la validación de la hipótesis previamente establecida, evidenciando que los datos multimedia analizados mediante técnicas de IA permiten crear otro tipo de información útil para el usuario.

[EN] Great advances in the fields of Information and Communication Technologies, IoT and AI have led to a series of systems whose use has increased exponentially over the past few years. This has encouraged the generation of huge amounts of data of a heterogeneous nature. The state of the art gathers many proposals for the exploitation of these data, but they all focus on the resolution of specific problems, favouring the waste of information. This work proposes a modular and flexible architecture to implement an intelligent hybrid system for the analysis of multimedia content. Thanks to the adaptation of the ETL concept and the application of data pipelines the system can support the concurrence of several analysis processes running in parallel. With the aim of verifying the potential of the proposed architecture, two frameworks are designed. Both are oriented to the automatic composition of descriptive music based on audiovisual content and they are put into operation in two well-differentiated case studies where diverse metadata extraction techniques and algorithms are applied within the context of machine learning. The performance of the algorithms and the social acceptance of the music are taken into account to validate the results obtained in this work. In closing, the proposal favours the validation of the previously established hypothesis, proving that the analysis of multimedia data through AI techniques allows the creation of other relevant information.

[EN]Great advances in the fields of Information and Communication Technologies, IoT and AI have led to a series of systems whose use has increased exponentially over the past few years. This has encouraged the generation of huge amounts of data of a heterogeneous nature. The state of the art gathers many proposals for the exploitation of these data, but they all focus on the resolution of specific problems, favouring the waste of information. This work proposes a modular and flexible architecture to implement an intelligent hybrid system for the analysis of multimedia content. Thanks to the adaptation of the ETL concept and the application of data pipelines the system can support the concurrence of several analysis processes running in parallel. With the aim of verifying the potential of the proposed architecture, two frameworks are designed. Both are oriented to the automatic composition of descriptive music based on audiovisual content and they are put into operation in two well-differentiated case studies where diverse metadata extraction techniques and algorithms are applied within the context of machine learning. The performance of the algorithms and the social acceptance of the music are taken into account to validate the results obtained in this work. In closing, the proposal favours the validation of the previously established hypothesis, proving that the analysis of multimedia data through AI techniques allows the creation of other relevant information.