Browsing by Author "Panda, Renato"
Now showing 1 - 10 of 18
Results Per Page
Sort Options
- Audio Features for Music Emotion Recognition: a SurveyPublication . Panda, Renato; Malheiro, Ricardo; Paiva, Rui PedroThe design of meaningful audio features is a key need to advance the state-of-the-art in Music Emotion Recognition (MER). This work presents a survey on the existing emotionally-relevant computational audio features, supported by the music psychology literature on the relations between eight musical dimensions (melody, harmony, rhythm, dynamics, tone color, expressivity, texture and form) and specific emotions. Based on this review, current gaps and needs are identified and strategies for future research on feature engineering for MER are proposed, namely ideas for computational audio features that capture elements of musical form, texture and expressivity that should be further researched. Finally, although the focus of this article is on classical feature engineering methodologies (based on handcrafted features), perspectives on deep learning-based approaches are discussed.
- "Back in my day...": A Preliminary Study on the Differences in Generational Groups Perception of Musically-evoked EmotionPublication . Louro, Pedro; Panda, RenatoThe increasingly globalized world we live in today and the wide availability of music at our fingertips have led to more diverse musical tastes within younger generations than in older generations. Moreover, these disparities are still not well understood, and the extent to which they affect listeners' preferences and perception of music. Focusing on the latter, this study explores the differences in emotional perception of music between the Millennials and Gen Z generations. Interviews were conducted with six participants equally distributed between both generations by recording their listening experience and emotion perception on two previously compiled sets of songs representing each group. Significant differences between generations and possible contributing factors were found in the analysis of the conducted interviews. Findings point to differences in the perception of energy of songs with specific messages of suffering for love, as well as a tendency from the younger group to perceive a well-defined emotion in songs representing their generation in contrast to neutral responses from the other group. These findings are preliminary, and further studies are needed to understand their extent. Nevertheless, valuable insights can be extracted to improve music recommendation systems.
- A Comparison Study of Deep Learning Methodologies for Music Emotion RecognitionPublication . Louro, Pedro; Redinho, Hugo; Malheiro, Ricardo; Paiva, Rui Pedro; Panda, RenatoClassical machine learning techniques have dominated Music Emotion Recognition. However, improvements have slowed down due to the complex and time-consuming task of handcrafting new emotionally relevant audio features. Deep learning methods have recently gained popularity in the field because of their ability to automatically learn relevant features from spectral representations of songs, eliminating such necessity. Nonetheless, there are limitations, such as the need for large amounts of quality labeled data, a common problem in MER research. To understand the effectiveness of these techniques, a comparison study using various classical machine learning and deep learning methods was conducted. The results showed that using an ensemble of a Dense Neural Network and a Convolutional Neural Network architecture resulted in a state-of-the-art 80.20% F1 score, an improvement of around 5% considering the best baseline results, concluding that future research should take advantage of both paradigms, that is, combining handcrafted features with feature learning.
- Decoding Musical Valence and Arousal: Exploring the Neural Correlates of Music-Evoked Emotions and the Role of Expressivity FeaturesPublication . Sayal, Alexandre; Guedes, Ana Gabriela; Almeida, Inês A. T.; Jardim Pereira, Daniela; Lima, César F.; Panda, Renato; Paiva, Rui Pedro; Sousa, Teresa; Castelo-Branco, Miguel; Bernardino, Inês; Direito, BrunoMusic conveys both basic emotions, like joy and sadness, and complex ones, such as tenderness and nostalgia. Its effects on emotion regulation and reward have attracted much research attention, as the neural correlates of music-evoked emotions may inform neurorehabilitation interventions. Here, we used fMRI to decode and examine the neural correlates of perceived valence and arousal in music excerpts. Twenty participants were scanned while listening to 96 music excerpts, classified beforehand into four categories varying in valence and arousal. Music modulated activity in cortical regions, most noticeably in music-specific subregions of the auditory cortex, thalamus, and regions of the reward network such as the amygdala. Using multivoxel pattern analysis, we created a computational model to decode the perceived valence and arousal of the music excerpts with above-chance accuracy. We further explored associations between musical features and brain activity in valence-, arousal-, reward-, and auditory-related networks. The results emphasize the involvement of distinct musical features, notably expressive features such as vibrato and tonal and spectral dissonance in valence, arousal, and reward brain networks. Using ecologically valid music stimuli, we contribute to delineating the neural correlates of music-evoked emotions with potential implications in the development of novel music-based neurorehabilitation strategies.
- Envisaging a global infrastructure to exploit the potential of digitised collectionsPublication . Groom, Quentin; Dillen, Mathias; Addink, Wouter; Ariño, Arturo H.; Bölling, Christian; Bonnet, Pierre; Cecchi, Lorenzo; Ellwood, Elizabeth R.; Figueira, Rui; Gagnier, Pierre-Yves; Grace, Olwen; Güntsch, Anton; Hardy, Helen; Huybrechts, Pieter; Hyam, Roger; Joly, Alexis; Kommineni, Vamsi Krishna; Larridon, Isabel; Livermore, Laurence; Lopes, Ricardo Jorge; Meeus, Sofie; Miller, Jeremy; Milleville, Kenzo; Panda, Renato; Pignal, Marc; Poelen, Jorrit; Ristevski, Blagoj; Robertson, Tim; Rufino, Ana C.; Santos, Joaquim; Schermer, Maarten; Scott, Ben; Seltmann, Katja; Teixeira, Heliana; Trekels, Maarten; Gaikwad, JitendraTens of millions of images from biological collections have become available online over the last two decades. In parallel, there has been a dramatic increase in the capabilities of image analysis technologies, especially those involving machine learning and computer vision. While image analysis has become mainstream in consumer applications, it is still used only on an artisanal basis in the biological collections community, largely because the image corpora are dispersed. Yet, there is massive untapped potential for novel applications and research if images of collection objects could be made accessible in a single corpus. In this paper, we make the case for infrastructure that could support image analysis of collection objects. We show that such infrastructure is entirely feasible and well worth investing in.
- Exploring Deep Learning Methodologies for Music Emotion RecognitionPublication . Louro, Pedro; Redinho, Hugo; Malheiro, Ricardo; Paiva, Rui Pedro; Panda, RenatoClassical machine learning techniques have dominated Music Emotion Recognition (MER). However, improvements have slowed down due to the complex and time-consuming task of handcrafting new emotionally relevant audio features. Deep Learning methods have recently gained popularity in the field because of their ability to automatically learn relevant features from spectral representations of songs, eliminating such necessity. Nonetheless, there are limitations, such as the need for large amounts of quality labeled data, a common problem in MER research. To understand the effectiveness of these techniques, a comparison study using various classical machine learning and deep learning methods was conducted. The results showed that using an ensemble of a Dense Neural Network and a Convolutional Neural Network architecture resulted in a state-of-the-art 80.20% F1-score, an improvement of around 5% considering the best baseline results, concluding that future research should take advantage of both paradigms, that is, conbining handcrafted features with feature learning.
- Exploring Song Segmentation for Music Emotion Variation DetectionPublication . Ferreira, Tomas; Redinho, Hugo; Louro, Pedro L.; Malheiro, Ricardo; Paiva, Rui Pedro; Panda, RenatoThis paper evaluates the impact of song segmentation on Music Emotion Variation Detection (MEVD). In particular, the All-In-One song-structure segmentation system was employed to this end and compared to a fixed 1.5-sec window approach. Acoustic features were extracted for each obtained segment/window, which were classified with SVMs. The attained results (best F1-score of 55.9%) suggest that, despite its promise, the potential of this song segmentation approach was not fully exploited, possibly due to the small employed dataset. Nevertheless, preliminary results are encouraging.
- How Does the Spotify API Compare to the Music Emotion Recognition State-of-the-Art?Publication . Panda, Renato; Redinho, Hugo; Gonçalves, Carolina; Malheiro, Ricardo; Paiva, Rui PedroFeatures are arguably the key factor to any machine learning problem. Over the decades, myriads of audio features and recently feature-learning approaches have been tested in Music Emotion Recognition (MER) with scarce improvements. Here, we shed some light on the suitability of the audio features provided by the Spotify API, the leading music streaming service, when applied to MER. To this end, 12 Spotify API features were obtained for 704 of our 900-song dataset, annotated in terms of Russell’s quadrants. These are compared to emotionally-relevant features obtained previously, using feature ranking and emotion classification experiments. We verified that energy, valence and acousticness features from Spotify are highly relevant to MER. However, the 12-feature set is unable to meet the performance of the features available in the state-of-the-art (58.5% vs. 74.7% F1-measure). Combining Spotify and state-of-the-art sets leads to small improvements with fewer features (top5: +2.3%, top10: +1.1%), while not improving the maximum results (100 features). From this we conclude that Spotify provides some higher-level emotionally-relevant features. Such extractors are desirable, since they are closer to human concepts and allow for interpretable rules to be extracted (harder with hundreds of abstract features). Still, additional emotionally-relevant features are needed to improve MER.
- Improving Deep Learning Methodologies for Music Emotion RecognitionPublication . Louro, Pedro Lima; Redinho, Hugo; Malheiro, Ricardo; Paiva, Rui Pedro; Panda, RenatoMusic Emotion Recognition (MER) has traditionally relied on classical machine learning techniques. Progress on these techniques has plateaued due to the demanding process of crafting new, emotionally-relevant audio features. Recently, deep learning (DL) methods have surged in popularity within MER, due to their ability of automatically learning features from the input data. Nonetheless, these methods need large, high-quality labeled datasets, a well-known hurdle in MER studies. We present a comparative study of various classical and DL techniques carried out to evaluate these approaches. Most of the presented methodologies were developed by our team, if not stated otherwise. It was found that a combination of Dense Neural Networks (DNN) and Convolutional Neural Networks (CNN) achieved an 80.20% F1-score, marking an improvement of approximately 5% over the best previous results. This indicates that future research should blend both manual feature engineering and automated feature learning to enhance results.
- Inteligência para a Sustentabilidade das CidadesPublication . Pinho, Henrique J. O.; Lopes de Oliveira, Luís Miguel; Coelho, Paulo; Frazão Correia, Pedro; Panda, Renato