Bruno Di Giorgi, M. Lévy, Richard R. Sharp · arXiv (Cornell University) · 2022
Vocoders are models capable of transforming a low-dimensional spectral representation of an audio signal, typically the mel spectrogram, to a waveform. Modern speech generation pipelines use a vocoder as their final component. Recent vocoder models developed for speech achieve a high degree of realism, such that it is natural to wonder how they would perform on music signals. Compared to speech, the heterogeneity and structure of the musical sound texture offers new challenges. In this work we focus on one specific artifact that some vocoder models designed for speech tend to exhibit when applied to music: the perceived instability of pitch when synthesizing sustained notes. We argue that the characteristic sound of this artifact is due to the lack of horizontal phase coherence, which is often the result of using a time-domain target space with a model that is invariant to time-shifts, such as a convolutional neural network. We propose a new vocoder model that is specifically designed for music. Key to improving the pitch stability is the choice of a shift-invariant target space that consists of the magnitude spectrum and the phase gradient. We discuss the reasons that inspired us to re-formulate the vocoder task, outline a working example, and evaluate it on musical signals. Our method results in 60% and 10% improved reconstruction of sustained notes and chords with respect to existing models, using a novel harmonic error metric.
Cited by 3Open access
E. A. Nehary, Sreeraman C. Rajan · 2025
Automatic classification of phonocardiogram (PCG) acquired through an electronic stethoscope is currently done using deep learning methods. Deep learning methods need large PCG datasets. Such datasets often exhibit heterogeneity due to differences in sensor specifications, recording locations, recording environments, and patient pathologies. Heterogeneous data pose significant challenges for classification, and training a single deep learning model that can generalize across such heterogeneous data is difficult. This work proposes a single deep learning model with a trainable fractional spectrogram (TFS) as input instead of traditional spectrograms. Using the newly proposed trainable fractional order <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$a$</tex> as input, and our previously proposed domain-balanced training, the model adapts the representation to provide improved classification despite heterogeneity in PCG. Preliminary results show that TFS outperforms the traditional spectrogram, with improvements of approximately 4% in average balanced accuracy and 11% in average Matthews correlation coefficient.
Cited by 0
Bertrand David, Romain Hennequin, Jean-Louis Durrieu et al. · HAL (Le Centre pour la Communication Scientifique Directe) · 2010
International audience
Cited by 0
Lifan Yang, Shih-Lun Lin, Shang‐Da Yang · Optics Express · 2010
We experimentally demonstrate that an interferometric spectrogram (i.e., fringe- and frequency-resolved autocorrelation trace) with high signal-to-noise ratio can be processed by three different procedures to accurately retrieve the spectral phase profile. We also show that the data redundancy built in the interferometric spectrogram permits simultaneous retrieval of multiple spectral phase solutions, and their weighted average may give a highly robust result against the measurement noise.
Cited by 4Open access
Antikainen, Aku, Agrawal, Govind · Figshare · 2019
The total spectrogram over all cores of a hexagonal seven-core fiber at various propagation distances. The cores have a diameter of 6 µm and are separated by 15.5 µm. The input pulse is a 15 kW 100 fs sech-pulse launched in the center core.
Cited by 0Open access
Antikainen, Aku, Agrawal, Govind · Figshare · 2019
The total spectrogram over all cores of a hexagonal seven-core fiber at various propagation distances. The cores have a diameter of 6 µm and are separated by 25 µm. The input pulse is a 15 kW 100 fs sech-pulse launched in the center core.
Cited by 0Open access
Siti Nur Aishah, Ahmad Shah · UMP Institutional Repository (Universiti Malaysia Pahang) · 2012
In this project, the chemical concentration identified based on spectrogram. In chemistry, concentration is defined as the abundance of a constituent divided by the total volume of a mixture. The term concentration can be applied to any kind of chemical mixture, but most frequently it refers to solutes in homogeneous solutions. Meanwhile, spectrogram is a time-varying spectral representation (forming an image of graph) that shows how the spectral density of a signal varies with time. It is an instrument used to measure properties of light over a specific portion of the electromagnetic spectrum, typically used in spectroscopic analysis to identify materials. The purpose of this project was to develop the systems that can identify the concentration of a compound by using spectrogram. Using MATLAB, the chemical concentration can calculated based on the compound of the solution. By using Acrylic acid as an example; which the simplest unsaturated carboxylic acid which has double bond and carboxyl group in C3 one molecule with the formula CH2=CHCOOH, we can determine the concentration percentages of the compound that form acrylic acid. Based on the graph that plotted by using MATLAB, the area under graph calculate. With the area, we can calculate the concentration directly based on the data from HPLC-high performance liquid chromatography.
Cited by 0Open access
Jack A. Culotta (21387226), Marie L. Ervin (21387229), Brooke J. Vetter (4925914) et al. · Figshare · 2025
Upper: Spectrogram and Lower: power spectrum. Hydrophone recording of the broadband sound stimulus played in the large choice tank for conditioning and phonotaxis experiments. Recorded in the middle of the water column directly above the speaker. (TIFF)
Cited by 0Open access
Emmanuelle C. Leroy (3339648), Flore Samaran (445299), Julien Bonnel (3339651) et al. · Figshare · 2016
<p>The noisy frequency band between 18 and 28 Hz is formed by the Antarctic blue whale and fin whale chorus.</p>
Cited by 0Open access
Uttara Bahad, Rishika Sardana, Sakshi Sanjay Chalke et al. · International Research Journal of Modernization in Engineering Technology and Science · 2023
The thing that is common to all the people across the world, or the thing that transforms boundaries and connects us all is music.According to statistics every human hears approximately 15 minutes of music every day.Music is one category that all humans consume in all their moods, no matter if they are feeling happy, sad, angry, or lonely.Hence to cater to this need of music generating industry is a multi-billion industry.Everyday millions of songs/music all around the world is being produced and a variety of people like directors, producers, composers, singers, music-writers are included in this lengthy process.Through our research we aim to produce a mock model which will make music creating a less time consuming, lengthy, and costly process.This can be done by using AutoEncoders which are a part of unsupervised neural networks.This machine learning technique will help us generate music, which follows a specific pattern of chords.We aim to create a realistic music tone which is a synthetic data.This can be done by training VAE with Keras and by performing Short-Time/Inverse Short-Time Fourier Transformations.
Cited by 0Open access
Asinenka, A. M. · Репозиторий БГУИР (BSUIR Repository) · 2024
This article explains spectrogram of the speech signal (analysis and processing) \nwith MATLAB to get its frequency-domain representation. In real life, we come across many \nsignals that are variations of the form f(t), where «t» is independent variable «time» in most \ncases. Temperature, pressure, pulse rate, etc can be plotted a long the time axis to see variations \nacross time .
Cited by 0Open access
Wen Xue, M. Sandler · IET Signal Processing · 2009
The authors propose a time–frequency (T–F) analysis method that uses a time- and frequency- dependent resolution to represent a signal. The method is based on the idea of splitting the T–F plane into equal-TF-area Heisenberg boxes in some optimal way that closely matches spectral events. Compared with existing methods based on orthogonal decompositions, by lifting the orthogonality constraint, extra freedom is gained in the way the T–F plane can be partitioned, which enables time and frequency adaptation at the same time. A best tiling selection algorithm of quadratic complexity is derived using dynamic programming to find the optimal frame from a family. Experiments show the advantage of this more flexible representation.
Cited by 1
Feiyang Yu, T. W. Lin, Ke Xu · Applied Optics · 1985
A white-light optical processing technique of generating speech spectrogram is presented. The speech spectrograms thus generated are frequency color-coded resulting in easier visual discrimination. The temporal-to-spatial conversion of speech signal is accomplished by means of density modulation with a CRT scanner. The scaling procedure of the speech spectrogram as well as the frequency resolution limit of the system is discussed. Experimental demonstrations of the proposed frequency color coded speech spectrograms generated by the proposed white-light optical processing technique are given.
Cited by 3
Kaichiro Hatazaki, Yasuhiro Komori, Takeshi Kawabata et al. · International Conference on Acoustics, Speech, and Signal Processing · 2003
A method is presented for phoneme segmentation by an expert system utilizing spectrogram reading strategy and knowledge. The expert system detects phonemes in a spectrogram and determines their boundaries as well as their coarse categories. To simulate a human expert spectrogram reading process, the system performs assumption-based inference with certainty factors, and top-down acoustic feature extraction under phonetic context hypotheses. The system, into which Japanese consonant segmentation knowledge is incorporated, is able to detect about 90% of the phonemes correctly. In particular, the phoneme boundaries detected by the system are as accurate as those detected by human experts. The result is that the phonemes obtained by the expert system can be identified using a stochastic phoneme recognition method.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 17
Karim Habashy, Julio J. Valdés, Madison Cohen-McFarlane et al. · 2022
A variety of technologies can support aging in place, including smart home sensing that can enable independent living through real-time data analysis. In this work, we study cough sound analysis as the cough is a key symptom of many respiratory illnesses and conditions. Based on a data set of cough recordings, we propose a two-pronged approach: the first leverages unsupervised learning to compute intrinsic dimensions of the data and maps raw data for visualizations, and the second uses the insight to train machine learning models through transfer learning on Vision Transformer models. Data augmentation approaches are implemented to improve the performance of the models and our top-performing model achieves an F1-score of 0.804. This study suggests the feasibility of using smart sensing and deep learning for gaining insights into the health of older adults.
Cited by 9
Zerui Han, Xin Zhou · 2023
In this paper, a radio signal detection algorithm is proposed. This algorithm detects the radio signal from complex electromagnetic environment to obtain the center frequency, bandwidth and starting and ending time. First, to detect the time and frequency information simultaneously, the I/Q data in time domain is transformed into spectrogram. Then, the noise threshold is calculated by equidistant binning statistics, and the spectrogram is binarized based on the threshold. A kind of filter operator is designed for binary image denoising. Then we use DBSCAN clustering algorithm to cluster signal points in binary spectrogram. Finally, after separating the overlapping signals and filtering the false signals, the center frequency, bandwidth, starting and stopping time of each signal can be extracted. For experimental evaluation, Average Precision (AP) and average Distance-DIOU (ADIOU) are used as evaluation indexes. Radio signal data with center frequency near 2.4GHz and bandwidth 40 MHz is collected. And 420 segments of test data are extracted for manual labeling. Detection experiments on the 420 test samples reveal that the algorithm we proposed can achieve an AP value of 0.839, and the ADIOU value reaches 0.674. The result shows that the algorithm can detect radio signals accurately.
Cited by 0
Matthias Pätzold, Néji Youssef · 2015
The analysis of the Doppler power spectral density (PSD) of measured and simulated data is an important topic in the area of mobile radio channel modelling. In this paper, we estimate the Doppler PSD of multipath fading channels by using the concept of the spectrogram. The spectrogram is a spectral representation that gives insight into how the distribution of the spectral density of a signal changes over time. The multipath fading channel is modelled by a sum-of-cisoids (SOC) process. A closed-form solution is presented for the spectrogram and the corresponding time-dependent autocorrelation function (ACF). The closed-form solutions disclose several unwanted effects that come with the limited sample space. One phenomenon is that the spectrogram changes with time, even if the underlying SOC process is ergodic. Another phenomenon is that the bandwidth of the spectrogram is not limited to the physical range determined by the maximum Doppler shift. These two phenomena can lead to misinterpretations of the spectrogram. Measures are discussed to reduce these effects. The results of this paper are important for the performance assessment of channel simulators as well as for the evaluation of measurement data collected in all kinds of mobile radio channels.
Cited by 9
Ama Marina Kreme, Adrien Meynard · arXiv (Cornell University) · 2024
We present a spectrogram separation method tailored for mixtures comprising two nonstationary components. By exploiting the unique characteristics of their time-frequency representations, we propose an inverse problem formulation to estimate the spectrograms of the components. We then introduce an alternating optimization algorithm that ensures the consistency of the estimated spectrograms. The efficacy of the algorithm is evaluated through testing on synthetic mixtures and is applied to a bioacoustic signal.
Cited by 0Open access
Radu Bălan · 2010
This paper presents a framework for discrete-time signal reconstruction from absolute values of its short-time Fourier coefficients. Our approach has two steps. In step one we reconstruct a band-diagonal matrix associated to the rank-one operator K <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> = xx*. In step two we recover the signal x by solving an optimization problem. The two steps are somewhat independent, and one purpose of this paper is to present a framework that decouples the two problems. The solution to the first step is connected to the problem of constructing frames for spaces of Hilbert-Schmidt operators. The second step is somewhat more elusive. Due to inherent redundancy in recovering x from its associated rank-one operator K <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> , the reconstruction problem allows for imposing supplemental conditions. In this paper we make one such choice that yields a fast and robust reconstruction. However this choice may not necessarily be optimal in other situations. It is worth mentioning that this second step is related to the problem of finding a rank-one approximation to a matrix with missing data.
Cited by 35
Purnama, Muhammad Thohawi Elziyad, Dewi, Cytra Meyliana Surya, Dhamayanti, Yeni et al. · Figshare · 2022
This file reported spectrogram analysis in respective bird sound.
Cited by 0Open access