high resolution piano transcription with pivots and pedals
high resolution piano transcription
Automatic music transcription (AMT) is the task of transcribing audio recordings into symbolic representations such as Musical Instrument Digital Interface (MIDI). Recently, neural networks based methods have been successfully applied to AMT and achieved state-of-the-art results. However, most previous AMT systems predict the presence or absence of notes frame-wise, meaning that their transcription resolution is limited by the hop size time between adjacent frames. Moreover, they are sensitive to misaligned onsets and offsets labels of audio recordings. Therefore, we propose a high resolution piano transcription system with pivots and pedals that can deal with these issues.
We introduce a log mel spectrogram and a new training algorithm to train our proposed piano transcription system. Specifically, we use the log mel spectrogram to model each piano note as a set of overlapping sine waveforms, and we use the ATR loss function to penalize the amplitudes of these overlapping sine waves. We then decompose each audio clip into multiple spectrograms, and we compute the multiplication of these spectrograms to extract the individual piano notes.
The main goal of our piano transcription system is to capture the nuances of piano performance. We achieve this by including the pedals in the piano-playing process, and leveraging the interaction between the manual keyboard and the damper pedal. We also propose a new metric that reflects the tempo stability of a pianist’s pedal-playing. Compared to other metrics, ours is more precise and is capable of quantifying the dynamical behaviour of a pianist’s pedal-playing.
high resolution piano transcription with pivots and pedals
Our metric is based on the assumption that the damper pedal is always on. As such, the metric can be used as an objective performance measure to assess a pianist’s ability to control the pedal-playing dynamics. It is worth noting that the metric can be extended to include the use of both the left and right pedals, as long as both are on at the same time.
While the onsets and frames system of [25] is a benchmark piano transcription system, it is not sensitive to the shifts in onsets and offsets within a single frame, as illustrated in Fig 1. The onset target of this AMT system reflects an average of several onsets over two adjacent frames. We propose a more precise onset target that takes into account the exact onset times of each note, as shown in Fig 2.
Pedal transcription is challenging because it requires careful coordination of the feet and hands. For example, the right foot must be perfectly aligned with the pedal key, and the left foot must remain on top of the keys to prevent them from slipping off. Furthermore, if the pedal is released prematurely, it can cause a drum-like “thump” that resonates through the piano case. For these reasons, it is critical for a pianist to understand and practice pedal-playing dynamics. Fortunately, a good understanding of the pedal-keyboard linkage allows pianists to quickly detect and correct any inconsistencies in their pedal-playing.