Checklist of which Librosa functions are in Pliers

[koudyk]

Checklist of which Librosa functions are in Pliers

@PeerHerholz @adelavega

How to understand this checklist:

• I bolded functions that aren't in Pliers and might be useful.
• I put as star * beside functions that aren't in Pliers and I'm not sure whether they'd be useful because I don't understand what they do.
• I made some notes in italics.
• For the most part, I'm thinking about what could be useful in Neuroscout. E.g., I left out functions that change the audio (pitch shifting, time warping)

Link to the Librosa docs

FEATURE EXTRACTION

SPECTRAL

• chroma_stft
• chroma_cqt
• chroma_cens
• melspectrogram
• mfcc
• rms
• spectral_centroid
• spectral_bandwidth
• spectral_contrast
• spectral_flatness
• spectral_rolloff
• poly_features
• tonnetz
• zero_crossing_rate

AUDITORY

• tempogram

FEATURE MANIPULATION

(These functions might be useful in the transformations tab in Neuroscout)

• delta
• stack_memory

ONSET DETECTION

• onset_detect
• onset_backtrack
• onset_strength (This function would be useful if you want a single vector of onset strength rather than a vector for each frequency band, which is what the next function gives you. In Neuroscout, for example, I think that users likely wouldn't need onset strengths for every frequency band.)
• onset_strength_multi

BEAT AND TEMPO

• beat_track
• tempo

SPECTROGRAM DECOMPOSITION

• decompose
• hpss (This function is called by the harmonic-percussive source separation functions below, but returns time-frequency representations of the harmonic and percussive sources, not time-series. I can't see it being useful for Neuroscout, but it might be useful for other applications)
• nn_filter

EFFECTS

HARMONIC-PERCUSSIVE SOURCE SEPARATION

• hpss (Redundant with the combination of the next two functions.)
• harmonic
• percussive

TEMPORAL SEGMENTATION

RECURRENCE AND SELF-SIMILARITY

(Might be useful in Neuroscout for investigating something like the neural correlates of echoic or short-term memory)

• recurrence_matrix
• recurrence_to_lag
• lag_to_recurrence
• timelag_filter

TEMPORAL CLUSTERING

(Might be useful in Neuroscout for comparing the temporal structure of stimuli and brain responses)

• agglomerative
• subsegment

SEQUENTIAL MODELING

VITERBI DECODING

• viterbi *
• viterbi_discriminative *
• viterbi_binary *

TRANSITION MATRICES

• transitions_uniform *
• transitions_loop *
• transitions_cycle *
• transitions_local *

[adelavega]

Amazing! Thank you so much. This is a very helpful roadmap :)

[PeerHerholz]

Thx @koudyk, this is amazing. If you would be up, we could start working on this together, otherwise, I'd start to do it this week. Anyhow, we should track which functions we added here. @adelavega: how should we organize PRs, one big one with functions or smaller ones based on e.g., the feature type?

[PeerHerholz]

@koudyk and I had a brief meeting and decided to open a branch to implement the missing librosa functions. Once we have something, we'll check how to organize the PRs.

[tyarkoni]

That sound awesome, thanks! Re: PR, one big one would probably be easier from a reviewing standpoint, but if you find multiple smaller ones more efficient, that's also totally fine.

[adelavega]

@rbroc this is a listing of Librosa features in pliers, take a look!

[PeerHerholz]

JFYI: @koudyk and I are still working on implementing the missing features. Should have them ready soon!

[adelavega]

hey @PeerHerholz did you ever get a chance to work on this? If not no worries just wondering

[PeerHerholz]

Yeah, I did start working on a few features in my fork, but only finished some...

[adelavega]

ah okay just wondering. i was just perusing older issues.

if you have anything that partially covers some of this that is done and you'd like to submit go for it though.