Skip to content

Commit

Permalink
v0.1.7
Browse files Browse the repository at this point in the history 
- Little edit in method `mpmToolbox.gui.score.ScoreDisplayPanel.mouseMoved()` that changes the mouse cursor to hand symbol when moved to a clickable overlay element.
- Fixed potential division-by-zero bug in `mpmToolbox.gui.syncPlayer.SyncPlayer.PlaybackRunnable.run()`.
- Reorganized some classes, i.e., classes `Score`, `ScoreNote`, `ScorePage` moved from package `mpmToolbox.gui.score` to package `mpmToolbox.projectData.score`.
- Added "Hide Overlay" button to the score widget (classes `mpmToolbox.gui.score.ScoreDocumentData` and `ScoreDisplayPanel`) that allows to show the score without the overlays.
- Addition to the spectrogram context menu to switch between normalized and non-normalized display.
- New method `getPart()` in classes `mpmToolbox.gui.msmTree.MsmTreeNode` and `mpmToolbox.gui.mpmTree.MpmTreeNode` to retrieve the MSM `part` element that the node belongs to.
- New package `mpmToolbox.supplementary.avlTree` that implements the AVL Tree data structure.
- New package `mpmToolbox.projectData.alignment` with several new classes that serve to associate measurements in audio recordings with MSM data, display them as piano roll and interact with it.
- Alignment data is stored in `mpr` project files.
- Added class `mpmToolbox.gui.audio.PianoRollPanel` which is also the basis for the classes `WaveformPanel` and `SpectrogramPanel` in the same package.
- Several optimizations when editing `Performance` names, adding and removing `Performance` or `Audio` objects from and to the project in order to reduce update traffic between the widgets and the re-rendering of performances for overlay display in the audio widget.
- Added button "Align Frequencies with MIDI Pitches" to the spectrogram specs. These set the min. and max frequency of the spectrogram. Use these to align it vertically with the piano roll.
- Added combobox and sub-class `PartChooserItem` to class `mpmToolbox.gui.audio.AudioDocumentData` to choose the musical part or select all parts to be displayed by the piano roll overlay.
  • Loading branch information
@axelberndt
axelberndt committed Dec 21, 2021
1 parent a271f6c commit c1b38e0
Show file tree
Hide file tree
Showing 13 changed files with 850 additions and 142 deletions.
9 changes: 6 additions & 3 deletions history.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,19 @@


#### v0.1.7
- Little edit in method `mpmToolbox.gui.score.ScoreDisplayPanel.mouseMoved()` that changes the mouse cursor to hand symbol when moved to a clickable overlay element.
- Fixed potential division-by-zero bug in `mpmToolbox.gui.syncPlayer.SyncPlayer.PlaybackRunnable.run()`.
- Reorganized some classes, i.e., classes `Score`, `ScoreNote`, `ScorePage` moved from package `mpmToolbox.gui.score` to package `mpmToolbox.projectData.score`.
- Added "Hide Overlay" button to the score widget (classes `mpmToolbox.gui.score.ScoreDocumentData` and `ScoreDisplayPanel`) that allows to show the score without the overlays.
- Addition to the spectrogram context menu to switch between normalized and non-normalized display.
- New method `getPart()` in classes `mpmToolbox.gui.msmTree.MsmTreeNode` and `mpmToolbox.gui.mpmTree.MpmTreeNode` to retrieve the MSM `part` element that the node belongs to.
- New package `mpmToolbox.supplementary.avlTree` that implements AVL Tree data structure.
- New package `mpmToolbox.projectData.alignment` with several new classes that serve to associate measurements in audio recordings with MSM data.
- New package `mpmToolbox.supplementary.avlTree` that implements the AVL Tree data structure.
- New package `mpmToolbox.projectData.alignment` with several new classes that serve to associate measurements in audio recordings with MSM data, display them as piano roll and interact with it.
- Alignment data is stored in `mpr` project files.
- Added class `mpmToolbox.gui.audio.PianoRollPanel` which is also the basis for the classes `WaveformPanel` and `SpectrogramPanel` in the same package.
- Several optimizations when editing `Performance` names, adding and removing `Performance` or `Audio` objects from and to the project in order to reduce update traffic between the widgets and the re-rendering of performances for overlay display in the audio widget.
- Added buttons "Min. MIDI Pitch" and "Max. MIDI Pitch" to the spectrogram specs. These set the min. and max frequency of the spectrogram. Use these to align it with the piano roll.
- Added button "Align Frequencies with MIDI Pitches" to the spectrogram specs. These set the min. and max frequency of the spectrogram. Use these to align it vertically with the piano roll.
- Added combobox and sub-class `PartChooserItem` to class `mpmToolbox.gui.audio.AudioDocumentData` to choose the musical part or select all parts to be displayed by the piano roll overlay.


#### v0.1.6
Expand Down
177 changes: 154 additions & 23 deletions src/mpmToolbox/gui/audio/AudioDocumentData.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,21 @@
import com.alee.api.data.Orientation;
import com.alee.extended.split.WebMultiSplitPane;
import com.alee.extended.tab.DocumentData;
import com.alee.laf.button.WebButton;
import com.alee.laf.combobox.WebComboBox;
import com.alee.laf.panel.WebPanel;
import meico.mei.Helper;
import meico.mpm.elements.Performance;
import meico.supplementary.KeyValue;
import mpmToolbox.gui.ProjectPane;
import mpmToolbox.gui.Settings;
import mpmToolbox.projectData.Audio;
import mpmToolbox.projectData.alignment.Alignment;
import mpmToolbox.supplementary.Tools;
import nu.xom.Element;

import java.awt.*;
import java.awt.event.ItemEvent;
import java.awt.event.MouseEvent;
import java.awt.event.MouseWheelEvent;

Expand All @@ -25,14 +32,18 @@ public class AudioDocumentData extends DocumentData<WebPanel> {

private final WaveformPanel waveform;
private final SpectrogramPanel spectrogram;
private final PianoRollPanel pianoRoll;
// private final PianoRollPanel pianoRoll;

private int channelNumber = -1; // index of the waveform/channel to be rendered to image; -1 means all channels
private int leftmostSample = -1; // index of the first sample to be rendered to image
private int rightmostSample = -1; // index of the last sample to be rendered to image

private Alignment alignment; // this is the alignment with the piano roll overlay used in the sub-panels

private final WebComboBox partChooser = new WebComboBox(); // with this combobox the user can select whether all musical part or only on individual part should be displayed in the piano roll overlay
// private final WebSwitch globalLocalSwitch = new WebSwitch();
private final WebButton resetAlignment = new WebButton("Reset Alignment");

/**
* constructor
* @param parent
Expand All @@ -41,18 +52,107 @@ public AudioDocumentData(@NotNull ProjectPane parent) {
super("Audio", "Audio", null);
this.parent = parent;

this.makePartChooser();
// this.makeGlobalLocalSwitch();
this.makeResetButton();

this.waveform = new WaveformPanel(this);
this.spectrogram = new SpectrogramPanel(this);
this.pianoRoll = new PianoRollPanel(this);
// this.pianoRoll = new PianoRollPanel(this);

this.setComponent(this.audioPanel);
this.setClosable(false);

this.updateAudio(false);
this.updateAlignment(false);
this.updateAudioTools();

this.draw();
}

/**
* helper method for the constructor; it creates the contents of the part chooser
*/
private void makePartChooser() {
this.partChooser.setPadding(Settings.paddingInDialogs);
this.partChooser.setToolTip("Select the part to be displayed in the piano roll overlay.");

this.partChooser.addItem(new PartChooserItem("All parts", null));

for (Element partElt : this.getParent().getMsm().getParts()) {
int number = Integer.parseInt(Helper.getAttributeValue("number", partElt));
String name = "Part " + number + " " + Helper.getAttributeValue("name", partElt);
this.partChooser.addItem(new PartChooserItem(name, number));
}

this.partChooser.addItemListener(itemEvent -> {
if (itemEvent.getStateChange() == ItemEvent.SELECTED) {
this.repaintAllComponents();
}
});
}

// /**
// * helper method for the constructor; it creates the switch between local and global editing mode
// */
// private void makeGlobalLocalSwitch() {
// WebLabel global = new WebLabel("Global");
// global.setPadding(Settings.paddingInDialogs);
//
// WebLabel local = new WebLabel("Part");
// local.setPadding(Settings.paddingInDialogs);
//
// this.globalLocalSwitch.setSwitchComponents(global, local);
//
// WebLabel gripperLabel = new WebLabel("Edit");
// gripperLabel.setPadding(Settings.paddingInDialogs);
// gripperLabel.setHorizontalAlignment(WebLabel.CENTER);
// this.globalLocalSwitch.getGripper().add(gripperLabel);
//
// this.globalLocalSwitch.setSelected(true);
// }

/**
* define the button to reset the alignment
*/
private void makeResetButton() {
this.resetAlignment.setPadding(Settings.paddingInDialogs);
this.resetAlignment.addActionListener(actionEvent -> {
this.alignment.reset();

// in case of audio alignment being reset, scale the initial alignment to the milliseconds length of the audio; so all notes are visible and in a good starting position
if (this.getParent().getSyncPlayer().isAudioAlignmentSelected()) {
Audio audio = this.getParent().getSyncPlayer().getSelectedAudio();
double milliseconds = ((double) audio.getNumberOfSamples() / audio.getFrameRate()) * 1000.0;
this.alignment.scaleOverallTiming(milliseconds);
}

this.alignment.recomputePianoRoll();
this.repaintAllComponents();
});
}

/**
* this enables or disables the part chooser according to whether there is a performance selected in the syncPlayer
*/
public void updateAudioTools() {
this.partChooser.setEnabled(this.alignment != null);
this.resetAlignment.setEnabled(this.alignment != null);
}

/**
* get the number of the selected part
* @return the number or null if all parts are selected
*/
protected Integer getPianoRollPartNumber() {
if (this.partChooser.getSelectedItem() == null)
return null;

return ((PartChooserItem) this.partChooser.getSelectedItem()).getValue();
}



/**
* this draws the content of the audio analysis frame
*/
Expand All @@ -62,11 +162,18 @@ private void draw() {
splitPane.setContinuousLayout(true); // when the divider is moved the content is continuously redrawn
splitPane.add(this.waveform);
splitPane.add(this.spectrogram);
splitPane.add(this.pianoRoll);
// splitPane.add(this.pianoRoll); // TODO: this pane will be used to display and edit timing curves

GridBagLayout gridBagLayout = (GridBagLayout) this.audioPanel.getLayout();
Tools.addComponentToGridBagLayout(this.audioPanel, gridBagLayout, splitPane, 0, 0, 1, 1, 1.0, 1.0, 0, 0, GridBagConstraints.BOTH, GridBagConstraints.CENTER);
// Tools.addComponentToGridBagLayout(this.audioPanel, gridBagLayout, new WebLabel("Buttons go here"), 0, 1, 1, 1, 1.0, 0.0, 0, 0, GridBagConstraints.NONE, GridBagConstraints.SOUTH);

// the panel with the buttons
WebPanel buttonPanel = new WebPanel(new GridBagLayout());
GridBagLayout buttonLayout = (GridBagLayout) buttonPanel.getLayout();
Tools.addComponentToGridBagLayout(buttonPanel, buttonLayout, this.partChooser, 0, 1, 1, 1, 1.0, 0.0, 0, 0, GridBagConstraints.NONE, GridBagConstraints.CENTER);
// Tools.addComponentToGridBagLayout(buttonPanel, buttonLayout, this.globalLocalSwitch, 1, 1, 1, 1, 1.0, 0.0, 0, 0, GridBagConstraints.NONE, GridBagConstraints.CENTER);
Tools.addComponentToGridBagLayout(buttonPanel, buttonLayout, this.resetAlignment, 2, 1, 1, 1, 1.0, 0.0, 0, 0, GridBagConstraints.NONE, GridBagConstraints.CENTER);
Tools.addComponentToGridBagLayout(this.audioPanel, gridBagLayout, buttonPanel, 0, 1, 1, 1, 1.0, 0.0, 0, 0, GridBagConstraints.NONE, GridBagConstraints.CENTER);
}

/**
Expand All @@ -85,13 +192,13 @@ protected SpectrogramPanel getSpectrogramPanel() {
return this.spectrogram;
}

/**
* a getter for the piano roll panel
* @return
*/
protected PianoRollPanel getPianoRollPanel() {
return this.pianoRoll;
}
// /**
// * a getter for the piano roll panel
// * @return
// */
// protected PianoRollPanel getPianoRollPanel() {
// return this.pianoRoll;
// }

/**
* The sequence at which the child components update their visualizations is important.
Expand All @@ -100,7 +207,7 @@ protected PianoRollPanel getPianoRollPanel() {
protected void repaintAllComponents() {
this.waveform.repaint();
this.spectrogram.repaint();
this.pianoRoll.repaint();
// this.pianoRoll.repaint();
}

/**
Expand All @@ -110,7 +217,7 @@ protected void repaintAllComponents() {
protected void communicateMouseEventToAllComponents(MouseEvent e) {
this.waveform.setMousePosition(e);
this.spectrogram.setMousePosition(e);
this.pianoRoll.setMousePosition(e);
// this.pianoRoll.setMousePosition(e);
}

/**
Expand All @@ -133,7 +240,7 @@ public void updateAlignment(boolean doRepaint) {
}

if (doRepaint)
this.repaintAllComponents(); // repaint of all components
this.repaintAllComponents();
}

/**
Expand All @@ -160,7 +267,7 @@ public void updateAudio(boolean doRepaint) {
this.spectrogram.setAudio();

if (doRepaint)
this.repaintAllComponents(); // repaint of all components
this.repaintAllComponents();
}

/**
Expand Down Expand Up @@ -247,7 +354,7 @@ protected void setRightmostSample(int rightmostSample) {
* this shifts the visualisations left or right by the specified offset
* @param sampleOffset offset in samples
*/
protected void scroll(double sampleOffset) {
private void scroll(double sampleOffset) {
if (this.parent.getAudio() == null)
return;

Expand All @@ -261,22 +368,22 @@ protected void scroll(double sampleOffset) {
this.setRightmostSample((int) (this.rightmostSample + sampleOffset));
this.spectrogram.updateScroll();

this.repaintAllComponents(); // triggers repaint for all components
this.repaintAllComponents();
}

/**
* this is used when the visualisations are zoomed
* @param pivotSample
* @param zoomFactor
*/
protected void zoom(int pivotSample, double zoomFactor) {
private void zoom(int pivotSample, double zoomFactor) {
if (zoomFactor == 0.0)
return;

if (zoomFactor < 0.0) { // zoom in
int leftmostSample = pivotSample - (int) ((pivotSample - this.leftmostSample) * zoomFactor);
int rightmostSample = (int) ((this.rightmostSample - pivotSample) * zoomFactor) + pivotSample;
if ((rightmostSample - leftmostSample) > 1) { // make sure there are at least two samples to be drawn, if we zoom too far in, left==right, we cannot zoom out again
if ((rightmostSample - leftmostSample) > 1) { // make sure there are at least two samples to be drawn, if we zoom too far in, left==right, we cannot zoom out again
this.setLeftmostSample(leftmostSample);
this.setRightmostSample(rightmostSample);
}
Expand All @@ -292,14 +399,14 @@ else if (zoomFactor > 0.0) { // zoom out

this.spectrogram.updateZoom();

this.repaintAllComponents(); // triggers repaint for all components
this.repaintAllComponents();
}

/**
* process a mouse drag event; to be invoked by sub-panels WaveformPanel, SpectrogramPanel
* @param e
*/
protected void mouseDragged(MouseEvent e) {
protected void scroll(MouseEvent e) {
if (this.getWaveformPanel().mousePosition == null) {
this.getWaveformPanel().setMousePosition(e);
return;
Expand All @@ -311,7 +418,6 @@ protected void mouseDragged(MouseEvent e) {

this.communicateMouseEventToAllComponents(e);
this.scroll(sampleOffset);

}

/**
Expand All @@ -322,10 +428,35 @@ protected void mouseWheelMoved(MouseWheelEvent e){
if ((this.getAudio() == null) || (e.getWheelRotation() == 0))
return;

int pivotSample = this.getWaveformPanel().getSampleIndex(e.getPoint());
int pivotSample = this.getWaveformPanel().getSampleIndex(e.getPoint().getX());
double zoomFactor = (e.getWheelRotation() < 0) ? 0.9 : 1.1;

this.communicateMouseEventToAllComponents(e);
this.zoom(pivotSample, zoomFactor);
}

/**
* This class represents an item in the part chooser combobox of the audio analysis widget.
* @author Axel Berndt
*/
private static class PartChooserItem extends KeyValue<String, Integer> {
/**
* This constructor creates a part chooser item with the specified name key and part number.
* @param string
*/
private PartChooserItem(String string, Integer partNumber) {
super(string, partNumber);
}

/**
* All combobox items require this method. The override here makes sure that the string being returned
* is the performance's name instead of some Java Object ID.
* @return
*/
@Override
public String toString() {
return this.getKey();
}
}

}
Loading

0 comments on commit c1b38e0

Please sign in to comment.