/**
 * NETWORKING WITH UDP
 *
 * TutorialOne_04_Network is a copy of TutorialOne_03_Drawing with networking features added.
 * Refer to the notes for TutorialOne_03_Drawing for information about drawing and audio synthesis
 * in TutorialOne_04_Network. Here, we're going to explain just the networking features.
 *
 * TutorialOne_04_Network is designed to work with a Max patch, simpleAudioIO.maxpat, which is
 * included in the example code for PixelAudio. The simpleAudioIO patch is designed to process
 * audio output from TutorialOne_04_Network and to respond to UDP messages from TutorialOne_04_Network.
 * We'll discuss the audio setup first, for MacOS (something similar can be done in the Windows OS).
 * You can use the audio signal flow without using UDP. It's handy for situations where you want to
 * add a layer of audio processing and it will function with many audio applications besides Max.
 * Next, we'll discuss UDP network messaging. Finally we'll look at the PANetworkClientINF interface
 * and its implementation in NetworkDelegate, an inner class of TutorialOne_04_Network.
 *
 * Audio Setup with Blackhole in MacOS
 * ===================================
 *
 * Using the System Settings for Sound in MacOS and the BlackHole audio routing tool
 * (https://existential.audio/blackhole/), you can set up a wide range of inputs and outputs,
 * depending on your system software and hardware. Here's how to route between this sketch
 * and Max.
 *
 * Open simpleAudioIO.maxpat in Max
 *   1. Open MacOS System Settings control panel and select the Sound tab.
 *   2. In the Input and Output section, select BlackHole 16ch as the Output.
 *   3. In the Max Audio Status control panel, set Input Device to BlackHole 16ch.
 *   4. If you have external audio hardware, set that to your output.
 *   5. Turn on audio processing in Max.
 *   6. Run TutorialOne_04_Network and play some sounds.
 *
 * UDP: User Datagram Protocol
 * ===========================
 *
 * UDP (User Datagram Protocol) is a network communications protocol that is apt for
 * sending relatively short, real-time messages over a network. In Processing, it is implemented
 * with external libraries. I'm using the oscP5 library by Andreas Schlegel, which is included
 * in Processing's Contributed Libraries. You can install it with the Sketch -> Import Library...
 * -> Manage Libraries... menu command in Processing. UDP requires an IP address and a port
 * number to enable communications. In this example, we use 127.0.0.1 as the IP address.
 * The address 127.0.0.1 is the standard address for IPv4 loopback traffic, i.e., for services
 * running on the host computer. If you are communicating with a remote computer, you will need
 * the IP address for that computer and your own, either on the local area network or on the
 * internet, if the addresses are static. We're using ports 7400 and 7401. TutorialOne_04_Network
 * sends messages out on port 7400. The Max application simpleAudioIO receives messages on 7400,
 * using the udpreceive objects. It sends messages to TutorialOne_04_Network via IP address
 * 127.0.0.1 on port 7401 using the udpsend object. TutorialOne_04_Network receives messages
 * over port 7401. See the examples for oscP5 in Processing for more information.
 *
 * PANetworkClientINF and NetworkDelegate
 * ======================================
 *
 * Since UDP communications are independent of the PixelAudio library, they are implemented in the
 * example code, not in the library. The PANetworkClientINF interface is included in the example software.
 * It defines the methods that an implementing class must provide:
 *   1. public PixelAudioMapper getMapper();
 *   2. public int playSample(int samplePos);
 *   3. public void playPoints(ArrayList pts);
 *   4. public void parseKey(char key, int keyCode);
 *   5. public void controlMsg(String control, float val);
 *   6. public PApplet getPApplet();
 *
 * If you are running Processing in an IDE like Eclipse, the PApplet class you create should implement
 * PANetworkClientINF and provide these methods. In Processing, the PApplet host class doesn't provide
 * a way to implement an interface directly, but you can still provide all the required methods and
 * create a network delegate to handle UDP communications. In TutorialOne_04_Network we use the
 * NetworkDelegate subclass. In Eclipse, NetworkDelegate's constructors require a
 * PANetworkClientINF object as their first argument. In Processing, the constructors should require
 * an instance of your PApplet class, which is indicated with the name of your application.
 *
 *   -- Eclipse:    NetworkDelegate(PANetworkClientINF app, String remoteFromAddr, String remoteToAddr, int inPort, int outPort)
 *   -- Processing: NetworkDelegate(TutorialOne_04_Network app, String remoteFromAddr, String remoteToAddr, int inPort, int outPort)
 *
 * Messages and Methods
 * ====================
 *
 * To find the points at which networking code is implemented in methods in this sketch,
 * search for "// *****>>> NETWORKING <<<***** //".
 * The NetworkDelegate class provides a number of messaging methods, each prefaced with "oscSend".
 * For receiving remote messages, it calls oscP5's "plug" method (in initOscPlugs()) to link
 * incoming messages to local methods. In addition, it provides some boolean toggles, such as
 * isNetSendDrawingPoints, to turn features on or off at various points in the
 * TutorialOne_04_Network code--if you aren't going to use data, don't send it.
 * In this sketch, we concentrate on:
 *   -- sending mouse coordinates and audio buffer position to simpleAudioIO
 *   -- turning Max reverb on or off with the ']' or '[' keys (we've left '{' and '}' open in simpleAudioIO)
 *   -- changing Max reverb settings with 'q' or 'Q' keys
 *
 * See NetworkDelegate comments for more information. There is quite a lot more that can be accomplished
 * with UDP communications. In my collaborations with composer Christopher Walczak, it's been very useful
 * in performances where we use mouse position to control objects in Max or trigger brushstrokes to send
 * control values to Max. It's not an integral part of PixelAudio, but I will post useful examples as
 * I develop them.
 *
 * Here are the key commands for this sketch:
 *
 * Press UP ARROW to increase audio gain by 3 dB.
 * Press DOWN ARROW to decrease audio gain by 3 dB.
 * Press RIGHT ARROW to increase Sampler audio gain by 3 dB.
 * Press LEFT ARROW to decrease Sampler audio gain by 3 dB.
 * Press SHIFT-RIGHT ARROW to increase Granular audio gain by 3 dB.
 * Press SHIFT-LEFT ARROW to decrease Granular audio gain by 3 dB.
 * Press ' ' (spacebar) to trigger a brush if we're hovering over a brush, otherwise trigger a point event.
 * Press '1' to set brushstroke under cursor to PathMode ALL_POINTS.
 * Press '2' to set brushstroke under cursor to PathMode REDUCED_POINTS.
 * Press '3' to set brushstroke under cursor to PathMode CURVE_POINTS.
 * Press 't' to set brushstroke under cursor to use Sampler or Granular synth.
 * Press 'f' to set brushstroke under cursor to FIXED hop mode for granular events.
 * Press 'g' to set brushstroke under cursor to GESTURE hop mode for granular events.
 * Press 'y' to select granular or sampler synth for click response.
 * Press 'r' to select long or short grain duration .
 * Press 'b' to toggle between long burst grains and short burst grains.
 * Press 'p' to jitter the pitch of granular gestures.
 * Press 'P' to adjust pitch of current brushstroke.
 * Press '.' to increment epsilon value of current brush (reduced points decrease).
 * Press ',' to decrement epsilon value of current brush (reduced points increase).
 * Press '>' to increment curve steps value of current brush.
 * Press '<' to decrement curve steps value of current brush.
 * Press 'm' to change the Sampler noise reduction profile.
 * Press 'e' to change the envelope we're using .
 * Press 'E' to toggle whether we adjust envelope duration in relation to gesture duration.
 * Press 'a' to  start or stop animation (bitmap rotation along the signal path).
 * Press 'd' to toggle play audio on new brush .
 * Press 'c' to apply color from image file to display image.
 * Press 'k' to apply the hue and saturation in the colors array to mapImage (not to baseImage).
 * Press 'K' to apply hue and saturation in colors to baseImage and mapImage.
 * Press 'j' to turn audio and image blending on or off.
 * Press 'n' to normalize the audio buffer to -6.0dB.
 * Press 'L' or 'l' to determine whether to load a new file to both image and audio or not.
 * Press 'o' or 'O' to open an audio or image file and load to image or audio buffer or both.
 * Press 's' to save display image to a PNG file.
 * Press 'S' to save audio buffer to a .wav file.
 * Press 'w' to write the image HSB Brightness channel to the audio buffer as transcoded sample values .
 * Press 'W' to write the audio buffer samples to the image as color values.
 * Press 'x' to delete the current active brush shape or the oldest brush shape.
 * Press 'X' to delete the most recent brush shape.
 * Press 'u' to mute audio.
 * Press 'V' to record a video.
 * Press ']' to send UDP message to Max (simpleAudioIO.maxpat): reverb ON.
 * Press '[' to send UDP message to Max (simpleAudioIO.maxpat): reverb OFF.
 * Press '}' to send UDP message to Max (simpleAudioIO.maxpat): unused.
 * Press '{' to send UDP message to Max (simpleAudioIO.maxpat): unused.
 * Press 'q' to send UDP message to Max (simpleAudioIO.maxpat): small reverb settings.
 * Press 'Q' to send UDP message to Max (simpleAudioIO.maxpat): big reverb settings.
 * Press 'h' or 'H' to show help message in the console.
 *
 */

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;

import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.UnsupportedAudioFileException;

//Mama's ever-lovin' blue-eyed PixelAudio library
import net.paulhertz.pixelaudio.*;
import net.paulhertz.pixelaudio.PixelAudioMapper.ChannelNames;
import net.paulhertz.pixelaudio.curves.*;
import net.paulhertz.pixelaudio.schedule.*;
import netP5.NetAddress;
import oscP5.OscMessage;
import oscP5.OscP5;
import net.paulhertz.pixelaudio.granular.*;
import net.paulhertz.pixelaudio.sampler.*;

//audio library
import ddf.minim.*;

//video export library
import com.hamoid.*;


/* ------------------------------------------------------------------ */
/*                       PIXELAUDIO VARIABLES                         */
/* ------------------------------------------------------------------ */

PixelAudio pixelaudio;     // our shiny new library
MultiGen multigen;         // a PixelMapGen that links together multiple PixelMapGens
int genWidth = 512;        // width of multigen PixelMapGens
int genHeight = 512;       // height of  multigen PixelMapGens
PixelAudioMapper mapper;   // object for reading, writing, and transcoding audio and image data
int mapSize;               // size of the display bitmap, audio signal, wavesynth pixel array, mapper arrays, etc.
// baseImage is a reference image that generally should not be changed except when you load a new file
PImage baseImage;          // unchanging source image
// mapImage can change, and often does so with reference to the stable baseImage, for example when animating
PImage mapImage;           // image for display, may be animated
PixelAudioMapper.ChannelNames chan = ChannelNames.ALL;    // also try ChannelNames.L (HSB brightness channel)
int[] spectrum;              // array of spectral colors


/* ------------------------------------------------------------------ */
/*                        FILE I/O VARIABLES                          */
/* ------------------------------------------------------------------ */

// audio file
File audioFile;
String audioFilePath;
String audioFileName;
String audioFileTag;
int audioFileLength;

// image file
File imageFile;
String imageFilePath;
String imageFileName;
String imageFileTag;
int imageFileWidth;
int imageFileHeight;

boolean isLoadToBoth = true;    // if true, load newly opened file both to audio and to video
boolean isBlending = false;     // flags blending of newly opened audio or image file with display/buffer

// system-specific path to example files data
// in Processing, for PixelAudio Tutorial examples, use this in setup(): 
// daPath = sketchPath("") + "../../examples_data/";
String daPath = "/Users/paulhz/Code/Workspace/PixelAudio/examples/examples_data/";


/* ------------------------------------------------------------------ */
/*                          AUDIO VARIABLES                           */
/* ------------------------------------------------------------------ */

/*
 * Audio playback support is added with the audio variables and audio methods.
 * You will also need the ADSRParams, PASamplerInstrument, PASamplerInstrumentPool
 * and TimedLocation classes. In setup(), call initAudio(), then add
 * a mousePressed() method that calls audioMousePressed(mouseX, mouseY)
 * and call runTimeArray() in your draw method.
 */

// ====== Minim audio library ====== //

Minim minim;          // library that handles audio
AudioOutput audioOut;      // line out to sound hardware
float outputGain = 0.0f;        // gain setting for audio output, decibels
boolean isBufferStale = false;  // flags that audioBuffer needs to be reset, not used in TutorialOneDrawing
float sampleRate = 48000;       // target audio engine rate used to configure audioOut
float fileSampleRate;           // sample rate of most recently opened file (before resampling)
float bufferSampleRate;         // sample rate of playBuffer, usually == audioOut.sampleRate()
float[] audioSignal;      // the audio signal as an array of floats
MultiChannelBuffer playBuffer;  // a buffer for playing the audio signal
int samplePos;                  // an index into the audio signal, selected by a mouse click on the display image
int audioLength;        // length of the audioSignal, same as the number of pixels in the display image

// ====== Sampler Synth ====== //

// Sampler Instrument duration and envelopes
final int noteDuration = 1000;  // average sample synth note duration, milliseconds, for reference
int envDuration;                // current envelope duration
boolean isAdjustEnvelope = true;    // if true, adjust envelope duration based on gesture duration
String[] envPresets = {"Soft", "Percussion", "Pad", "default"};    // envelope presets
int presetIndex = envPresets.length - 1;    // current preset index
float overlapFactor = 3.0f;     // envelope overlap
int envMinDurationMs = 40;        // min envelope duration, milliseconds
int envMaxDurationMs = 1280;      // max envelope duration, milliseconds

// Sampler Instrument setup
int samplelen;                  // calculated sample synth note length, samples
float samplerGain = AudioUtility.dbToLinear(-3.0f);    // linear gain for Sampler gesture event
float samplerPointGain = 0.75f; // linear gain for point events with the Sampler instrument
boolean isMuted = false;
PASamplerInstrumentPool pool;   // an allocation pool of PASamplerInstruments
int poolSize = 8;               // number of sampler instruments for polyphony
int samplerMaxVoices = 256;     // number of voices for each sampler instrument

// ADSR and its parameters
float maxAmplitude = 1.0f;          // set envelopes to 1.0f amplitude, then scale later with audio instrument gain
// ADSR envelope with maximum amplitude, attack Time, decay time, sustain level, and release time
ADSRParams defaultEnv = new ADSRParams(maxAmplitude, 0.2f, 0.1f, maxAmplitude * 0.75f, 0.5f);
// ADSRParams granularEnv = new ADSRParams(maxAmplitude, 0.125f, 0.125f, 0, 0);
float pitchScaling = 1.0f;          // factor for changing pitch
float defaultPitchScaling = 1.0f;   // pitch scaling factor
float lowPitchScaling = 0.5f;       // low pitch scaling, 0.5 => one octave down
float highPitchScaling = 2.0f;      // high pitch scaling, 2.0 => one octave up

// toggle for sampler events with a "granular" envelope
boolean useGranularSynth = false;   // which synth do we use for point events (mouse clicks and the like)
int granEnvDuration = 120;          // envelope duration in ms

// ====== Granular Synth ====== //

public float[] granSignal;                  // buffer source for granular (defaults to audioSignal)
public PAGranularInstrument gSynth;         // granular synthesis instrument
public PAGranularInstrumentDirector gDir;   // director of granular events
public float granularGain = AudioUtility.dbToLinear(-3.0f);    // linear gain for a granular gesture event
public float granularPointGain = 1.0f;      // linear gain for a granular point event
// parameters for granular synthesis
boolean useShortGrain = false;              // default to short grains, if true
int longSample = 4096;                      // number of samples for a moderately long grain
int shortSample = 512;                      // number of samples for a relatively short grain
int granSamples = useShortGrain ? shortSample : longSample;    // number of samples in a grain window
int hopSamples = granSamples/4;             // number of samples between each grain in a granular burst
GestureGranularParams gParamsGesture;       // granular parameters when hopMode == HopMode.GESTURE
GestureGranularParams gParamsFixed;         // granular parameters when hopMode == HopMode.FIXED
public int curveSteps = 8;                  // number of steps in curve representation of a brushstroke path
public int gMaxVoices = 256;                // number of voices managed by a PAGranularInstrument (gSynth)
boolean useLongBursts = true;               // controls the number of burst grains in a point event or gesture event
int maxBurstGrains = 8;
int minBurstGrains = 1;
int burstGrains = useLongBursts ? maxBurstGrains : minBurstGrains;
boolean usePitchedGrains = false;           // detune a granular gesture, if true


/* ------------------------------------------------------------------ */
/*                   ANIMATION AND VIDEO VARIABLES                    */
/* ------------------------------------------------------------------ */

// animation consists of rotating pixels / audio samples along the mapper's "signal path"
// it's an effect that looks glitchy or hypnotic, depending on your source material
// you an also use it to shift audio samples with respect to brushstrokes

int shift = 1024;                    // number of pixels to shift the animation
int totalShift = 0;                  // cumulative shift
boolean isAnimating = false;         // do we run animation or not?
boolean oldIsAnimating;              // keep track of animation state when opening a file
// animation variables for video recording
int videoSteps = 768;                // how many steps in an animation loop
boolean isRecordingVideo = false;    // are we recording? (only if we are animating)
int videoFrameRate = 24;             // frames per second
int step;                            // number of current step in animation loop
VideoExport videx;                   // hamoid library class for video export (requires ffmpeg)


/* ------------------------------------------------------------------ */
/*                         DRAWING VARIABLES                          */
/* ------------------------------------------------------------------ */

/*
 * Drawing uses classes in the package net.paulhertz.pixelaudio.curves to store gesture data
 * and draw Bezier curves and lines. It also provides very basic brushstroke modeling code.
 * Unlike most of the code in PixelAudio, which avoids dependencies on Processing, the
 * curves.* classes interface with Processing to draw to PApplets and PGraphics instances.
 * See the CurveMaker class for details of how drawing works. Because CurveMaker also stores
 * timing data for each point it stores, it can reproduce gestures as audio events.
 */

PACurveMaker curveMaker;             // a PACurveMaker instance for use when drawing a new brushstroke
public boolean isDrawMode = false;   // flag that drawing is enabled
public float epsilon = 4.0f;         // variable that controls point reduction in PACurveMaker

public PVector currentPoint = new PVector(-1, -1);  // point to add to brushstroke points
public ArrayList<PVector> allPoints; // in-progress brushstroke points
public ArrayList<Integer> allTimes;  // in-progress brushstroke times
public int startTime;                // in-progress brushstroke start time

public int dragColor = color(233, 199, 89, 128);    // color of line while dragging the mouse
public float dragWeight = 8.0f;                     // line weight

public int polySteps = 5;                           // polygon steps for brush, used by PACurveMaker

public ArrayList<AudioBrushLite> brushes;           // a list of AudioBrushLite, a class that wraps a PACurveMaker and a BrushConfig
// Hover + selection
public AudioBrushLite hoverBrush = null;            // brush the mouse is over, if there is one
public int hoverIndex = -1;                         // index of the brush in the brushes list
public AudioBrushLite activeBrush = null;           // “selected” brush (optional but useful)

// ====== Visual styling for brushstrokes ====== //

int readyBrushColor = color(34, 89, 55, 233);       // color for a brushstroke when available to be clicked
int hoverBrushColor = color(144, 89, 55, 233);      // color for brushstroke under the cursor
int selectedBrushColor = color(233, 199, 144, 233); // color for the selected (clicked or currently active for editing) brushstroke

int readyBrushColor1 = color(34, 55, 89, 178);    // color for a brushstroke when available to be clicked
int hoverBrushColor1 = color(55, 89, 144, 178);     // color for brushstroke under the cursor
int selectedBrushColor1 = color(199, 123, 55, 233);  // color for the selected (clicked or currently active for editing) brushstroke
int readyBrushColor2 = color(55, 34, 89, 178);    // color for a brushstroke when available to be clicked
int hoverBrushColor2 = color(89, 55, 144, 178);     // color for brushstroke under the cursor
int selectedBrushColor2 = color(123, 199, 55, 233);  // color for the selected (clicked or currently active for editing) brushstroke

int lineColor = color(233, 220, 178);               // brushstroke interior line color
int dimLineColor = color(178, 168, 136);            // brushstroke interior line color, dimmed
int circleColor = color(233, 178, 144);             // brushstroke interior point color
int dimCircleColor = color(178, 168, 136);          // brushstroke interiro point color, dimmed

// TimedLocation events
public ArrayList<SamplerBrushEvent> samplerTimeLocs;    // a list of timed events for Sampler brushes
ArrayList<TimedLocation> pointTimeLocs;             // a list of timed events for mouse clicks
ArrayList<TimedLocation> grainTimeLocs;             // a list of timed events for Granular brushes
int animatedCircleColor = color(233, 220, 199);     // color for animated circles when playing audio

PABoundsPolicy.PABoundaryMode boundaryMode = PABoundsPolicy.PABoundaryMode.CLIP;
PABoundsPolicy boundsPolicy;

/* ------------------------------------------------------------------ */
/*                       INTERACTION VARIABLES                        */
/* ------------------------------------------------------------------ */

// ====== Minimal gesture + audio synthesis configuration ====== //
// GesturePlayground example provides a complete configuration model with the GestureGranularConfig class

public enum PathMode {
  ALL_POINTS, REDUCED_POINTS, CURVE_POINTS
}    // select model for brush shape in PACurveMaker
public enum BrushOutput {
  SAMPLER, GRANULAR
}                        // select audio synthesis model
public enum HopMode {
  GESTURE, FIXED
}                               // select audio gesture sequence timing model

/**
 * Defines the curve drawing model for a brush using PACurveMaker.
 */
public static final class BrushConfig {
  /** PathMode for drawing */
  public PathMode pathMode = PathMode.ALL_POINTS;
  /** Point reduction parameter */
  public float rdpEpsilon = 4.0f;
  /** Number of polygon segments in a curve */
  public int curveSteps = 8;
  // leave room for future compatibility
}

/**
 * A light-weight version of the AudioBrush class for combining gestures and
 * audio synthesis. AudioBrushLite combines PACurveMaker gesture and curve
 * modeling with basic audio synthesis parameters. AudioBrush, introduced in
 * TutorialOne_05_GesturePlayground, exposes most of the parameters used for
 * granular synthesis in PixelAudio, along with a many of the brush and gesture
 * settings available in PACurveMaker.
 */
public static final class AudioBrushLite {
  /** PACurveMaker instance stores point and time data, provides drawing and scheduling methods  */
  private final PACurveMaker curve;
  /** Configuration variables for drawing */
  private final BrushConfig cfg;
  /** Audio output variable */
  private BrushOutput output;
  /** For granular instrument, timing model */
  private HopMode hopMode;
  /** Pitch scaling */
  private float pitchRatio = 1.0f;

  public AudioBrushLite(PACurveMaker curve, BrushConfig cfg, BrushOutput output, HopMode hopMode) {
    this.curve = curve;
    this.cfg = cfg;
    this.output = output;
    this.hopMode = hopMode;
    this.pitchRatio = 1.0f;
  }
  public PACurveMaker curve() {
    return curve;
  }
  public BrushConfig cfg() {
    return cfg;
  }
  public BrushOutput output() {
    return output;
  }
  public void setOutput(BrushOutput out) {
    this.output = out;
  }
  public float pitchRatio() {
    return pitchRatio;
  }
  public void setPitchRatio(float newRatio) {
    this.pitchRatio = newRatio;
  }
  public HopMode hopMode() {
    return hopMode;
  }
  public void setHopMode(HopMode hop) {
    this.hopMode = hop;
  }
}

/**
 *  A class to store a hit-test result, recording an AudioBrushLite instance and its index in brush list.
 */
public static final class BrushHit {
  public final AudioBrushLite brush;
  public final int index;
  public BrushHit(AudioBrushLite brush, int index) {
    this.brush = brush;
    this.index = index;
  }
}

// *****>>> NETWORKING <<<***** //
NetworkDelegate nd;
boolean isUseNetworkDelegate = false;
boolean isNetSendDrawingPoints = true;
boolean isNetSendOutsideBrushPoints = true;
boolean isNetSendBrushTriggers = false;
boolean isNetSendFileInfo = false;
boolean isNetSendGestures = false;

// application settings

boolean doPlayOnNewBrush = true;
boolean shiftIsDown = false;


// ---------------- APPLICATION ---------------- //

public void settings() {
  size(3 * genWidth, 2 * genHeight);
}

public void setup() {
  daPath = sketchPath("") + "../../examples_data/";
  // set a standard animation framerate
  frameRate(24);
  // 1. Initialize our library.
  pixelaudio = new PixelAudio(this);
  // 2. create a PixelMapGen object with dimensions equal to the display window.
  //    the call to "hilbertLoop3x2(genWidth, genHeight)" produces a MultiGen that is 3 * genWidth x 2 * genHeight,
  //    where genWidth == genHeight and genWidth is a power of 2 (a restriction on Hilbert curves)
  //
  // multigen = HilbertGen.hilbertLoop3x2(genWidth, genHeight);
  //
  //   Here's an alternative multigen that is good for visualizing audio location within the image
  //   It reads left to right, top to bottom, like a musical score. I've divided it 12 x 8, but 6 x 4 is good, too.
  //
  multigen = HilbertGen.hilbertRowOrtho(12, 8, width/12, height/8);
  // 3. Initialize a PixelAudioMapper with multigen
  mapper = new PixelAudioMapper(multigen);
  // area of the PixelAudioMapper == number of pixels in display == number of samples in audio buffer
  mapSize = mapper.getSize();
  // initialize the boundary policy for keeping points and indices in bounds
  boundsPolicy = PABoundsPolicy.fromWidthHeight(mapper.getWidth(), mapper.getHeight(), boundaryMode);
  // create an array of rainbow colors with mapSize elements
  spectrum = getColors(mapSize);
  // 4. create image for display
  initImages();
  // 5. set up the audio environment and variables
  initAudio();
  // 6. set up the drawing environment and variables
  initDrawing();
  // 7. set up networking
  // *****>>> NETWORKING <<<***** //
  isUseNetworkDelegate = true;
  initNetwork();
  // 8. output a help message to the console
  showHelp();
  // preload files, parameters will vary with the system where we're running
  preloadFiles(daPath, "Saucer_mixdown.wav");    // handy when debugging, too
}

/**
 * turn off audio processing when we exit
 */
public void stop() {
  if (pool != null) pool.close();
  if (minim != null) minim.stop();
  super.stop();
}

/**
 * Generates an array of rainbow colors using the HSB color space.
 * @param size    the number of entries in the colors array
 * @return an array of RGB colors ordered by hue
 */
public int[] getColors(int size) {
  int[] colorWheel = new int[size]; // an array for our colors
  pushStyle(); // save styles
  colorMode(HSB, colorWheel.length, 100, 100); // create colors in the HSB color space, giving hue a very wide range
  for (int i = 0; i < colorWheel.length; i++) {
    colorWheel[i] = color(i, 50, 70); // fill our array with colors, gradually changing hue
  }
  popStyle(); // restore styles, including the default RGB color space
  return colorWheel;
}

/**
 * Initializes mapImage with the colors array.
 * MapImage handles the color data for mapper and also serves as our display image.
 * BaseImage is intended as a reference image that typically only changes when you load a new image.
 */
public void initImages() {
  mapImage = createImage(width, height, ARGB);
  mapImage.loadPixels();
  mapper.plantPixels(spectrum, mapImage.pixels, 0, mapSize); // load colors to mapImage following signal path
  mapImage.updatePixels();
  baseImage = mapImage.copy();
}

/**
 * Initializes drawing and drawing interaction variables.
 */
public void initDrawing() {
  currentPoint = new PVector(-1, -1);    // used for drawing to the display
  brushes = new ArrayList<>();           // keep track of brushstrokes/gestures
  hoverBrush = null;
  hoverIndex = -1;
  activeBrush = null;
  samplerTimeLocs = new ArrayList<>();   // capture timing data when drawing
}

void initNetwork() {
  // *****>>> NETWORKING <<<***** //
  if (isUseNetworkDelegate) {
    String remoteAddress = "127.0.0.1";
    nd = new NetworkDelegate(this, remoteAddress, remoteAddress, 7401, 7400);
    nd.oscSendClear();
  }
}

/**
 * Preload an audio file using a file path and a filename.
 * @param path        the fully qualified path to the file's directory, ending with a '/'
 * @param filename    the name of the file
 */
public void preloadFiles(String path, String filename) {
  // the audio file we want to open on startup
  File audioSource = new File(path + filename);
  // load the file into audio buffer and Brightness channel of display image (mapImage)
  // if audio is also loaded to the image, will set baseImage to the new image
  fileSelected(audioSource);
  // overlay colors on mapImage
  mapImage.loadPixels();
  applyColor(spectrum, mapImage.pixels, mapper.getImageToSignalLUT());
  mapImage.updatePixels();
  // write mapImage to baseImage
  commitMapImageToBaseImage();
}

public void draw() {
  image(mapImage, 0, 0);    // draw mapImage to the display window
  handleDrawing();          // handle interactive drawing and audio events created by drawing
  if (isAnimating) {        // animation is different from windowed buffer
    animate();            // do some animation, here it's just to rotate mapImage.pixels by shift number of pixels
  }
}

public void animate() {
  stepAnimation();
  renderFrame(step);
}

/**
 * Step through the animation, called by the draw() method.
 * Will also record a frame of video, if we're recording.
 */
public void stepAnimation() {
  if (step >= videoSteps) {
    if (isRecordingVideo) {
      isRecordingVideo = false;
      videx.endMovie();
      isAnimating = oldIsAnimating;
      println("--- Completed video at frame " + videoSteps);
    }
    step = 0;
  } else {
    step += 1;
    if (isRecordingVideo) {
      if (videx == null) {
        println("----->>> start video recording ");
        videx = new VideoExport(this, "TutorialOneVideo.mp4");
        videx.setFrameRate(videoFrameRate);
        videx.startMovie();
      }
      videx.saveFrame();
      println("-- video recording frame " + step + " of " + videoSteps);
    }
  }
}

/**
 * Renders a frame of animation: moving along the signal path, copies baseImage pixels to
 * mapImage pixels, adjusting the index position of the copy using totalShift --
 * i.e. we don't actually rotate the pixels, we just shift the position they're copied to.
 *
 * @param step   current animation step
 */
public void renderFrame(int step) {
  // keep track of how much the pixel array is shifted
  totalShift = PixelAudioMapper.wrap(totalShift + shift, mapSize);
  // write the pixels in rgbSignal to mapImage, following the signal path
  mapImage.loadPixels();
  mapper.copyPixelsAlongPathShifted(baseImage.pixels, mapImage.pixels, totalShift);
  mapImage.updatePixels();
}

/**
 * Handles user's drawing actions: draws previously recorded brushstrokes,
 * tracks and generates interactive animation and audio events.
 */
public void handleDrawing() {
  // 1) draw existing brushes
  drawBrushShapes();
  // 2) update hover state (pure state update, no action)
  updateHover();
  // 3) if in the process of drawing, accumulate points while mouse is held down
  if (isDrawMode) {
    if (mousePressed) {
      addDrawingPoint(clipToWidth(mouseX), clipToHeight(mouseY));
    }
    if (allPoints != null && allPoints.size() > 2) {
      PACurveUtility.lineDraw(this, allPoints, dragColor, dragWeight);
    }
  }
  // 4) run scheduled events
  runSamplerBrushEvents();
  runPointEvents();
  runGrainEvents();
}

/**
 * @return a reference to the brushstroke the mouse is over, or null if there's no brushstroke.
 */
BrushHit findHoverHit() {
  if (brushes == null || brushes.isEmpty()) return null;    // no brushes
  // check from topmost (end) to bottom (start)
  for (int i = brushes.size() - 1; i >= 0; i--) {
    AudioBrushLite b = brushes.get(i);
    if (mouseInPoly(b.curve().getBrushPoly())) {
      return new BrushHit(b, i);
    }
  }
  return null;
}

/**
 * Update the hoverBrush and hoverIndex global variables.
 */
void updateHover() {
  BrushHit hit = findHoverHit();
  if (hit != null) {
    hoverBrush = hit.brush;
    hoverIndex = hit.index;
  } else {
    hoverBrush = null;
    hoverIndex = -1;
  }
}

/**
 * Displays a line of text to the screen, usually in the draw loop. Handy for debugging.
 * typical call: writeToScreen("When does the mind stop and the world begin?", 64, 1000, 24, true);
 *
 * @param msg     message to write
 * @param x       x coordinate
 * @param y       y coordinate
 * @param weight  font weight
 * @param isWhite if true, white text, otherwise, black text
 */
public void writeToScreen(String msg, int x, int y, int weight, boolean isWhite) {
  int fill1 = isWhite? 0 : 255;
  int fill2 = isWhite? 255 : 0;
  pushStyle();
  textSize(weight);
  float tw = textWidth(msg);
  int pad = 4;
  fill(fill1);
  rect(x - pad, y - pad - weight, x + tw + pad, y + weight/2 + pad);
  fill(fill2);
  text(msg, x, y);
  popStyle();
}

/**
 * The built-in mousePressed handler for Processing, clicks are handled in mouseClicked().
 */
public void mousePressed() {
  if (isDrawMode) {
    initAllPoints();    // start capturing points and times for a brushstroke
  }
}

public void mouseDragged() {
  // we don't do anything here, drawing will be handled by the draw() loop
}

public void mouseReleased() {
  if (isDrawMode && allPoints != null) {
    if (allPoints.size() > 2) {
      initCurveMakerAndAddBrush();    // finalize curve drawing into a new PACurveMaker instance, add a brush
    }
    allPoints.clear();                  // clear points accumulated while dragging the mouse
    // mouseClicked() handles actual clicks
  }
}

public void mouseClicked() {
  BrushHit hit = findHoverHit();    // are we over a brush?
  if (hit != null) {                // yes, let's figure out what to do
    activeBrush = hit.brush;      // flag the brush as the activeBrush
    if (activeBrush.output() == BrushOutput.SAMPLER) {
      scheduleSamplerBrushClick(activeBrush);    // Sampler brush clicked event
    } else {
      scheduleGranularBrushClick(activeBrush);   // Granular brush clicked event
    }
  } else {
    // click outside any brush is handled here
    audioMouseClick(mouseX, mouseY);
  }
  // *****>>> NETWORKING <<<***** //
  int x = clipToWidth(mouseX);
  int y = clipToHeight(mouseY);
  int pos = getSamplePos(x, y);
  if (nd != null && isNetSendOutsideBrushPoints) nd.oscSendMousePressed(x, y, pos);
}


/**
 * Built-in keyPressed handler, forwards events to parseKey.
 */
@Override
  public void keyPressed() {
  if (key == CODED && keyCode == SHIFT) {
    shiftIsDown = true;
    // println("-->> shiftIsDown " + shiftIsDown);
    return;
  }
  if (key != CODED) {
    parseKey(key, keyCode);
  } else {
    if (keyCode == UP) {
      adjustAudioGain(shiftIsDown ? 3.0f : 1.0f);
      println("---- audio gain is "+ nf(audioOut.getGain(), 0, 2) +"dB");
    } else if (keyCode == DOWN) {
      adjustAudioGain(shiftIsDown ? -3.0f : -1.0f);
      println("---- audio gain is "+ nf(audioOut.getGain(), 0, 2) +"dB");
    } else if (keyCode == RIGHT) {
      if (!shiftIsDown) {
        adjustPoolGain(3.0f);
        println("---- pool gain is "+ nf(pool.getGainDb(), 0, 2) +"dB");
      } else {
        adjustGranGain(3.0f);
        println("---- granular gain is "+ nf(gDir.getInstrument().getGlobalGainDb(), 0, 2) +"dB");
      }
    } else if (keyCode == LEFT) {
      if (!shiftIsDown) {
        adjustPoolGain(-3.0f);
        println("---- pool gain is "+ nf(pool.getGainDb(), 0, 2) +"dB");
      } else {
        adjustGranGain(-3.0f);
        println("---- granular gain is "+ nf(gDir.getInstrument().getGlobalGainDb(), 0, 2) +"dB");
      }
    }
  }
}

@Override
  public void keyReleased() {
  if (key == CODED && keyCode == SHIFT) {
    shiftIsDown = false;
    // println("-->> shiftIsDown " + shiftIsDown);
  }
}

/**
 * Handles key press events passed on by the built-in keyPressed method.
 * By moving key event handling outside the built-in keyPressed method,
 * we make it possible to post key commands without an actual key event.
 * Methods and interfaces and even other threads can call parseKey().
 * This opens up many possibilities and a some risks, too.
 *
 * @param key        the key that was pressed, as a char
 * @param keyCode    keyCode for the key that was pressed
 */
public void parseKey(char key, int keyCode) {
  String msg;
  switch(key) {
  case ' ': // (spacebar) trigger a brush if we're hovering over a brush, otherwise trigger a point event
    if (hoverBrush != null) {
      if (hoverBrush.output() == BrushOutput.SAMPLER) {
        scheduleSamplerBrushClick(hoverBrush);    // Sampler brush clicked event
      } else {
        scheduleGranularBrushClick(hoverBrush);   // Granular brush clicked event
      }
    } else {
      audioMouseClick(mouseX, mouseY);
    }
    break;
  case '1': // set brushstroke under cursor to PathMode ALL_POINTS
    // if (activeBrush != null) activeBrush.cfg().pathMode = PathMode.ALL_POINTS;
    if (hoverBrush != null) hoverBrush.cfg().pathMode = PathMode.ALL_POINTS;
    println("-- hover brush path mode is now " + PathMode.ALL_POINTS);
    break;
  case '2': // set brushstroke under cursor to PathMode REDUCED_POINTS
    if (hoverBrush != null) hoverBrush.cfg().pathMode = PathMode.REDUCED_POINTS;
    println("-- hover brush path mode is now " + PathMode.REDUCED_POINTS);
    break;
  case '3': // set brushstroke under cursor to PathMode CURVE_POINTS
    if (hoverBrush != null) hoverBrush.cfg().pathMode = PathMode.CURVE_POINTS;
    println("-- hover brush path mode is now " + PathMode.CURVE_POINTS);
    break;
  case 't': // set brushstroke under cursor to use Sampler or Granular synth
    if (hoverBrush != null) {
      hoverBrush.setOutput(hoverBrush.output() == BrushOutput.SAMPLER ? BrushOutput.GRANULAR : BrushOutput.SAMPLER);
      hoverBrush.cfg.pathMode = defaultPathModeFor(hoverBrush.output());
      println("-- hover brush output is now " + hoverBrush.output());
    }
    break;
  case 'f': // set brushstroke under cursor to FIXED hop mode for granular events
    if (hoverBrush != null) {
      hoverBrush.hopMode = HopMode.FIXED;
      println("-- hover brush hop mode is now " + hoverBrush.hopMode());
    }
    break;
  case 'g': // set brushstroke under cursor to GESTURE hop mode for granular events
    if (hoverBrush != null) {
      hoverBrush.hopMode = HopMode.GESTURE;
      println("-- hover brush hop mode is now " + hoverBrush.hopMode());
    }
    break;
  case 'y': // select granular or sampler synth for click response
    useGranularSynth = !useGranularSynth;
    String synth = (useGranularSynth) ? "granular." : "sampler.";
    println("-- default synth for click events is "+ synth);
    break;
  case 'r': // select long or short grain duration
    useShortGrain = !useShortGrain;
    granSamples = useShortGrain ? shortSample : longSample;
    hopSamples = granSamples/4;
    initGranularParams();
    println("-- Granular synth grain length = "+ granSamples +" samples with "+ hopSamples +" samples hop length");
    break;
  case 'b': // toggle between long burst grains and short burst grains
    useLongBursts = !useLongBursts;
    burstGrains = useLongBursts ? maxBurstGrains : 1;
    initGranularParams();
    msg = useLongBursts ? "long bursts of "+ maxBurstGrains : "short bursts of "+ minBurstGrains ;
    println("-- Granular synth uses "+ msg +" grains.");
    break;
  case 'p': // jitter the pitch of granular gestures
    usePitchedGrains = !usePitchedGrains;
    msg = (usePitchedGrains) ? " jitter granular pitch." : " steady granular pitch.";
    println("-- Play granular synth with "+ msg);
    break;
  case 'P': // adjust pitch of current brushstroke
    if (hoverBrush == null) return;
    pitchScaling = hoverBrush.pitchRatio();
    if (pitchScaling == defaultPitchScaling) {
      pitchScaling = lowPitchScaling;
    } else if (pitchScaling == lowPitchScaling) {
      pitchScaling = highPitchScaling;
    } else if (pitchScaling == highPitchScaling) {
      pitchScaling = defaultPitchScaling;
    } else {
      pitchScaling = defaultPitchScaling;
    }
    hoverBrush.setPitchRatio(pitchScaling);
    msg = "hover brush pitch scaling set to "+ hoverBrush.pitchRatio();
    println("-- "+ msg);
    break;
  case '.': // increment epsilon value of current brush (reduced points decrease)
    if (hoverBrush != null) {
      float e = hoverBrush.curve.getEpsilon();
      e = e < 32 ? e + 1 : e;
      setBrushEpsilon(hoverBrush, e);
      println("-- epsilon for current brush is "+ hoverBrush.curve.getEpsilon());
    }
    break;
  case ',': // decrement epsilon value of current brush (reduced points increase)
    if (hoverBrush != null) {
      float e = hoverBrush.curve.getEpsilon();
      e = e > 1 ? e - 1 : e;
      setBrushEpsilon(hoverBrush, e);
      println("-- epsilon for current brush is "+ hoverBrush.curve.getEpsilon());
    }
    break;
  case '>': // increment curve steps value of current brush
    if (hoverBrush != null) {
      int cs = hoverBrush.curve.getCurveSteps();
      cs = cs < 32 ? cs + 1 : cs;
      setBrushCurveSteps(hoverBrush, cs);
      println("-- curveSteps for current brush is "+ hoverBrush.curve.getCurveSteps());
    }
    break;
  case '<': // decrement curve steps value of current brush
    if (hoverBrush != null) {
      int cs = hoverBrush.curve.getCurveSteps();
      cs = cs > 1 ? cs - 1 : cs;
      setBrushCurveSteps(hoverBrush, cs);
      println("-- curveSteps for current brush is "+ hoverBrush.curve.getCurveSteps());
    }
    break;
  case 'm': // change the Sampler noise reduction profile
    pool.cycleMixProfile();
    println("-- mix profile is "+ pool.getMixProfile().name());
    break;
  case 'e': // change the envelope we're using
    presetIndex = (presetIndex + 1) % envPresets.length;
    defaultEnv = envPreset(envPresets[presetIndex]);
    println("-- default envelope is "+ envPresets[presetIndex] +", "+ defaultEnv.toString());
    break;
  case 'E': // toggle whether we adjust envelope duration in relation to gesture duration
    isAdjustEnvelope = !isAdjustEnvelope;
    println("-- isAdjustEnvelope is "+ isAdjustEnvelope);
    break;
  case 'a': //  start or stop animation (bitmap rotation along the signal path)
    isAnimating = !isAnimating;
    println("-- animation is " + isAnimating);
    break;
  case '/': // turn drawing on or off
    // turn off animation (if you insist, you can try drawing with it on, just press the 'a' key)
    isAnimating = false;
    isDrawMode = !isDrawMode;
    println(isDrawMode ? "----- Drawing is turned on" : "----- Drawing is turned off");
    break;
  case 'd': // toggle play audio on new brush
    doPlayOnNewBrush = !doPlayOnNewBrush;
    println("-- play audio when creating a new brush is "+ (doPlayOnNewBrush ? "ON" : "OFF"));
    break;
  case 'c': // apply color from image file to display image
    chooseColorImage();
    break;
  case 'k': // apply the hue and saturation in the colors array to mapImage (not to baseImage)
    applyColorMapToDisplay(false);
    break;
  case 'K': // apply hue and saturation in colors to baseImage and mapImage
    applyColorMapToDisplay(true);
    break;
  case 'j': // turn audio and image blending on or off
    isBlending = !isBlending;
    println("-- isBlending is "+ isBlending);
    break;
  case 'n': // normalize the audio buffer to -6.0dB
    audioSignal = playBuffer.getChannel(0);
    normalize(audioSignal, -6.0f);
    updateAudioChain(audioSignal);
    ensureSamplerReady();
    ensureGranularReady();
    println("---- normalized");
    break;
  case 'L':
  case 'l': // determine whether to load a new file to both image and audio or not
    isLoadToBoth = !isLoadToBoth;
    msg = isLoadToBoth ? "loads to both audio and image. " : "loads only to selected format. ";
    println("---- isLoadToBoth is "+ isLoadToBoth +", opening a file "+ msg);
    break;
  case 'o':
  case 'O': // open an audio or image file and load to image or audio buffer or both
    chooseFile();
    break;
  case 's': // save display image to a PNG file
    saveToImage();
    break;
  case 'S': // save audio buffer to a .wav file
    saveToAudio();
    break;
  case 'w': // write the image HSB Brightness channel to the audio buffer as transcoded sample values
    // TODO refactor with loadImageFile() and loadAudioFile() code
    // prepare to copy image data to audio variables
    // resize the buffer to mapSize, if necessary -- signal will not be overwritten
    if (playBuffer.getBufferSize() != mapper.getSize()) playBuffer.setBufferSize(mapper.getSize());
    audioSignal = playBuffer.getChannel(0);
    // transcode mapImage brightness channel in HSB color space to audio sample values
    renderMapImageToAudio(PixelAudioMapper.ChannelNames.L);
    commitMapImageToBaseImage();
    // now that the image data has been written to audioSignal, set playBuffer channel 0 to the new audio data
    updateAudioChain(audioSignal);
    println("--->> Wrote image to audio as audio data.");
    // load the buffer of our PASamplerInstrument (it will use a copy)
    ensureSamplerReady();
    ensureGranularReady();
    break;
  case 'W': // write the audio buffer samples to the image as color values
    renderAudioToMapImage(chan, totalShift);
    println("--->> Wrote audio to image as pixel data.");
    break;
  case 'x': // delete the current active brush shape or the oldest brush shape
    if (activeBrush != null) removeHoveredOrOldestBrush();
    break;
  case 'X': // delete the most recent brush shape
    removeNewestBrush();
    break;
  case 'u': // mute audio
    isMuted = !isMuted;
    if (isMuted) {
      audioOut.mute();
    } else {
      audioOut.unmute();
    }
    msg = isMuted ? "muted" : "unmuted";
    println("---- audio out is "+ msg);
    break;
  case 'V': // record a video
    // records a complete video loop with following actions:
    // Go to frame 0, turn recording on, turn animation on,
    // record animSteps number of frames.
    // not of much use in this sketch, but here to keep code complete
    step = 0;
    renderFrame(step);
    isRecordingVideo = true;
    oldIsAnimating = isAnimating;
    isAnimating = true;
    break;
    // *****>>> NETWORKING <<<***** // key commands for networking
  case ']': // send UDP message to Max (simpleAudioIO.maxpat): reverb ON
    if (nd != null) nd.oscSendOnOff(1, true);
    break;
  case '[': // send UDP message to Max (simpleAudioIO.maxpat): reverb OFF
    if (nd != null) nd.oscSendOnOff(1, false);
    break;
  case '}': // send UDP message to Max (simpleAudioIO.maxpat): unused
    if (nd != null) nd.oscSendOnOff(2, true);
    break;
  case '{': // send UDP message to Max (simpleAudioIO.maxpat): unused
    if (nd != null) nd.oscSendOnOff(2, false);
    break;
  case 'q': // send UDP message to Max (simpleAudioIO.maxpat): small reverb settings
    if (nd != null) {
      nd.oscSendSetnum(1, 0.125f);
      nd.oscSendSetnum(2, 2048.0f);
      nd.oscSendSetnum(3, 0.125f);
      nd.oscSendSetnum(4, 2560.0f);
    }
    break;
  case 'Q': // send UDP message to Max (simpleAudioIO.maxpat): big reverb settings
    if (nd != null) {
      nd.oscSendSetnum(1, 0.3f);
      nd.oscSendSetnum(2, 4096.0f);
      nd.oscSendSetnum(3, 0.3f);
      nd.oscSendSetnum(4, 5120.0f);
    }
    break;
  case 'h':
  case 'H': // show help message in the console
    showHelp();
    break;
  default:
    break;
  }
}

/**
 * to generate help output, run RegEx search/replace on parseKey case lines with:
 * // case ('.'): // (.+)
 * // println(" * Press $1 to $2.");
 */
public void showHelp() {
  println(" * Press UP ARROW to increase audio gain by 3 dB.");
  println(" * Press DOWN ARROW to decrease audio gain by 3 dB.");
  println(" * Press RIGHT ARROW to increase Sampler audio gain by 3 dB.");
  println(" * Press LEFT ARROW to decrease Sampler audio gain by 3 dB.");
  println(" * Press SHIFT-RIGHT ARROW to increase Granular audio gain by 3 dB.");
  println(" * Press SHIFT-LEFT ARROW to decrease Granular audio gain by 3 dB.");
  println(" * Press ' ' (spacebar) to trigger a brush if we're hovering over a brush, otherwise trigger a point event.");
  println(" * Press '1' to set brushstroke under cursor to PathMode ALL_POINTS.");
  println(" * Press '2' to set brushstroke under cursor to PathMode REDUCED_POINTS.");
  println(" * Press '3' to set brushstroke under cursor to PathMode CURVE_POINTS.");
  println(" * Press 't' to set brushstroke under cursor to use Sampler or Granular synth.");
  println(" * Press 'f' to set brushstroke under cursor to FIXED hop mode for granular events.");
  println(" * Press 'g' to set brushstroke under cursor to GESTURE hop mode for granular events.");
  println(" * Press 'y' to select granular or sampler synth for click response.");
  println(" * Press 'r' to select long or short grain duration .");
  println(" * Press 'b' to toggle between long burst grains and short burst grains.");
  println(" * Press 'p' to jitter the pitch of granular gestures.");
  println(" * Press 'P' to adjust pitch of current brushstroke.");
  println(" * Press '.' to increment epsilon value of current brush (reduced points decrease).");
  println(" * Press ',' to decrement epsilon value of current brush (reduced points increase).");
  println(" * Press '>' to increment curve steps value of current brush.");
  println(" * Press '<' to decrement curve steps value of current brush.");
  println(" * Press 'm' to change the Sampler noise reduction profile.");
  println(" * Press 'e' to change the envelope we're using .");
  println(" * Press 'E' to toggle whether we adjust envelope duration in relation to gesture duration.");
  println(" * Press 'a' to  start or stop animation (bitmap rotation along the signal path).");
  println(" * Press 'd' to toggle play audio on new brush .");
  println(" * Press 'c' to apply color from image file to display image.");
  println(" * Press 'k' to apply the hue and saturation in the colors array to mapImage (not to baseImage).");
  println(" * Press 'K' to apply hue and saturation in colors to baseImage and mapImage.");
  println(" * Press 'j' to turn audio and image blending on or off.");
  println(" * Press 'n' to normalize the audio buffer to -6.0dB.");
  println(" * Press 'L' or 'l' to determine whether to load a new file to both image and audio or not.");
  println(" * Press 'o' or 'O' to open an audio or image file and load to image or audio buffer or both.");
  println(" * Press 's' to save display image to a PNG file.");
  println(" * Press 'S' to save audio buffer to a .wav file.");
  println(" * Press 'w' to write the image HSB Brightness channel to the audio buffer as transcoded sample values .");
  println(" * Press 'W' to write the audio buffer samples to the image as color values.");
  println(" * Press 'x' to delete the current active brush shape or the oldest brush shape.");
  println(" * Press 'X' to delete the most recent brush shape.");
  println(" * Press 'u' to mute audio.");
  println(" * Press 'V' to record a video.");
  println(" * Press ']' to send UDP message to Max (simpleAudioIO.maxpat): reverb ON.");
  println(" * Press '[' to send UDP message to Max (simpleAudioIO.maxpat): reverb OFF.");
  println(" * Press '}' to send UDP message to Max (simpleAudioIO.maxpat): unused.");
  println(" * Press '{' to send UDP message to Max (simpleAudioIO.maxpat): unused.");
  println(" * Press 'q' to send UDP message to Max (simpleAudioIO.maxpat): small reverb settings.");
  println(" * Press 'Q' to send UDP message to Max (simpleAudioIO.maxpat): big reverb settings.");
  println(" * Press 'h' or 'H' to show help message in the console.");
}

/**
 * Sets audioOut.gain.
 * @param g   gain value for audioOut, in decibels
 */
public void adjustAudioGain(float g) {
  float ag = audioOut.getGain();
  ag += g;
  if (ag > 12.0f || ag < -64.0f) return;
  audioOut.setGain(ag);
  outputGain = audioOut.getGain();
}

/**
 * Sets Sampler instrument <code>pool</code> gain in dB.
 * @param g   gain increment or decrement, in decibels
 */
public void adjustPoolGain(float g) {
  float pg = pool.getGainDb();
  pg += g;
  if (pg > 12.0f || pg < -64.0f) return;
  pool.setGainDb(pg);
}

/**
 * Sets Granular instrument gain in dB.
 * @param g   gain value for audioOut, in decibels
 */
public void adjustGranGain(float g) {
  float gg = gDir.getInstrument().getGlobalGainDb();
  gg += g;
  if (gg > 12.0f || gg < -64.0f) return;
  gDir.getInstrument().setGlobalGainDb(gg);
}


/**
 * Utility method for applying hue and saturation values from a source array of RGB values
 * to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
 *
 * @param colorSource    a source array of RGB data from which to obtain hue and saturation values
 * @param graySource     an target array of RGB data from which to obtain brightness values
 * @param lut            a lookup table, must be the same size as colorSource and graySource
 * @return the graySource array of RGB values, with hue and saturation values changed
 * @throws IllegalArgumentException if array arguments are null or if they are not the same length
 */
public int[] applyColor(int[] colorSource, int[] graySource, int[] lut) {
  return applyColorShifted(colorSource, graySource, lut, 0);
}

/**
 * Utility method for applying hue and saturation values from a source array of RGB values
 * to the brightness values in a target array of RGB values, using a lookup table to redirect indexing,
 * taking into account any pixels that were shifted.
 *
 * @param colorSource    a source array of RGB data from which to obtain hue and saturation values
 * @param graySource     an target array of RGB data from which to obtain brightness values
 * @param lut            a lookup table, must be the same size as colorSource and graySource
 * @param shift          number of pixels/array indices to shift
 * @return the graySource array of RGB values, with hue and saturation values changed
 * @throws IllegalArgumentException if array arguments are null or if they are not the same length
 */
public int[] applyColorShifted(int[] colorSource, int[] graySource, int[] lut, int shift) {
  if (colorSource == null || graySource == null || lut == null)
    throw new IllegalArgumentException("colorSource, graySource and lut cannot be null.");
  if (colorSource.length != graySource.length || colorSource.length != lut.length)
    throw new IllegalArgumentException("colorSource, graySource and lut must all have the same length.");
  int n = graySource.length;
  int s = ((shift % n) + n) % n; // wrap + allow negative shifts
  float[] hsbPixel = new float[3];
  for (int i = 0; i < n; i++) {
    int srcIdx = lut[i] + s;
    if (srcIdx >= n) srcIdx -= n; // faster than % in tight loop
    graySource[i] = PixelAudioMapper.applyColor(colorSource[srcIdx], graySource[i], hsbPixel);
  }
  return graySource;
}

/**
 * Apply color map hue and saturation to mapImage or baseImage.
 *
 * @param updateBaseImage    if true, update baseImage, otherwise just update mapImage
 */
public void applyColorMapToDisplay(boolean updateBaseImage) {
  if (updateBaseImage) {
    baseImage.loadPixels();
    applyColor(spectrum, baseImage.pixels, mapper.getImageToSignalLUT());
    baseImage.updatePixels();
    refreshMapImageFromBase();
  } else {
    refreshMapImageFromBase();
    mapImage.loadPixels();
    applyColorShifted(spectrum, mapImage.pixels, mapper.getImageToSignalLUT(), totalShift);
    mapImage.updatePixels();
  }
}
