Package net.paulhertz.pixelaudio.example

# Class TutorialOne_02_Animation

java.lang.Object
processing.core.PApplet
net.paulhertz.pixelaudio.example.TutorialOne_02_Animation

All Implemented Interfaces:
  `processing.core.PConstants`

public class TutorialOne_02_Animation extends processing.core.PApplet

This example sketch builds on TutorialOne_01_FileIO, which provided commands to open and display audio and image files, transcode image pixel data to audio samples, transcode audio samples to image pixel data, and save audio and image files. TutorialOne_01_FileIO also introduced a digital audio instrument, PASamplerInstrument, and provided an interface which responds to mouse clicks by playing the audio samples corresponding to the click location in the display image. In this sketch, we add panning and pitch control to PASamplerInstrument `playSample(...)` methods. The other audio instrument, the Granular Synthesis instrument, will be introduced later on. This sketch introduces the animation feature built-in to PixelAudioMapper, pixel-shifting along the Signal Path, which shifts pixels in the display and changes the correspondence between screen coordinates and audio signal index position. The animation provides an analog to visual "audio streaming" and can be used to create a sort of "music box" audio player.

       QUICK START

       1. Launch the sketch and then press the 'o' key to open an image or audio
          file. The "audioBlend.wav" file is good for experimenting. The audio
          data in it will be transcoded to gray scale pixel data and displayed in
          the window. Click on the display image to trigger audio events. To see
          the signal path as a color overlay, press the 'K' key.

       2. Press 'm' to run the "Music Box" routine, which triggers an audio event
          every frame in the same location while animating the display image with
          pixel shifting. Use the UP and DOWN arrow keys to change the amount of
          pixel shifting, and the 'A' key to reverse the direction of shifting.
          Use the 'p' and 'P' keys to change the pitch of the audio. Press 'r'
          to turn random ADSR envelope selection on and off.
          See below for more information.

We add a simple form of animation in this sketch, which consists of shifting the pixels in the display image along the Signal Path. This is a simple way to create an animation that is directly tied to the audio data. When the image we are animating is a representation of audio data or some other sort of periodic pattern, pixel-shfting can result in hypnotic animated patterns. This is demonstrated by the the WaveSynth class and the WaveSynthEditor and ArgosyMixer sample sketches. When the pixels in the display image are shifted, the correspondence between pixel coordinates and audio buffer index changes. This is particularly evident when the image is a visualization of the audio buffer. If you are looking at an image that represents audio data, such as Saucer_mixdown.wav, you can see how animation changes the apparent position of the audio data. When animation is running, repeated clicks at the same location in the window will trigger different audio events.

Pixel shifting animation works by moving the pixels in the display image `mapImage` once every frame by the number of pixels specified by the variable `shift`, and tracking the accummulated shifting with another variable, `totalShift`. The audio buffer itself is never shifted, nor is the canonic image `baseImage`. We just use totalShift to determine where to locate pixels or audio samples. The shifting is managed by the PixelAudioMapper object `mapper`, which provides methods for accessing pixel and audio data with shifting taken into account. Of course, if you only want to animate the image, you can just ignore the value of totalShift when accessing the audioBuffer.

The shift variable can be tied to the audio sample rate and the video frame rate in such a way that the animation is synchronized with the audio when we trigger an audio event once every frame. If the audio sample rate is 44100 and the video frame rate is 24, as it is in this example, then we can set shift to 44100/24, which is about 1838 ('m' key). If we then trigger an audio event once every frame in the same location in the window, the audio events will advance through the buffer in sync with the animation, and the audio will play more or less as it would if it were streaming from a file. If you double the shift or cut it in half (UP ARROW or DOWN ARROW), the audio events will happen at double or half speed. If you reverse the sign of shift ('A' key), the animation will run backwards, and the audio events will also happen in reverse order.

We also provide commands for saving an animation to video ('V' key) and saving the display image and audio buffer to files. To visualize the Signal Path, you can use the 'k' or 'K' keys to overlay a spectrum of color on the display image. The 'K' key will write the spectrum to `baseImage`, so that it will persist when running animation. The 'k' key will write color data to `mapImage` only.

We continue to use the Sampler instrument for audio events, showing how it can control the pitch and panning of an audio event. The playSample() methods in this sketch introduce the most complete audio triggering method available in PASamplerInstrument, one which can set sample start position, length, amplitude, ADSR-style envelope, pitch scaling and stereo pan location.

       samplelen = synth.playSample(samplePos, (int) samplelen, amplitude, env, pitchScaling, pan);

Pitch scaling by default is 1.0, which is to say that samples will be played back at the recorded frequency. Panning, if it is not supplied as an argument to playSample(), is centered in the stereo field. We calculate the panning value by mapping the x-coordinate of the mouse position to the stereo pan value.

Still to come, as the tutorial advances:
-- drawing to trigger audio events
-- the Granular Synthesis instruments
-- UDP communication with Max and other media applications

The key commands:

- Press UP ARROW to increment pixel shift.
- Press DOWN ARROW to decrement pixel shift.
- Press ' ' (spacebar) to play sample at current mouse position.
- Press 'a' to turn animation on or off.
- Press 'A' to change animation direction.
- Press 'f' to rotate pixels by shift value .
- Press 'F' to rotate pixels by shift value .
- Press 'm' to play audio events synced to audio sample rate and video frame rate.
- Press 'c' to apply color from image file to display image (mapImage) only.
- Press 'C' to apply color from image file to base image and map image.
- Press 'k' to apply hue and saturation in the spectrum array to mapImage .
- Press 'K' to apply hue and saturation in the spectrum array to baseImage.
- Press 'o' or 'O' to open an audio or image file.
- Press 'p' to select low pitch scaling or default pitch scaling.
- Press 'P' to select high pitch scaling or default pitch scaling.
- Press 'd' or 'D' to turn rain on and off.
- Press 'r' or 'R' to toggle isRandomADSR, to use default envelope or a random choice.
- Press 's' to save display image to a PNG file.
- Press 'S' to save audio buffer to a .wav file.
- Press 'V' to record a video from frame 0 to frame animSteps.
- Press 'w' to show data values in display.
- Press 'h' or 'H' to show help message.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `(package private) ArrayList<ADSRParams>`
  `adsrList`
  `(package private) int`
  `animSteps`
  `(package private) float`
  `attackTime`
  `(package private) File`
  `audioFile`
  `(package private) int`
  `audioFileLength`
  `(package private) String`
  `audioFileName`
  `(package private) String`
  `audioFilePath`
  `(package private) String`
  `audioFileTag`
  `(package private) float`
  `audioGain`
  `(package private) int`
  `audioLength`
  `(package private) ddf.minim.AudioOutput`
  `audioOut`
  `(package private) float[]`
  `audioSignal`
  `(package private) processing.core.PImage`
  `baseImage`
  `(package private) float`
  `bufferSampleRate`
  `(package private) PixelAudioMapper.ChannelNames`
  `chan`
  `(package private) int`
  `count`
  `(package private) float`
  `decayTime`
  `(package private) ADSRParams`
  `defaultEnv`
  `(package private) float`
  `defaultPitchScaling`
  `(package private) boolean`
  `doResample`
  `(package private) int`
  `fileIndex`
  `(package private) float`
  `fileSampleRate`
  `(package private) int`
  `genHeight`
  `(package private) int`
  `genWidth`
  `(package private) float`
  `highPitchScaling`
  `(package private) File`
  `imageFile`
  `(package private) int`
  `imageFileHeight`
  `(package private) String`
  `imageFileName`
  `(package private) String`
  `imageFilePath`
  `(package private) String`
  `imageFileTag`
  `(package private) int`
  `imageFileWidth`
  `(package private) boolean`
  `isAnimating`
  `(package private) boolean`
  `isPlayMusicBox`
  `(package private) boolean`
  `isRaining`
  `(package private) boolean`
  `isRandomADSR`
  `(package private) boolean`
  `isRecordingVideo`
  `(package private) boolean`
  `isShowOverlay`
  `(package private) float`
  `lowPitchScaling`
  `(package private) processing.core.PImage`
  `mapImage`
  `(package private) PixelAudioMapper`
  `mapper`
  `(package private) int`
  `mapSize`
  `(package private) float`
  `maxAmplitude`
  `(package private) ddf.minim.Minim`
  `minim`
  Minim audio library
  `(package private) MultiGen`
  `multigen`
  `(package private) int`
  `noteDuration`
  `(package private) boolean`
  `oldIsAnimating`
  `(package private) float`
  `pitchScaling`
  `(package private) PixelAudio`
  `pixelaudio`
  `(package private) ddf.minim.MultiChannelBuffer`
  `playBuffer`
  `(package private) float`
  `releaseTime`
  `(package private) float`
  `sampleRate`
  `(package private) float`
  `samplerGain`
  `(package private) int`
  `shift`
  `(package private) int`
  `shiftInc`
  `(package private) int[]`
  `spectrum`
  `(package private) int`
  `step`
  `(package private) float`
  `sustainLevel`
  `(package private) PASamplerInstrument`
  `synth`
  `(package private) ArrayList<TimedLocation>`
  `timeLocsArray`
  `(package private) int`
  `totalShift`
  `(package private) int`
  `videoFrameRate`
  `(package private) com.hamoid.VideoExport`
  `videx`

  ### Fields inherited from class processing.core.PApplet

  `args, ARGS_BGCOLOR, ARGS_DISABLE_AWT, ARGS_DISPLAY, ARGS_EDITOR_LOCATION, ARGS_EXTERNAL, ARGS_FULL_SCREEN, ARGS_HIDE_STOP, ARGS_LOCATION, ARGS_PRESENT, ARGS_SKETCH_FOLDER, ARGS_STOP_COLOR, ARGS_UI_SCALE, ARGS_WINDOW_COLOR, DEFAULT_HEIGHT, DEFAULT_WIDTH, disableAWT, displayHeight, displayWidth, dmouseX, dmouseY, emouseX, emouseY, exitCalled, EXTERNAL_MOVE, EXTERNAL_STOP, finished, firstMouse, focused, frameCount, frameRate, frameRateLastNanos, g, height, insideDraw, javaPlatform, javaVersion, javaVersionName, key, keyCode, keyEvent, keyPressed, keyRepeatEnabled, looping, matchPatterns, mouseButton, mouseEvent, mousePressed, mouseX, mouseY, pixelDensity, pixelHeight, pixels, pixelWidth, platform, pmouseX, pmouseY, ratioLeft, ratioScale, ratioTop, recorder, redraw, rheight, rmouseX, rmouseY, rwidth, surface, useNativeSelect, width, windowX, windowY`

  ### Fields inherited from interface processing.core.PConstants

  `ADD, ALPHA, ALT, AMBIENT, ARC, ARGB, ARROW, BACKSPACE, BASELINE, BEVEL, BEZIER_VERTEX, BLEND, BLUR, BOTTOM, BOX, BREAK, BURN, CENTER, CHATTER, CHORD, CLAMP, CLOSE, CODED, COMPLAINT, CONTROL, CORNER, CORNERS, CROSS, CURVE_VERTEX, CUSTOM, DARKEST, DEG_TO_RAD, DELETE, DIAMETER, DIFFERENCE, DILATE, DIRECTIONAL, DISABLE_ASYNC_SAVEFRAME, DISABLE_BUFFER_READING, DISABLE_DEPTH_MASK, DISABLE_DEPTH_SORT, DISABLE_DEPTH_TEST, DISABLE_KEY_REPEAT, DISABLE_NATIVE_FONTS, DISABLE_OPENGL_ERRORS, DISABLE_OPTIMIZED_STROKE, DISABLE_STROKE_PERSPECTIVE, DISABLE_STROKE_PURE, DISABLE_TEXTURE_MIPMAPS, DODGE, DOWN, DXF, ELLIPSE, ENABLE_ASYNC_SAVEFRAME, ENABLE_BUFFER_READING, ENABLE_DEPTH_MASK, ENABLE_DEPTH_SORT, ENABLE_DEPTH_TEST, ENABLE_KEY_REPEAT, ENABLE_NATIVE_FONTS, ENABLE_OPENGL_ERRORS, ENABLE_OPTIMIZED_STROKE, ENABLE_STROKE_PERSPECTIVE, ENABLE_STROKE_PURE, ENABLE_TEXTURE_MIPMAPS, ENTER, EPSILON, ERODE, ESC, EXCLUSION, FX2D, GIF, GRAY, GROUP, HALF_PI, HAND, HARD_LIGHT, HINT_COUNT, HSB, IMAGE, INVERT, JAVA2D, JPEG, LANDSCAPE, LEFT, LIGHTEST, LINE, LINE_LOOP, LINE_STRIP, LINES, LINUX, MACOS, MACOSX, MAX_FLOAT, MAX_INT, MIN_FLOAT, MIN_INT, MITER, MODEL, MODELVIEW, MOVE, MULTIPLY, NORMAL, OPAQUE, OPEN, OPENGL, ORTHOGRAPHIC, OTHER, OVERLAY, P2D, P3D, PATH, PDF, PERSPECTIVE, PI, PIE, platformNames, POINT, POINTS, POLYGON, PORTRAIT, POSTERIZE, PROBLEM, PROJECT, PROJECTION, QUAD, QUAD_BEZIER_VERTEX, QUAD_STRIP, QUADRATIC_VERTEX, QUADS, QUARTER_PI, RAD_TO_DEG, RADIUS, RECT, REPEAT, REPLACE, RETURN, RGB, RIGHT, ROUND, SCREEN, SHAPE, SHIFT, SOFT_LIGHT, SPAN, SPHERE, SPOT, SQUARE, SUBTRACT, SVG, TAB, TARGA, TAU, TEXT, THIRD_PI, THRESHOLD, TIFF, TOP, TRIANGLE, TRIANGLE_FAN, TRIANGLE_STRIP, TRIANGLES, TWO_PI, UP, VERTEX, WAIT, WHITESPACE, WINDOWS, X, Y, Z`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `TutorialOne_02_Animation()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `animate()`
  `int[]`
  `applyColor(int[] colorSource, int[] graySource, int[] lut)`
  Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
  `void`
  `applyColorMapToDisplay(boolean updateBaseImage)`
  Apply color map hue and saturation to mapImage or baseImage.
  `int[]`
  `applyColorShifted(int[] colorSource, int[] graySource, int[] lut, int pixelShift)`
  Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
  `void`
  `applyImageColor(File imgFile, processing.core.PImage targetImage)`
  Apply the hue and saturation of a chosen image file to the brightness channel of the display image.
  `void`
  `audioFileSelectedWrite(File selection)`
  `void`
  `audioMouseClick(int x, int y)`
  Typically called from mousePressed with mouseX and mouseY, generates audio events.
  `int`
  `calcSampleLen()`
  `void`
  `chooseColorImage()`
  Call to initiate process of opening an image file to get its color data.
  `void`
  `chooseColorImageAndStore()`
  `void`
  `chooseFile()`
  Wrapper method for Processing's selectInput command
  `int`
  `clipToHeight(int y)`
  `int`
  `clipToWidth(int x)`
  `void`
  `colorFileSelected(File selectedFile)`
  callback method for chooseColorImage()
  `void`
  `colorFileSelectedToStore(File selectedFile)`
  `void`
  `commitMapImageToBaseImage()`
  `void`
  `commitNewBaseImage(processing.core.PImage img)`
  `void`
  `doRain()`
  drop some random audio events, like unto the gentle rain
  `void`
  `draw()`
  `void`
  `drawCircle(int x, int y)`
  Draws a circle at the location of an audio trigger (mouseDown event).
  `(package private) void`
  `ensureSamplerReady()`
  Prepares Sampler instruments and assets
  `void`
  `fileSelected(File selectedFile)`
  callback method for chooseFile(), handles standard audio and image formats for Processing.
  `int[]`
  `getColors(int size)`
  Generates an array of rainbow colors using the HSB color space.
  `int`
  `getSamplePos(int x, int y)`
  Calculates the index of the image pixel within the signal path, taking the shifting of pixels and audioSignal into account.
  `void`
  `imageFileSelectedWrite(File selection)`
  `void`
  `initADSRList()`
  `void`
  `initAudio()`
  CALL THIS METHOD IN SETUP() Initializes Minim audio library and audio variables.
  `void`
  `initImages()`
  Initializes mapImage with the colors array, copies mapImage to baseImage.
  `void`
  `keyPressed()`
  built-in keyPressed handler, forwards events to parseKey.
  `void`
  `loadAudioFile(File audFile)`
  Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage As in most PixelAudio examples, we provide built-in resampling of audio data from the file to match the sampling rate of the buffer to audioOut.sampleOut().
  `void`
  `loadImageFile(File imgFile)`
  Attempts to load image data from a selected file into mapImage, then calls writeImageToAudio() to transcode HSB brightness color data from the image to audio and writes it to playBuffer and audioSignal.
  `static void`
  `main(String[] args)`
  `void`
  `mouseClicked()`
  `void`
  `mouseDragged()`
  `void`
  `mousePressed()`
  The built-in mousePressed handler for Processing, not used yet...
  `void`
  `mouseReleased()`
  `void`
  `parseKey(char key, int keyCode)`
  Handles key press events passed on by the built-in keyPressed method.
  `int`
  `playSample(int samplePos, int samplelen, float amplitude, float pan)`
  Plays an audio sample with PASamplerInstrument and default ADSR.
  `int`
  `playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pan)`
  Plays an audio sample with PASamplerInstrument and custom ADSR.
  `void`
  `refreshMapImageFromBase()`
  `void`
  `renderAudioToMapImage(PixelAudioMapper.ChannelNames chan, int shift)`
  Render audio into mapImage at an arbitrary phase offset.
  `void`
  `renderFrame(int step)`
  `void`
  `renderFrame(int step, boolean isInverseShift)`
  Renders a frame of animation: moving along the signal path, copies baseImage pixels to mapImage pixels, adjusting the index position of the copy using totalShift i.e.
  `void`
  `renderMapImageToAudio(PixelAudioMapper.ChannelNames chan)`
  Writes a specified channel of mapImage to audioSignal.
  `void`
  `runTimeArray()`
  Run the animation for audio events.
  `void`
  `saveAudioToFile(float[] samples, float sampleRate, String fileName)`
  Saves audio data to 16-bit integer PCM format, which Processing can also open.
  `void`
  `saveImageToFile(processing.core.PImage img, String fileName)`
  `void`
  `saveToAudio()`
  `void`
  `saveToImage()`
  `void`
  `settings()`
  `void`
  `setup()`
  `void`
  `showHelp()`
  to generate help output, run RegEx search/replace on parseKey case lines with: // case ('.'): // (.+) // println(" * Press $1 to $2.");
  `void`
  `stepAnimation()`
  Step through the animation, called by the draw() method.
  `void`
  `stop()`
  turn off audio processing when we exit
  `(package private) void`
  `updateAudioChain(float[] sig)`
  `(package private) void`
  `updateAudioChain(float[] sig, float bufferSampleRate)`
  Bottleneck "commit" method for audio state.
  `void`
  `writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan, int shift)`
  Transcodes audio data in sig[] and writes it to color channel chan of img using the lookup tables in mapper to redirect indexing.
  `void`
  `writeImageToAudio(processing.core.PImage img, PixelAudioMapper mapper, float[] sig, PixelAudioMapper.ChannelNames chan, int shift)`
  This method writes a color channel from an image to playBuffer, fulfilling a central concept of the PixelAudio library: image is sound.
  `void`
  `writeToScreen(String msg, int x, int y, int weight, boolean isWhite)`
  Displays a line of text to the screen, usually in the draw loop.

  ### Methods inherited from class processing.core.PApplet

  `abs, abs, acos, alpha, ambient, ambient, ambient, ambientLight, ambientLight, append, append, append, append, append, append, applyMatrix, applyMatrix, applyMatrix, applyMatrix, applyMatrix, arc, arc, arraycopy, arraycopy, arraycopy, arrayCopy, arrayCopy, arrayCopy, asin, atan, atan2, attrib, attrib, attrib, attribColor, attribNormal, attribPosition, background, background, background, background, background, background, background, beginCamera, beginContour, beginPGL, beginRaw, beginRaw, beginRecord, beginRecord, beginShape, beginShape, bezier, bezier, bezierDetail, bezierPoint, bezierTangent, bezierVertex, bezierVertex, binary, binary, binary, binary, blend, blend, blendColor, blendMode, blue, box, box, brightness, calcSketchPath, camera, camera, ceil, checkAlpha, checkExtension, choice, choice, circle, clear, clip, color, color, color, color, color, color, color, color, colorMode, colorMode, colorMode, colorMode, concat, concat, concat, concat, concat, concat, concat, constrain, constrain, copy, copy, copy, cos, createFont, createFont, createFont, createGraphics, createGraphics, createGraphics, createImage, createInput, createInput, createInputRaw, createOutput, createOutput, createPath, createPath, createPrimaryGraphics, createReader, createReader, createReader, createShape, createShape, createShape, createWriter, createWriter, createWriter, cursor, cursor, cursor, cursor, curve, curve, curveDetail, curvePoint, curveTangent, curveTightness, curveVertex, curveVertex, dataFile, dataPath, day, debug, degrees, delay, dequeueEvents, desktopFile, desktopPath, die, die, directionalLight, displayDensity, displayDensity, dispose, dist, dist, edge, ellipse, ellipseMode, emissive, emissive, emissive, endCamera, endContour, endPGL, endRaw, endRecord, endShape, endShape, exec, exec, exit, exitActual, exitCalled, exp, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, fill, fill, fill, fill, fill, fill, filter, filter, filter, floor, flush, focusGained, focusLost, frameMoved, frameRate, frustum, fullScreen, fullScreen, fullScreen, fullScreen, get, get, get, getExtension, getGraphics, getMatrix, getMatrix, getMatrix, getSurface, green, handleDraw, handleKeyEvent, handleMethods, handleMouseEvent, hex, hex, hex, hex, hideMenuBar, hint, hour, hue, image, image, image, imageMode, initSurface, insertFrame, isLooping, join, join, keyPressed, keyReleased, keyReleased, keyTyped, keyTyped, launch, lerp, lerpColor, lerpColor, lightFalloff, lights, lightSpecular, line, line, link, listFiles, listFiles, listPaths, loadBytes, loadBytes, loadBytes, loadFont, loadImage, loadImage, loadJSONArray, loadJSONArray, loadJSONObject, loadJSONObject, loadPixels, loadShader, loadShader, loadShape, loadShape, loadStrings, loadStrings, loadStrings, loadStrings, loadTable, loadTable, loadXML, loadXML, log, loop, mag, mag, main, main, main, makeGraphics, map, mask, match, matchAll, max, max, max, max, max, max, method, millis, min, min, min, min, min, min, minute, modelX, modelY, modelZ, month, mouseClicked, mouseDragged, mouseEntered, mouseEntered, mouseExited, mouseExited, mouseMoved, mouseMoved, mousePressed, mouseReleased, mouseWheel, mouseWheel, nf, nf, nf, nf, nf, nf, nfc, nfc, nfc, nfc, nfp, nfp, nfp, nfp, nfs, nfs, nfs, nfs, noClip, noCursor, noFill, noise, noise, noise, noiseDetail, noiseDetail, noiseSeed, noLights, noLoop, norm, normal, noSmooth, noStroke, noTexture, noTint, orientation, ortho, ortho, ortho, parseBoolean, parseBoolean, parseBoolean, parseBoolean, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseChar, parseChar, parseChar, parseChar, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseJSONArray, parseJSONObject, parseXML, parseXML, pause, perspective, perspective, pixelDensity, point, point, pointLight, pop, popMatrix, popStyle, postEvent, postWindowMoved, postWindowResized, pow, print, print, print, print, print, print, print, print, print, printArray, printCamera, println, println, println, println, println, println, println, println, println, println, println, printMatrix, printProjection, printStackTrace, push, pushMatrix, pushStyle, quad, quadraticVertex, quadraticVertex, radians, random, random, randomGaussian, randomSeed, rect, rect, rect, rectMode, red, redraw, registerMethod, requestImage, requestImage, resetMatrix, resetShader, resetShader, resume, reverse, reverse, reverse, reverse, reverse, reverse, reverse, rotate, rotate, rotateX, rotateY, rotateZ, round, runSketch, runSketch, runSketch, saturation, save, saveBytes, saveBytes, saveBytes, saveFile, saveFrame, saveFrame, saveJSONArray, saveJSONArray, saveJSONObject, saveJSONObject, savePath, saveStream, saveStream, saveStream, saveStream, saveStream, saveStrings, saveStrings, saveStrings, saveTable, saveTable, saveXML, saveXML, scale, scale, scale, screenX, screenX, screenY, screenY, screenZ, second, selectCallback, selectFolder, selectFolder, selectFolder, selectInput, selectInput, selectInput, selectOutput, selectOutput, selectOutput, set, set, setMatrix, setMatrix, setMatrix, setSize, shader, shader, shape, shape, shape, shapeMode, shearX, shearY, shell, shininess, shorten, shorten, shorten, shorten, shorten, shorten, shorten, showDepthWarning, showDepthWarningXYZ, showMethodWarning, showMissingWarning, showSurface, showVariationWarning, sin, size, size, size, sketchDisplay, sketchFile, sketchFullScreen, sketchHeight, sketchOutputPath, sketchOutputStream, sketchPath, sketchPath, sketchPixelDensity, sketchRenderer, sketchSmooth, sketchWidth, sketchWindowColor, smooth, smooth, sort, sort, sort, sort, sort, sort, sort, sort, sort, sort, specular, specular, specular, sphere, sphereDetail, sphereDetail, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, split, split, splitTokens, splitTokens, spotLight, sq, sqrt, square, start, startSurface, str, str, str, str, str, str, str, str, str, str, stroke, stroke, stroke, stroke, stroke, stroke, strokeCap, strokeJoin, strokeWeight, style, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, tan, text, text, text, text, text, text, text, text, text, text, text, textAlign, textAlign, textAscent, textDescent, textFont, textFont, textLeading, textMode, textSize, texture, textureMode, textureWrap, textWidth, textWidth, textWidth, thread, tint, tint, tint, tint, tint, tint, translate, translate, triangle, trim, trim, unbinary, unhex, unregisterMethod, updatePixels, updatePixels, urlDecode, urlEncode, vertex, vertex, vertex, vertex, vertex, windowMove, windowMoved, windowRatio, windowResizable, windowResize, windowResized, windowTitle, year`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### pixelaudio

    PixelAudio pixelaudio

  - ### multigen

    MultiGen multigen

  - ### genWidth

    int genWidth

  - ### genHeight

    int genHeight

  - ### mapper

    PixelAudioMapper mapper

  - ### mapSize

    int mapSize

  - ### baseImage

    processing.core.PImage baseImage

  - ### mapImage

    processing.core.PImage mapImage

  - ### chan

    PixelAudioMapper.ChannelNames chan

  - ### spectrum

    int[] spectrum

  - ### audioFile

    File audioFile

  - ### audioFilePath

    String audioFilePath

  - ### audioFileName

    String audioFileName

  - ### audioFileTag

    String audioFileTag

  - ### audioFileLength

    int audioFileLength

  - ### imageFile

    File imageFile

  - ### imageFilePath

    String imageFilePath

  - ### imageFileName

    String imageFileName

  - ### imageFileTag

    String imageFileTag

  - ### imageFileWidth

    int imageFileWidth

  - ### imageFileHeight

    int imageFileHeight

  - ### minim

    ddf.minim.Minim minim
    Minim audio library

  - ### audioOut

    ddf.minim.AudioOutput audioOut

  - ### audioGain

    float audioGain

  - ### sampleRate

    float sampleRate

  - ### fileSampleRate

    float fileSampleRate

  - ### bufferSampleRate

    float bufferSampleRate

  - ### doResample

    boolean doResample

  - ### audioSignal

    float[] audioSignal

  - ### playBuffer

    ddf.minim.MultiChannelBuffer playBuffer

  - ### audioLength

    int audioLength

  - ### noteDuration

    int noteDuration

  - ### synth

    PASamplerInstrument synth

  - ### samplerGain

    float samplerGain

  - ### defaultEnv

    ADSRParams defaultEnv

  - ### maxAmplitude

    float maxAmplitude

  - ### attackTime

    float attackTime

  - ### decayTime

    float decayTime

  - ### sustainLevel

    float sustainLevel

  - ### releaseTime

    float releaseTime

  - ### adsrList

    ArrayList<ADSRParams> adsrList

  - ### isRandomADSR

    boolean isRandomADSR

  - ### pitchScaling

    float pitchScaling

  - ### defaultPitchScaling

    float defaultPitchScaling

  - ### lowPitchScaling

    float lowPitchScaling

  - ### highPitchScaling

    float highPitchScaling

  - ### timeLocsArray

    ArrayList<TimedLocation> timeLocsArray

  - ### count

    int count

  - ### fileIndex

    int fileIndex

  - ### shift

    int shift

  - ### shiftInc

    int shiftInc

  - ### totalShift

    int totalShift

  - ### isAnimating

    boolean isAnimating

  - ### oldIsAnimating

    boolean oldIsAnimating

  - ### animSteps

    int animSteps

  - ### isRecordingVideo

    boolean isRecordingVideo

  - ### videoFrameRate

    int videoFrameRate

  - ### step

    int step

  - ### videx

    com.hamoid.VideoExport videx

  - ### isRaining

    boolean isRaining

  - ### isShowOverlay

    boolean isShowOverlay

  - ### isPlayMusicBox

    boolean isPlayMusicBox

- ## Constructor Details

  - ### TutorialOne_02_Animation

    public TutorialOne_02_Animation()

- ## Method Details

  - ### main

    public static void main(String[] args)

  - ### settings

    public void settings()
    Overrides:
      `settings` in class `processing.core.PApplet`

  - ### setup

    public void setup()
    Overrides:
      `setup` in class `processing.core.PApplet`

  - ### stop

    public void stop()
    turn off audio processing when we exit
    Overrides:
      `stop` in class `processing.core.PApplet`

  - ### getColors

    public int[] getColors(int size)
    Generates an array of rainbow colors using the HSB color space.
    Parameters:
      `size` - the number of entries in the colors array
    Returns:
      an array of RGB colors ordered by hue

  - ### initImages

    public void initImages()
    Initializes mapImage with the colors array, copies mapImage to baseImage. MapImage handles the color data for mapper and also serves as our display image. BaseImage is intended as a reference image that typically only changes when you load a new image.

  - ### draw

    public void draw()
    Overrides:
      `draw` in class `processing.core.PApplet`

  - ### animate

    public void animate()

  - ### stepAnimation

    public void stepAnimation()
    Step through the animation, called by the draw() method. Will also record a frame of video, if we're recording.

  - ### renderFrame

    public void renderFrame(int step, boolean isInverseShift)
    Renders a frame of animation: moving along the signal path, copies baseImage pixels to mapImage pixels, adjusting the index position of the copy using totalShift i.e. we don't actually rotate the pixels, we just shift the position they're copied to
    Parameters:
      `isInverseShift` - boolean, if true shift is negative

  - ### renderFrame

    public void renderFrame(int step)

  - ### doRain

    public void doRain()
    drop some random audio events, like unto the gentle rain

  - ### writeToScreen

    public void writeToScreen(String msg, int x, int y, int weight, boolean isWhite)
    Displays a line of text to the screen, usually in the draw loop. Handy for debugging. typical call: writeToScreen("When does the mind stop and the world begin?", 64, 1000, 24, true);
    Parameters:
      `msg` - message to write
      `x` - x coordinate
      `y` - y coordinate
      `weight` - font weight
      `isWhite` - if true, white text, otherwise, black text

  - ### mousePressed

    public void mousePressed()
    The built-in mousePressed handler for Processing, not used yet...
    Overrides:
      `mousePressed` in class `processing.core.PApplet`

  - ### mouseClicked

    public void mouseClicked()
    Overrides:
      `mouseClicked` in class `processing.core.PApplet`

  - ### mouseDragged

    public void mouseDragged()
    Overrides:
      `mouseDragged` in class `processing.core.PApplet`

  - ### mouseReleased

    public void mouseReleased()
    Overrides:
      `mouseReleased` in class `processing.core.PApplet`

  - ### keyPressed

    public void keyPressed()
    built-in keyPressed handler, forwards events to parseKey.
    Overrides:
      `keyPressed` in class `processing.core.PApplet`

  - ### parseKey

    public void parseKey(char key, int keyCode)
    Handles key press events passed on by the built-in keyPressed method. By moving key event handling outside the built-in keyPressed method, we make it possible to post key commands without an actual key event. Methods and interfaces and even other threads can call parseKey(). This opens up many possibilities and a some dangers, too.
    Parameters:
      `key` -
      `keyCode` -

  - ### showHelp

    public void showHelp()
    to generate help output, run RegEx search/replace on parseKey case lines with: // case ('.'): // (.+) // println(" * Press $1 to $2.");

  - ### applyColor

    public int[] applyColor(int[] colorSource, int[] graySource, int[] lut)
    Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
    Parameters:
      `colorSource` - a source array of RGB data from which to obtain hue and saturation values
      `graySource` - an target array of RGB data from which to obtain brightness values
      `lut` - a lookup table, must be the same size as colorSource and graySource
    Returns:
      the graySource array of RGB values, with hue and saturation values changed
    Throws:
      `IllegalArgumentException` - if array arguments are null or if they are not the same length

  - ### applyColorShifted

    public int[] applyColorShifted(int[] colorSource, int[] graySource, int[] lut, int pixelShift)
    Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
    Parameters:
      `colorSource` - a source array of RGB data from which to obtain hue and saturation values
      `graySource` - an target array of RGB data from which to obtain brightness values
      `lut` - a lookup table, must be the same size as colorSource and graySource
      `pixelShift` - total amount of pixel shifting in the image
    Returns:
      the graySource array of RGB values, with hue and saturation values changed
    Throws:
      `IllegalArgumentException` - if array arguments are null or if they are not the same length

  - ### applyColorMapToDisplay

    public void applyColorMapToDisplay(boolean updateBaseImage)
    Apply color map hue and saturation to mapImage or baseImage.
    Parameters:
      `updateBaseImage` - if true, update baseImage, otherwise just update mapImage

  - ### chooseColorImage

    public void chooseColorImage()
    Call to initiate process of opening an image file to get its color data.

  - ### chooseColorImageAndStore

    public void chooseColorImageAndStore()

  - ### colorFileSelected

    public void colorFileSelected(File selectedFile)
    callback method for chooseColorImage()
    Parameters:
      `selectedFile` - the File the user selected

  - ### colorFileSelectedToStore

    public void colorFileSelectedToStore(File selectedFile)

  - ### applyImageColor

    public void applyImageColor(File imgFile, processing.core.PImage targetImage)
    Apply the hue and saturation of a chosen image file to the brightness channel of the display image.
    Parameters:
      `imgFile` - selected image file, source of hue and saturation values
      `targetImage` - target image where brightness will remain unchanged

  - ### chooseFile

    public void chooseFile()
    Wrapper method for Processing's selectInput command

  - ### fileSelected

    public void fileSelected(File selectedFile)
    callback method for chooseFile(), handles standard audio and image formats for Processing. If a file has been successfully selected, continues with a call to loadAudioFile() or loadImageFile().
    Parameters:
      `selectedFile` - the File the user selected

  - ### loadAudioFile

    public void loadAudioFile(File audFile)
    Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage As in most PixelAudio examples, we provide built-in resampling of audio data from the file to match the sampling rate of the buffer to audioOut.sampleOut().
    Parameters:
      `audFile` - an audio file

  - ### renderAudioToMapImage

    public void renderAudioToMapImage(PixelAudioMapper.ChannelNames chan, int shift)
    Render audio into mapImage at an arbitrary phase offset. In simple examples, shift will usually be 0. In later examples or in your own code, shifted renders may be layered, blended, or used as temporary performance/display operations without committing them to baseImage.

  - ### writeAudioToImage

    public void writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan, int shift)
    Transcodes audio data in sig[] and writes it to color channel chan of img using the lookup tables in mapper to redirect indexing. Calls mapper.mapSigToImgShifted(), which will throw an IllegalArgumentException if sig.length != img.pixels.length or sig.length != mapper.getSize().
    Parameters:
      `sig` - an array of float, should be audio data in the range [-1.0, 1.0]
      `mapper` - a PixelAudioMapper
      `img` - a PImage
      `chan` - a color channel
      `shift` - the number of indices to shift when writing audio

  - ### loadImageFile

    public void loadImageFile(File imgFile)
    Attempts to load image data from a selected file into mapImage, then calls writeImageToAudio() to transcode HSB brightness color data from the image to audio and writes it to playBuffer and audioSignal.
    Parameters:
      `imgFile` - an image file

  - ### writeImageToAudio

    public void writeImageToAudio(processing.core.PImage img, PixelAudioMapper mapper, float[] sig, PixelAudioMapper.ChannelNames chan, int shift)
    This method writes a color channel from an image to playBuffer, fulfilling a central concept of the PixelAudio library: image is sound. Calls mapper.mapImgToSig(), which will throw an IllegalArgumentException if img.pixels.length != sig.length or img.width * img.height != mapper.getWidth() * mapper.getHeight(). Sets totalShift = 0 on completion: the image and audio are now in sync. TODO
    Parameters:
      `img` - a PImage, a source of data
      `mapper` - a PixelAudioMapper, handles mapping between image and audio signal
      `sig` - an target array of float in audio format
      `chan` - a color channel
      `shift` - number of indices to shift

  - ### renderMapImageToAudio

    public void renderMapImageToAudio(PixelAudioMapper.ChannelNames chan)
    Writes a specified channel of mapImage to audioSignal.
    Parameters:
      `chan` - the selected color channel

  - ### commitMapImageToBaseImage

    public void commitMapImageToBaseImage()

  - ### commitNewBaseImage

    public void commitNewBaseImage(processing.core.PImage img)

  - ### refreshMapImageFromBase

    public void refreshMapImageFromBase()

  - ### saveToAudio

    public void saveToAudio()

  - ### audioFileSelectedWrite

    public void audioFileSelectedWrite(File selection)

  - ### saveAudioToFile

    public void saveAudioToFile(float[] samples, float sampleRate, String fileName) throws IOException, UnsupportedAudioFileException
    Saves audio data to 16-bit integer PCM format, which Processing can also open. This same method can be called as a static method in PixelAudio.
    Parameters:
      `samples` - an array of floats in the audio range (-1.0f, 1.0f)
      `sampleRate` - audio sample rate for the file
      `fileName` - name of the file to save to
    Throws:
      `IOException` - an Exception you'll need to handle to call this method
      `UnsupportedAudioFileException` - another Exception

  - ### saveToImage

    public void saveToImage()

  - ### imageFileSelectedWrite

    public void imageFileSelectedWrite(File selection)

  - ### saveImageToFile

    public void saveImageToFile(processing.core.PImage img, String fileName)

  - ### initAudio

    public void initAudio()
    CALL THIS METHOD IN SETUP() Initializes Minim audio library and audio variables.

  - ### ensureSamplerReady

    void ensureSamplerReady()
    Prepares Sampler instruments and assets

  - ### initADSRList

    public void initADSRList()

  - ### audioMouseClick

    public void audioMouseClick(int x, int y)
    Typically called from mousePressed with mouseX and mouseY, generates audio events.
    Parameters:
      `x` - x-coordinate within a PixelAudioMapper's width
      `y` - y-coordinate within a PixelAudioMapper's height

  - ### clipToWidth

    public int clipToWidth(int x)
    Parameters:
      `x` - a value to constrain to the current window width
    Returns:
      the constrained value

  - ### clipToHeight

    public int clipToHeight(int y)
    Parameters:
      `y` - a value to constrain to the current window height
    Returns:
      the constrained value

  - ### getSamplePos

    public int getSamplePos(int x, int y)
    Calculates the index of the image pixel within the signal path, taking the shifting of pixels and audioSignal into account. See the MusicWindowBox sketch for use of a windowed buffer in this calculation.
    Parameters:
      `x` - an x coordinate within mapImage and display bounds
      `y` - a y coordinate within mapImage and display bounds
    Returns:
      the index of the sample corresponding to (x,y) on the signal path

  - ### playSample

    public int playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pan)
    Plays an audio sample with PASamplerInstrument and custom ADSR.
    Parameters:
      `samplePos` - position of the sample in the audio buffer
      `samplelen` - length of the sample (will be adjusted)
      `amplitude` - amplitude of the sample on playback
      `env` - an ADSRParams envelope for the sample
    Returns:
      the calculated sample length in samples

  - ### playSample

    public int playSample(int samplePos, int samplelen, float amplitude, float pan)
    Plays an audio sample with PASamplerInstrument and default ADSR.
    Parameters:
      `samplePos` - position of the sample in the audio buffer
      `samplelen` - length of the sample (will be adjusted)
      `amplitude` - amplitude of the sample on playback
    Returns:
      the calculated sample length in samples

  - ### calcSampleLen

    public int calcSampleLen()

  - ### updateAudioChain

    void updateAudioChain(float[] sig, float bufferSampleRate)
    Bottleneck "commit" method for audio state. Takes an arbitrary input signal and installs it as the canonical audio signal used by the application. This method: - Resizes/pads/truncates the input to mapper.getSize() - Copies the data to ensure no external aliasing - Updates audioSignal (canonical signal handled by application code) - Updates playBuffer (audio buffer used by Minim audio library methods) - Propagates the buffer to active instruments: edit for your own instruments This is the ONLY method that should mutate the global audio signal state. In PixelAudio examples, the signal is typically loaded from a file, but it could also be signal cached in memory, a signal generated by code, audio captured live, etc.
    Parameters:
      `sig` - an audio signal
      `bufferSampleRate` - audio sample rate for sig, usually obtained when reading an audio file

  - ### updateAudioChain

    void updateAudioChain(float[] sig)

  - ### runTimeArray

    public void runTimeArray()
    Run the animation for audio events.

  - ### drawCircle

    public void drawCircle(int x, int y)
    Draws a circle at the location of an audio trigger (mouseDown event).
    Parameters:
      `x` - x coordinate of circle
      `y` - y coordinate of circle