Package net.paulhertz.pixelaudio.example

Class TutorialOne_04_Network

java.lang.Object
processing.core.PApplet
net.paulhertz.pixelaudio.example.TutorialOne_04_Network
All Implemented Interfaces:
PANetworkClientINF, processing.core.PConstants
public class TutorialOne_04_Network extends processing.core.PApplet implements PANetworkClientINF

QUICK START

  1. Launch TutorialOne_03_Drawing. The display window opens with a pre-loaded file, "Saucer_mixdown.wav". The audio data is displayed as grayscale values. A spectrum of rainbow colors overlaid on the image follows the Signal Path, the mapping of the audio signal to the image pixels created by the PixelMapGen multigen and managed by the PixelAudioMapper mapper. This particular PixelMapGen reads time from top to bottom, left to right, in eight rows.
  2. Click on the image or press the spacebar with the cursor over the image to play a Sampler instrument sound.
  3. Press 'd' to enable drawing. Drag the mouse to draw a line. Try varying the speed of your gesture: it will be recorded to the PACurveMaker instance that records your actions. When you release the mouse, a brushstroke appears.
  4. Click on the brushstroke. The Sampler instrument plays new brushstokes, by default, using the ALL_POINTS representation of the curve. Press the '1', '2' or '3' key to change the representation of the curve. REDUCED_POINTS (2) draws fewer points. CURVE_POINTS (3) draws many more. The Sampler instrument sounds good with all points or reduced points, mostly because it has a fairly long envelope. It may sound too dense with curve points.
  5. Draw a few more brushstrokes to experiment with drawing and the Sampler instrument. Draw fast and slow, right to left or left to right (forwards or backwards in time), vertically, horizontally, or diagonally.
  6. To switch a brushstroke to use the Granular instrument, hover over it and press 't'. The representation of the curve will change to CURVE_POINTS. Click on the curve to play it. Experiment with the other representations. The curve steps representation of the brushstroke provides enough density to provide a continuous sound for the short envelopes of grains. The reduced points representation may sound too sparse. Press 't' again to switch the curve back to use the Sampler instrument.
See the various key commands for various ways to alter the sound of the granular synth and other features. Check out the JavaDocs comments code comments on the various methods for detailed information about the features of TutorialOne_03_Drawing. For an GUI with greater control over drawing, gesture, and audio synthesis, see the GesturePlayground sketch.

NEW FEATURES

This sketch continues the Tutorial One sequence for the PixelAudio library for Processing. To the previous tools for reading, writing, and transcoding audio and image files, triggering audio events, and animating pixels to change audio sample order, it adds the capability of drawing in the display window. Lines drawn on the display are captured as points and times and used to create brushstrokes that can be activated to trigger audio events. In PixelAudio's conceptual framework, the combination of points and times constitutes a "gesture". Along with mapping of points to audio samples using PixelAudioMapper, "gesture" is a core concept of the PixelAudio library. Check out PAGesture for a formal definition of gesture.

PixelAudio provides two types of audio synthesis: the Sampler intruments, introduced in previous sketches, and the Granular instruments. This sketch introduces PixelAudio's granular synthesis engine with the PAGranularInstrumentDirector class and provides another class for Sampler instruments, PASamplerInstrumentPool. The Granular instruments and the Sampler instruments both depend on a hierarchy of classes that implement a signal-processing chain. It's unlikely that you will have to deal with the low-level processing. PAGranularInstrumentDirector and PASamplerInstrumentPool provide the control knobs you would want for a virtual electronic instrument while keeping the math and audio engine hooks in the background. This sketch and GesturePlayground provide a good introduction to the functions available in the granular synth. GesturePlayground goes into more detail than TutorialOne_03_Drawing, and provides a GUI to tweak almost every Granular instrument feature. In TutorialOne_03_Drawing we are principally concerned with showing how to create interactive brushstrokes by drawing on the screen.

Points + Times = Gestures

The drawing tools and commands are a substantial addition. To implement them we call on a whole new package of code, PixelAudio Curves Package. To turn gestures into timing information that can be used to schedule audio events, particularly with granular synthesis, we rely on the PixelAudio Schedule Package. The workhorse of the Curves package is PACurveMaker, which is used to * capture point and time information when you draw. A PACurveMaker instance is intialized with a list of unique points and a list of time offsets where both lists have the same number of elements. Each point is paired to the relative time when it was recorded. Points drawn on the display stored in PACurveMaker's ALL_POINTS representation of the gesture points. PACurveMaker can also reduce the number of points captured, using the Ramer-Douglas-Peucker (RDP) algorithm, to create the REDUCED_POINTS representation of a gesture. RDP controls point reduction with a numerical value, epsilon, which you can vary to control the number of reduced points in a gesture. PACurveMaker can turn the reduced points representation of a drawn line into a Bezier curve, the CURVE_POINTS representation of the gesture. The curve can be divided polygonal segments, with the potential to generate an audio event at each vertex. The number of divisions is controlled by the PACurveMaker.setCurveSteps(int curveSteps) method. In the GesturePlayground sketch you can vary the curve divisions with the GUI.

The CURVE_POINTS curve is used to create a stylized brushstroke. The brushstroke is an PABezShape object. PABezShape provides a pointInPoly() method that you can use to detect the mouse hovering over or clicking within a brushstroke. TutorialOne_03_Drawing shows how the brushstroke can be activated as an animated UI element and used to trigger audio events.

Audio Processing

The PAGranularInstrumentDirector class manages the high level processes for granular synthesis. Probably all the functionality you will commonly need is available in its methods. The various playGestureNow(...) methods allow you to control the timing, panning, pitch, and gain of individual grains, if you want to. Parameters for grain shaping and are set with the GestureGranularParams class. The GestureEventParams class supports arrays of values to set timing, pan, gain, and pitch for individual grains. The timing and pan settings for individual grains can also be passed as arrays to overloaded playGestureNow(...) methods.

Here's an outline of the granular synbthesis chain, which you can feel free to skip: PAGranularInstrumentDirector sets up the PABurstGranularSource and passes it to the granular synth processing chain, first to PAGranularInstrument and then to PAGranularSampler, where the PABurstGranularSource is one of various parameters used to create an AudioScheduler scheduler. In PAGranularSampler.uGenerate(), scheduler.processBlock() is called, the audio signal leftMix and rightMix variables are initialized to 0 and used to accumulate the currently active PAGranularVoice instances. After that, uGenerate applies power normalization and soft clipping to the signal and returns its value. All of this is set in motion through Minim's UGen interface, which PAGranularSampler extends. 

 Here are the key commands for this sketch:
 
 Press ' ' to spacebar triggers a brush if we're hovering over a brush, otherwise it triggers a point event.
 Press '1' to set brushstroke under cursor to PathMode ALL_POINTS.
 Press '2' to set brushstroke under cursor to PathMode REDUCED_POINTS.
 Press '3' to set brushstroke under cursor to PathMode CURVE_POINTS.
 Press 't' to set brushstroke under cursor to use Sampler or Granular synth.
 Press 'f' to set brushstroke under cursor to FIXED hop mode for granular events.
 Press 'g' to set brushstroke under cursor to GESTURE hop mode for granular events.
 Press 'e' to select granular or sampler synth for click response.
 Press 'r' to select long or short grain duration .
 Press 'p' to jitter the pitch of granular gestures.
 Press 'a' to  start or stop animation (bitmap rotation along the signal path).
 Press 'd' to turn drawing on or off.
 Press 'c' to apply color from image file to display image.
 Press 'k' to apply the hue and saturation in the colors array to mapImage (shifted when animating).
 Press 'K' to color display with spectrum and write to base image.
 Press 'j' to turn audio and image blending on or off.
 Press 'n' to normalize the audio buffer to -6.0dB.
 Press 'l' or 'L' to determine whether to load a new file to both image and audio or not.
 Press 'o' or 'O' to open an audio or image file and load to image or audio buffer or both.
 Press 'w' to write the image HSB Brightness channel to the audio buffer as transcoded sample values .
 Press 'W' to write the audio buffer samples to the image as color values.
 Press 'x' to delete the current active brush shape or the oldest brush shape.
 Press 'X' to delete the most recent brush shape.
 Press 'u' to mute audio.
 Press 'V' to record a video.
 Press 'h' or 'H' to show help message in the console.
REVISIONS Rendering rule: mapper.mapSigToImgShifted(audioSignal, mapImage.pixels, chan, totalShift); Click rule: int signalPos = mapper.lookupSignalPosShifted(x, y, totalShift);

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    TutorialOne_04_Network.AudioBrushLite
    A light-weight version of the AudioBrush class for combining gestures and audio synthesis.
    static final class 
    TutorialOne_04_Network.BrushConfig
    Defines the curve drawing model for a brush using PACurveMaker.
    static final class 
    TutorialOne_04_Network.BrushHit
     
    static enum 
    TutorialOne_04_Network.BrushOutput
     
    static enum 
    TutorialOne_04_Network.HopMode
     
    class 
    TutorialOne_04_Network.NetworkDelegate
    A class to handle network connections over UDP, for example, with a Max or Pd patch.
    static enum 
    TutorialOne_04_Network.PathMode
     

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    TutorialOne_04_Network.AudioBrushLite
    activeBrush
     
    ArrayList<processing.core.PVector>
    allPoints
     
    ArrayList<Integer>
    allTimes
     
    (package private) int
    animatedCircleColor
     
    (package private) File
    audioFile
     
    (package private) int
    audioFileLength
     
    (package private) String
    audioFileName
     
    (package private) String
    audioFilePath
     
    (package private) String
    audioFileTag
     
    (package private) int
    audioLength
     
    (package private) ddf.minim.AudioOutput
    audioOut
     
    (package private) float[]
    audioSignal
     
    (package private) processing.core.PImage
    baseImage
     
    ArrayList<TutorialOne_04_Network.AudioBrushLite>
    brushes
     
    (package private) float
    bufferSampleRate
     
    (package private) int
    burstGrains
     
    (package private) PixelAudioMapper.ChannelNames
    chan
     
    (package private) int
    circleColor
     
    (package private) int[]
    colors
     
    processing.core.PVector
    currentPoint
     
    (package private) PACurveMaker
    curveMaker
     
    int
    curveSteps
     
    (package private) String
    daPath
     
    (package private) ADSRParams
    defaultEnv
     
    (package private) float
    defaultPitchScale
     
    (package private) int
    dimCircleColor
     
    (package private) int
    dimLineColor
     
    int
    dragColor
     
    float
    dragWeight
     
    float
    epsilon
     
    (package private) int
    eventStep
     
    (package private) float
    fileSampleRate
     
    PAGranularInstrumentDirector
    gDir
     
    (package private) int
    genHeight
     
    (package private) int
    genWidth
     
    int
    gMaxVoices
     
    (package private) GestureGranularParams
    gParamsFixed
     
    (package private) GestureGranularParams
    gParamsGesture
     
    (package private) ArrayList<TimedLocation>
    grainTimeLocs
     
    (package private) int
    granEnvDuration
     
    (package private) int
    granSamples
     
    float[]
    granSignal
     
    float
    granularGain
     
    float
    granularPointGain
     
    PAGranularInstrument
    gSynth
     
    (package private) float
    highPitchScaling
     
    (package private) int
    hopSamples
     
    TutorialOne_04_Network.AudioBrushLite
    hoverBrush
     
    (package private) int
    hoverBrushColor
     
    (package private) int
    hoverBrushColor1
     
    (package private) int
    hoverBrushColor2
     
    int
    hoverIndex
     
    (package private) File
    imageFile
     
    (package private) int
    imageFileHeight
     
    (package private) String
    imageFileName
     
    (package private) String
    imageFilePath
     
    (package private) String
    imageFileTag
     
    (package private) int
    imageFileWidth
     
    (package private) boolean
    isAnimating
     
    (package private) boolean
    isBlending
     
    (package private) boolean
    isBufferStale
     
    boolean
    isDrawMode
     
    (package private) boolean
    isIgnoreOutsideBounds
     
    (package private) boolean
    isLoadToBoth
     
    (package private) boolean
    isMuted
     
    (package private) boolean
    isRecordingVideo
     
    (package private) boolean
    isUseNetworkDelegate
     
    (package private) int
    lineColor
     
    (package private) int
    longSample
     
    (package private) float
    lowPitchScaling
     
    (package private) processing.core.PImage
    mapImage
     
    (package private) PixelAudioMapper
    mapper
     
    (package private) int
    mapSize
     
    (package private) float
    maxAmplitude
     
    (package private) int
    maxBurstGrains
     
    (package private) ddf.minim.Minim
    minim
     
    (package private) MultiGen
    multigen
     
    (package private) TutorialOne_04_Network.NetworkDelegate
    nd
     
    (package private) int
    noteDuration
     
    (package private) boolean
    oldIsAnimating
     
    (package private) float
    outputGain
     
    (package private) float
    pitchScaling
     
    (package private) PixelAudio
    pixelaudio
     
    (package private) ddf.minim.MultiChannelBuffer
    playBuffer
     
    (package private) ArrayList<TimedLocation>
    pointTimeLocs
     
    int
    polySteps
     
    (package private) PASamplerInstrumentPool
    pool
     
    (package private) int
    readyBrushColor
     
    (package private) int
    readyBrushColor1
     
    (package private) int
    readyBrushColor2
     
    (package private) int
    samplelen
     
    (package private) int
    samplePos
     
    (package private) float
    sampleRate
     
    (package private) int
    samplerMaxVoices
     
    ArrayList<TimedLocation>
    samplerTimeLocs
     
    (package private) int
    selectedBrushColor
     
    (package private) int
    selectedBrushColor1
     
    (package private) int
    selectedBrushColor2
     
    (package private) int
    shift
     
    (package private) int
    shortSample
     
    int
    startTime
     
    (package private) int
    step
     
    (package private) float
    synthGain
     
    (package private) float
    synthPointGain
     
    (package private) int
    totalShift
     
    (package private) boolean
    useGranularSynth
     
    (package private) boolean
    useLongBursts
     
    (package private) boolean
    usePitchedGrains
     
    (package private) boolean
    useShortGrain
     
    (package private) int
    videoFrameRate
     
    (package private) int
    videoSteps
     
    (package private) com.hamoid.VideoExport
    videx
     

    Fields inherited from class processing.core.PApplet

    args, ARGS_BGCOLOR, ARGS_DISABLE_AWT, ARGS_DISPLAY, ARGS_EDITOR_LOCATION, ARGS_EXTERNAL, ARGS_FULL_SCREEN, ARGS_HIDE_STOP, ARGS_LOCATION, ARGS_PRESENT, ARGS_SKETCH_FOLDER, ARGS_STOP_COLOR, ARGS_UI_SCALE, ARGS_WINDOW_COLOR, DEFAULT_HEIGHT, DEFAULT_WIDTH, disableAWT, displayHeight, displayWidth, dmouseX, dmouseY, emouseX, emouseY, exitCalled, EXTERNAL_MOVE, EXTERNAL_STOP, finished, firstMouse, focused, frameCount, frameRate, frameRateLastNanos, g, height, insideDraw, javaPlatform, javaVersion, javaVersionName, key, keyCode, keyEvent, keyPressed, keyRepeatEnabled, looping, matchPatterns, mouseButton, mouseEvent, mousePressed, mouseX, mouseY, pixelDensity, pixelHeight, pixels, pixelWidth, platform, pmouseX, pmouseY, ratioLeft, ratioScale, ratioTop, recorder, redraw, rheight, rmouseX, rmouseY, rwidth, surface, useNativeSelect, width, windowX, windowY

    Fields inherited from interface processing.core.PConstants

    ADD, ALPHA, ALT, AMBIENT, ARC, ARGB, ARROW, BACKSPACE, BASELINE, BEVEL, BEZIER_VERTEX, BLEND, BLUR, BOTTOM, BOX, BREAK, BURN, CENTER, CHATTER, CHORD, CLAMP, CLOSE, CODED, COMPLAINT, CONTROL, CORNER, CORNERS, CROSS, CURVE_VERTEX, CUSTOM, DARKEST, DEG_TO_RAD, DELETE, DIAMETER, DIFFERENCE, DILATE, DIRECTIONAL, DISABLE_ASYNC_SAVEFRAME, DISABLE_BUFFER_READING, DISABLE_DEPTH_MASK, DISABLE_DEPTH_SORT, DISABLE_DEPTH_TEST, DISABLE_KEY_REPEAT, DISABLE_NATIVE_FONTS, DISABLE_OPENGL_ERRORS, DISABLE_OPTIMIZED_STROKE, DISABLE_STROKE_PERSPECTIVE, DISABLE_STROKE_PURE, DISABLE_TEXTURE_MIPMAPS, DODGE, DOWN, DXF, ELLIPSE, ENABLE_ASYNC_SAVEFRAME, ENABLE_BUFFER_READING, ENABLE_DEPTH_MASK, ENABLE_DEPTH_SORT, ENABLE_DEPTH_TEST, ENABLE_KEY_REPEAT, ENABLE_NATIVE_FONTS, ENABLE_OPENGL_ERRORS, ENABLE_OPTIMIZED_STROKE, ENABLE_STROKE_PERSPECTIVE, ENABLE_STROKE_PURE, ENABLE_TEXTURE_MIPMAPS, ENTER, EPSILON, ERODE, ESC, EXCLUSION, FX2D, GIF, GRAY, GROUP, HALF_PI, HAND, HARD_LIGHT, HINT_COUNT, HSB, IMAGE, INVERT, JAVA2D, JPEG, LANDSCAPE, LEFT, LIGHTEST, LINE, LINE_LOOP, LINE_STRIP, LINES, LINUX, MACOS, MACOSX, MAX_FLOAT, MAX_INT, MIN_FLOAT, MIN_INT, MITER, MODEL, MODELVIEW, MOVE, MULTIPLY, NORMAL, OPAQUE, OPEN, OPENGL, ORTHOGRAPHIC, OTHER, OVERLAY, P2D, P3D, PATH, PDF, PERSPECTIVE, PI, PIE, platformNames, POINT, POINTS, POLYGON, PORTRAIT, POSTERIZE, PROBLEM, PROJECT, PROJECTION, QUAD, QUAD_BEZIER_VERTEX, QUAD_STRIP, QUADRATIC_VERTEX, QUADS, QUARTER_PI, RAD_TO_DEG, RADIUS, RECT, REPEAT, REPLACE, RETURN, RGB, RIGHT, ROUND, SCREEN, SHAPE, SHIFT, SOFT_LIGHT, SPAN, SPHERE, SPOT, SQUARE, SUBTRACT, SVG, TAB, TARGA, TAU, TEXT, THIRD_PI, THRESHOLD, TIFF, TOP, TRIANGLE, TRIANGLE_FAN, TRIANGLE_STRIP, TRIANGLES, TWO_PI, UP, VERTEX, WAIT, WHITESPACE, WINDOWS, X, Y, Z

  • Constructor Summary

    Constructors
    Constructor
    Description
    TutorialOne_04_Network()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addPoint(int x, int y)
    While user is dragging the mouses and isDrawMode == true, accumulates new points to allPoints and event times to allTimes.
    void
    animate()
     
    int[]
    applyColor(int[] colorSource, int[] graySource, int[] lut)
    Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
    int[]
    applyColorShifted(int[] colorSource, int[] graySource, int[] lut, int shift)
    Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing, taking into account any pixels that were shifted.
    void
    applyImageColor(File imgFile, processing.core.PImage targetImage)
    Apply the hue and saturation of a chosen image file to the brightness channel of the display image.
    void
    audioFileSelectedWrite(File selection)
     
    void
    audioMousePressed(int x, int y)
    Handles mouse clicks that happen outside a brushstroke.
    static void
    blendInto(ddf.minim.MultiChannelBuffer dest, ddf.minim.MultiChannelBuffer src, float weight, boolean normalize, float targetDB)
    Blends audio data from buffer "src" into buffer "dest" in place.
    PAGranularInstrument
    buildGranSynth(ddf.minim.AudioOutput out, ADSRParams env, int numVoices)
    Initializes a new PAGranularSynth instance that you probably would pass to a PAGranularInstrumentDirector.
    int
    calcSampleLen()
     
    ADSRParams
    calculateEnvelopeDb(float gainDb, int totalSamples, float sampleRate)
    Calculate an envelope of length totalSamples.
    ADSRParams
    calculateEnvelopeLinear(float gainDb, float totalMs)
    Calculate an envelope of length totalSamples.
    void
    chooseColorImage()
    Call to initiate process of opening an image file to get its color data.
    void
    chooseFile()
    Wrapper method for Processing's selectInput command
    int
    clipToHeight(int i)
    Clips parameter i to the interval (0..width-1)
    int
    clipToWidth(int i)
    Clips parameter i to the interval (0..width-1)
    void
    colorFileSelected(File selectedFile)
    callback method for chooseColorImage()
    void
    commitMapImageToBaseImage()
    Writes the mapImage, which may change with animation, to the baseImage, a reference image that usually only changes when a new file is loaded.
    void
    commitNewBaseImage(processing.core.PImage img)
    Copies the supplied PImage to mapImage and baseImage, sets totalShift to 0 (the images are identical).
    void
    controlMsg(String control, float val)
     
    (package private) static void
    debugIndexHeadroom(float[] buf, int[] startIndices, GestureGranularParams ggp)
     
    (package private) static void
    debugTimesMs(GestureSchedule s)
     
    (package private) TutorialOne_04_Network.PathMode
    defaultPathModeFor(TutorialOne_04_Network.BrushOutput out)
    Determines the path mode for a particular BrushOutput.
    void
    draw()
     
    void
    drawBrushes(List<TutorialOne_04_Network.AudioBrushLite> list, int readyColor, int hoverColor, int selectedColor)
    Draw brushstrokes on the display image.
    void
    drawBrushShapes()
    Enrty point for drawing brushstrokes on the screen.
    void
    drawCircle(int x, int y)
    Draws a circle at the location of an audio trigger (mouseDown event).
    (package private) void
    ensureGranularReady()
    Ensures that all resources and variable necessary for the Granular synth are ready to go.
    (package private) void
    ensureSamplerReady()
    Ensures that all resources and variable necessary for the Sampler synth are ready to go.
    void
    fileSelected(File selectedFile)
    callback method for chooseFile(), handles standard audio and image formats for Processing.
    (package private) TutorialOne_04_Network.BrushHit
    findHoverHit()
     
    (package private) float[]
    generateJitterPitch(int length, float deviationPitch)
    Generates an array of Gaussian values for shifting pitch, where 1.0 = no shift.
    (package private) int
    getBrushIndex(TutorialOne_04_Network.AudioBrushLite brush)
     
    int[]
    getColors(int size)
    Generates an array of rainbow colors using the HSB color space.
    processing.core.PVector
    getCoordFromSignalPos(int pos)
    Calculates the display image coordinates corresponding to a specified audio sample index.
    PixelAudioMapper
    getMapper()
     
    processing.core.PApplet
    getPApplet()
     
    (package private) ArrayList<processing.core.PVector>
    getPathPoints(TutorialOne_04_Network.AudioBrushLite b)
    Get the path points of a brushstroke, with the representation determined by the BrushConfig's path mode.
    int
    getSamplePos(int x, int y)
    Calculates the index of the image pixel within the signal path, taking the shifting of pixels and audioSignal into account.
    GestureSchedule
    getScheduleForBrush(TutorialOne_04_Network.AudioBrushLite b)
    Get a GestureSchedule (points + timing) for an AudioBrushLite instance.
    int
    handleClickOutsideBrush(int x, int y)
     
    void
    handleDrawing()
    Handles user's drawing actions: draws previously recorded brushstrokes, tracks and generates interactive animation and audio events.
    void
    imageFileSelectedWrite(File selection)
     
    void
    initAllPoints()
    Initializes allPoints and adds the current mouse location to it.
    void
    initAudio()
    CALL THIS METHOD IN SETUP() Initializes Minim audio library and audio variables.
    void
    initCurveMakerAndAddBrush()
    Initializes a PACurveMaker instance with allPoints as an argument to the factory method PACurveMaker.buildCurveMaker() and then fills in PACurveMaker instance variables from variables in the calling class (TutorialOneDrawing, here).
    void
    initDrawing()
    Initializes drawing and drawing interaction variables.
    void
    initGranularParams()
    Initializes global variables gParamsGesture and gParamsFixed, which provide basic settings for granular synthesis the follows gesture timing or fixed hop timing between grains.
    void
    initImages()
    Initializes mapImage with the colors array.
    processing.core.PVector
    jitterCoord(int x, int y, int deviationPx)
     
    void
    keyPressed()
    Built-in keyPressed handler, forwards events to parseKey.
    void
    loadAudioFile(File audFile)
    Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage.
    void
    loadImageFile(File imgFile)
    Attempts to load image data from a selected file into mapImage, then calls writeImageToAudio() to transcode HSB brightness channel to audio and writes it to playBuffer and audioSignal.
    static void
    main(String[] args)
     
    void
    mouseClicked()
     
    void
    mouseDragged()
     
    boolean
    mouseInPoly(ArrayList<processing.core.PVector> poly)
     
    void
    mousePressed()
    The built-in mousePressed handler for Processing, clicks are handled in mouseClicked().
    void
    mouseReleased()
     
    static void
    normalize(float[] signal, float targetPeakDB)
    Normalizes a single-channel signal array to a target RMS level in dBFS (decibels relative to full scale).
    void
    parseKey(char key, int keyCode)
    Handles key press events passed on by the built-in keyPressed method.
    void
    playGranularGesture(float[] buf, GestureSchedule sched, GestureGranularParams params)
    Calls PAGranularInstrumentDirector gDir to play a granular audio event.
    void
    playPoints(ArrayList<processing.core.PVector> pts)
     
    int
    playSample(int samplePos)
     
    int
    playSample(int samplePos, int samplelen, float amplitude, float pan)
    Plays an audio sample with default envelope and stereo pan.
    int
    playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pan)
    Plays an audio sample with a custom envelope and stereo pan.
    int
    playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pitch, float pan)
    Plays an audio sample with with a custom envelope, pitch and stereo pan.
    void
    preloadFiles(String path, String filename)
    Preload an audio file using a file path and a filename.
    void
    refreshMapImageFromBase()
    Writes baseImage to mapImage with an index position offset of totalShift.
    void
    removeHoveredOrOldestBrush()
    Removes the current active AudioBrushLite instance.
    void
    removeNewestBrush()
    Removes the most recent AudioBrushLite instance.
    void
    renderAudioToMapImage(PixelAudioMapper.ChannelNames chan, int shift)
    Transcodes audio data in audioSignal and writes it to color channel chan of mapImage.
    void
    renderFrame(int step)
    Renders a frame of animation: moving along the signal path, copies baseImage pixels to mapImage pixels, adjusting the index position of the copy using totalShift -- i.e.
    void
    renderMapImageToAudio(PixelAudioMapper.ChannelNames chan)
    Writes a specified channel of mapImage to audioSignal.
    void
    reset()
    Deprecated.
    void
    runGrainEvents()
    Tracks and runs TimedLocation events in the grainLocsArray list, which is associated with granular synthesis gestures.
    void
    runPointEvents()
    Tracks and runs TimedLocation events in the timeLocsArray list, which is associated with mouse clicks that trigger audio a the click point.
    void
    runSamplerBrushEvents()
    Execute audio / animation events for Sampler brushstrokes.
    void
    saveAudioToFile(float[] samples, float sampleRate, String fileName)
    Saves audio data to 16-bit integer PCM format, which Processing can also open.
    void
    saveImageToFile(processing.core.PImage img, String fileName)
     
    void
    saveToAudio()
    Calls Processing's selectOutput method to start the process of saving the current audio signal to a .wav file.
    void
    saveToImage()
    Calls Processing's selectOutput method to start the process of saving the mapImage (the offscreen copy of the display image) to a .png file.
    (package private) void
    scheduleGranularBrushClick(TutorialOne_04_Network.AudioBrushLite b)
    Schedule a Granular brush audio / animation event.
    (package private) void
    scheduleSamplerBrushClick(TutorialOne_04_Network.AudioBrushLite b)
    Schedule a Sampler brush audio / animation event.
    static int
    setAlpha(int argb, int alpha)
    Sets the alpha channel of an RGBA color.
    int
    setAlphaWithBlack(int argb, int alpha)
    Sets the alpha channel of an RGBA color, conditionally setting alpha = 0 if all other channels = 0.
    void
    setAudioGain(float g)
    Sets audioOut.gain.
    void
    setBrushEpsilon(TutorialOne_04_Network.AudioBrushLite b, float e)
    Sets epsilon value for the PACurveMaker associated with an AudioBrushLite instance.
    void
    settings()
     
    void
    setup()
     
    void
    showHelp()
    to generate help output, run RegEx search/replace on parseKey case lines with: // case ('.'): // (.+) // println(" * Press $1 to $2.");
    void
    stepAnimation()
    Step through the animation, called by the draw() method.
    void
    stop()
    turn off audio processing when we exit
    void
    storeGranularCurveTL(GestureSchedule sched, int startTime, boolean isGesture)
    Store scheduled granular synth / animation events for future activation.
    void
    storeSamplerCurveTL(GestureSchedule sched, int startTime)
    Store scheduled sampler synth / animation events for future activation.
    (package private) void
    updateAudioChain(float[] sig)
    Updates resources such as playBuffer and pool with a new signal, typcically when a new file is loaded.
    (package private) void
    updateHover()
    Update the hoverBrush and hoverIndex global variables.
    void
    writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan, int shift)
    Transcodes audio data in sig[] and writes it to color channel chan of img using the lookup tables in mapper to redirect indexing.
    void
    writeImageToAudio(processing.core.PImage img, PixelAudioMapper mapper, float[] sig, PixelAudioMapper.ChannelNames chan, int shift)
    This method writes a color channel from an image to playBuffer, fulfilling a central concept of the PixelAudio library: image is sound.
    void
    writeToScreen(String msg, int x, int y, int weight, boolean isWhite)
    Displays a line of text to the screen, usually in the draw loop.

    Methods inherited from class processing.core.PApplet

    abs, abs, acos, alpha, ambient, ambient, ambient, ambientLight, ambientLight, append, append, append, append, append, append, applyMatrix, applyMatrix, applyMatrix, applyMatrix, applyMatrix, arc, arc, arraycopy, arraycopy, arraycopy, arrayCopy, arrayCopy, arrayCopy, asin, atan, atan2, attrib, attrib, attrib, attribColor, attribNormal, attribPosition, background, background, background, background, background, background, background, beginCamera, beginContour, beginPGL, beginRaw, beginRaw, beginRecord, beginRecord, beginShape, beginShape, bezier, bezier, bezierDetail, bezierPoint, bezierTangent, bezierVertex, bezierVertex, binary, binary, binary, binary, blend, blend, blendColor, blendMode, blue, box, box, brightness, calcSketchPath, camera, camera, ceil, checkAlpha, checkExtension, choice, choice, circle, clear, clip, color, color, color, color, color, color, color, color, colorMode, colorMode, colorMode, colorMode, concat, concat, concat, concat, concat, concat, concat, constrain, constrain, copy, copy, copy, cos, createFont, createFont, createFont, createGraphics, createGraphics, createGraphics, createImage, createInput, createInput, createInputRaw, createOutput, createOutput, createPath, createPath, createPrimaryGraphics, createReader, createReader, createReader, createShape, createShape, createShape, createWriter, createWriter, createWriter, cursor, cursor, cursor, cursor, curve, curve, curveDetail, curvePoint, curveTangent, curveTightness, curveVertex, curveVertex, dataFile, dataPath, day, debug, degrees, delay, dequeueEvents, desktopFile, desktopPath, die, die, directionalLight, displayDensity, displayDensity, dispose, dist, dist, edge, ellipse, ellipseMode, emissive, emissive, emissive, endCamera, endContour, endPGL, endRaw, endRecord, endShape, endShape, exec, exec, exit, exitActual, exitCalled, exp, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, fill, fill, fill, fill, fill, fill, filter, filter, filter, floor, flush, focusGained, focusLost, frameMoved, frameRate, frustum, fullScreen, fullScreen, fullScreen, fullScreen, get, get, get, getExtension, getGraphics, getMatrix, getMatrix, getMatrix, getSurface, green, handleDraw, handleKeyEvent, handleMethods, handleMouseEvent, hex, hex, hex, hex, hideMenuBar, hint, hour, hue, image, image, image, imageMode, initSurface, insertFrame, isLooping, join, join, keyPressed, keyReleased, keyReleased, keyTyped, keyTyped, launch, lerp, lerpColor, lerpColor, lightFalloff, lights, lightSpecular, line, line, link, listFiles, listFiles, listPaths, loadBytes, loadBytes, loadBytes, loadFont, loadImage, loadImage, loadJSONArray, loadJSONArray, loadJSONObject, loadJSONObject, loadPixels, loadShader, loadShader, loadShape, loadShape, loadStrings, loadStrings, loadStrings, loadStrings, loadTable, loadTable, loadXML, loadXML, log, loop, mag, mag, main, main, main, makeGraphics, map, mask, match, matchAll, max, max, max, max, max, max, method, millis, min, min, min, min, min, min, minute, modelX, modelY, modelZ, month, mouseClicked, mouseDragged, mouseEntered, mouseEntered, mouseExited, mouseExited, mouseMoved, mouseMoved, mousePressed, mouseReleased, mouseWheel, mouseWheel, nf, nf, nf, nf, nf, nf, nfc, nfc, nfc, nfc, nfp, nfp, nfp, nfp, nfs, nfs, nfs, nfs, noClip, noCursor, noFill, noise, noise, noise, noiseDetail, noiseDetail, noiseSeed, noLights, noLoop, norm, normal, noSmooth, noStroke, noTexture, noTint, orientation, ortho, ortho, ortho, parseBoolean, parseBoolean, parseBoolean, parseBoolean, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseChar, parseChar, parseChar, parseChar, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseJSONArray, parseJSONObject, parseXML, parseXML, pause, perspective, perspective, pixelDensity, point, point, pointLight, pop, popMatrix, popStyle, postEvent, postWindowMoved, postWindowResized, pow, print, print, print, print, print, print, print, print, print, printArray, printCamera, println, println, println, println, println, println, println, println, println, println, println, printMatrix, printProjection, printStackTrace, push, pushMatrix, pushStyle, quad, quadraticVertex, quadraticVertex, radians, random, random, randomGaussian, randomSeed, rect, rect, rect, rectMode, red, redraw, registerMethod, requestImage, requestImage, resetMatrix, resetShader, resetShader, resume, reverse, reverse, reverse, reverse, reverse, reverse, reverse, rotate, rotate, rotateX, rotateY, rotateZ, round, runSketch, runSketch, runSketch, saturation, save, saveBytes, saveBytes, saveBytes, saveFile, saveFrame, saveFrame, saveJSONArray, saveJSONArray, saveJSONObject, saveJSONObject, savePath, saveStream, saveStream, saveStream, saveStream, saveStream, saveStrings, saveStrings, saveStrings, saveTable, saveTable, saveXML, saveXML, scale, scale, scale, screenX, screenX, screenY, screenY, screenZ, second, selectCallback, selectFolder, selectFolder, selectFolder, selectInput, selectInput, selectInput, selectOutput, selectOutput, selectOutput, set, set, setMatrix, setMatrix, setMatrix, setSize, shader, shader, shape, shape, shape, shapeMode, shearX, shearY, shell, shininess, shorten, shorten, shorten, shorten, shorten, shorten, shorten, showDepthWarning, showDepthWarningXYZ, showMethodWarning, showMissingWarning, showSurface, showVariationWarning, sin, size, size, size, sketchDisplay, sketchFile, sketchFullScreen, sketchHeight, sketchOutputPath, sketchOutputStream, sketchPath, sketchPath, sketchPixelDensity, sketchRenderer, sketchSmooth, sketchWidth, sketchWindowColor, smooth, smooth, sort, sort, sort, sort, sort, sort, sort, sort, sort, sort, specular, specular, specular, sphere, sphereDetail, sphereDetail, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, split, split, splitTokens, splitTokens, spotLight, sq, sqrt, square, start, startSurface, str, str, str, str, str, str, str, str, str, str, stroke, stroke, stroke, stroke, stroke, stroke, strokeCap, strokeJoin, strokeWeight, style, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, tan, text, text, text, text, text, text, text, text, text, text, text, textAlign, textAlign, textAscent, textDescent, textFont, textFont, textLeading, textMode, textSize, texture, textureMode, textureWrap, textWidth, textWidth, textWidth, thread, tint, tint, tint, tint, tint, tint, translate, translate, triangle, trim, trim, unbinary, unhex, unregisterMethod, updatePixels, updatePixels, urlDecode, urlEncode, vertex, vertex, vertex, vertex, vertex, windowMove, windowMoved, windowRatio, windowResizable, windowResize, windowResized, windowTitle, year

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • pixelaudio

      PixelAudio pixelaudio

    • multigen

      MultiGen multigen

    • genWidth

      int genWidth

    • genHeight

      int genHeight

    • mapper

      PixelAudioMapper mapper

    • mapSize

      int mapSize

    • baseImage

      processing.core.PImage baseImage

    • mapImage

      processing.core.PImage mapImage

    • chan

      PixelAudioMapper.ChannelNames chan

    • colors

      int[] colors

    • audioFile

      File audioFile

    • audioFilePath

      String audioFilePath

    • audioFileName

      String audioFileName

    • audioFileTag

      String audioFileTag

    • audioFileLength

      int audioFileLength

    • imageFile

      File imageFile

    • imageFilePath

      String imageFilePath

    • imageFileName

      String imageFileName

    • imageFileTag

      String imageFileTag

    • imageFileWidth

      int imageFileWidth

    • imageFileHeight

      int imageFileHeight

    • isLoadToBoth

      boolean isLoadToBoth

    • isBlending

      boolean isBlending

    • daPath

      String daPath

    • minim

      ddf.minim.Minim minim

    • audioOut

      ddf.minim.AudioOutput audioOut

    • isBufferStale

      boolean isBufferStale

    • sampleRate

      float sampleRate

    • fileSampleRate

      float fileSampleRate

    • bufferSampleRate

      float bufferSampleRate

    • audioSignal

      float[] audioSignal

    • playBuffer

      ddf.minim.MultiChannelBuffer playBuffer

    • samplePos

      int samplePos

    • audioLength

      int audioLength

    • noteDuration

      int noteDuration

    • samplelen

      int samplelen

    • synthGain

      float synthGain

    • synthPointGain

      float synthPointGain

    • outputGain

      float outputGain

    • isMuted

      boolean isMuted

    • pool

      PASamplerInstrumentPool pool

    • samplerMaxVoices

      int samplerMaxVoices

    • maxAmplitude

      float maxAmplitude

    • defaultEnv

      ADSRParams defaultEnv

    • pitchScaling

      float pitchScaling

    • defaultPitchScale

      float defaultPitchScale

    • lowPitchScaling

      float lowPitchScaling

    • highPitchScaling

      float highPitchScaling

    • usePitchedGrains

      boolean usePitchedGrains

    • useGranularSynth

      boolean useGranularSynth

    • granEnvDuration

      int granEnvDuration

    • granSignal

      public float[] granSignal

    • gSynth

      public PAGranularInstrument gSynth

    • gDir

      public PAGranularInstrumentDirector gDir

    • granularGain

      public float granularGain

    • granularPointGain

      public float granularPointGain

    • useShortGrain

      boolean useShortGrain

    • longSample

      int longSample

    • shortSample

      int shortSample

    • granSamples

      int granSamples

    • hopSamples

      int hopSamples

    • gParamsGesture

      GestureGranularParams gParamsGesture

    • gParamsFixed

      GestureGranularParams gParamsFixed

    • curveSteps

      public int curveSteps

    • gMaxVoices

      public int gMaxVoices

    • useLongBursts

      boolean useLongBursts

    • maxBurstGrains

      int maxBurstGrains

    • burstGrains

      int burstGrains

    • shift

      int shift

    • totalShift

      int totalShift

    • isAnimating

      boolean isAnimating

    • oldIsAnimating

      boolean oldIsAnimating

    • videoSteps

      int videoSteps

    • isRecordingVideo

      boolean isRecordingVideo

    • videoFrameRate

      int videoFrameRate

    • step

      int step

    • videx

      com.hamoid.VideoExport videx

    • curveMaker

      PACurveMaker curveMaker

    • isDrawMode

      public boolean isDrawMode

    • epsilon

      public float epsilon

    • currentPoint

      public processing.core.PVector currentPoint

    • allPoints

      public ArrayList<processing.core.PVector> allPoints

    • allTimes

      public ArrayList<Integer> allTimes

    • startTime

      public int startTime

    • dragColor

      public int dragColor

    • dragWeight

      public float dragWeight

    • polySteps

      public int polySteps

    • brushes

      public ArrayList<TutorialOne_04_Network.AudioBrushLite> brushes

    • hoverBrush

      public TutorialOne_04_Network.AudioBrushLite hoverBrush

    • hoverIndex

      public int hoverIndex

    • activeBrush

      public TutorialOne_04_Network.AudioBrushLite activeBrush

    • readyBrushColor

      int readyBrushColor

    • hoverBrushColor

      int hoverBrushColor

    • selectedBrushColor

      int selectedBrushColor

    • readyBrushColor1

      int readyBrushColor1

    • hoverBrushColor1

      int hoverBrushColor1

    • selectedBrushColor1

      int selectedBrushColor1

    • readyBrushColor2

      int readyBrushColor2

    • hoverBrushColor2

      int hoverBrushColor2

    • selectedBrushColor2

      int selectedBrushColor2

    • lineColor

      int lineColor

    • dimLineColor

      int dimLineColor

    • circleColor

      int circleColor

    • dimCircleColor

      int dimCircleColor

    • samplerTimeLocs

      public ArrayList<TimedLocation> samplerTimeLocs

    • pointTimeLocs

      ArrayList<TimedLocation> pointTimeLocs

    • grainTimeLocs

      ArrayList<TimedLocation> grainTimeLocs

    • eventStep

      int eventStep

    • animatedCircleColor

      int animatedCircleColor

    • isIgnoreOutsideBounds

      boolean isIgnoreOutsideBounds

    • nd

      TutorialOne_04_Network.NetworkDelegate nd

    • isUseNetworkDelegate

      boolean isUseNetworkDelegate

  • Constructor Details

    • TutorialOne_04_Network

      public TutorialOne_04_Network()

  • Method Details

    • main

      public static void main(String[] args)

    • settings

      public void settings()
      Overrides:
      settings in class processing.core.PApplet

    • setup

      public void setup()
      Overrides:
      setup in class processing.core.PApplet

    • stop

      public void stop()
      turn off audio processing when we exit
      Overrides:
      stop in class processing.core.PApplet

    • getColors

      public int[] getColors(int size)
      Generates an array of rainbow colors using the HSB color space.
      Parameters:
      size - the number of entries in the colors array
      Returns:
      an array of RGB colors ordered by hue

    • initImages

      public void initImages()
      Initializes mapImage with the colors array. MapImage handles the color data for mapper and also serves as our display image. BaseImage is intended as a reference image that usually only changes when you open a new image file.

    • initDrawing

      public void initDrawing()
      Initializes drawing and drawing interaction variables.

    • preloadFiles

      public void preloadFiles(String path, String filename)
      Preload an audio file using a file path and a filename.
      Parameters:
      path - the fully qualified path to the file's directory, ending with a '/'
      filename - the name of the file

    • draw

      public void draw()
      Overrides:
      draw in class processing.core.PApplet

    • animate

      public void animate()

    • stepAnimation

      public void stepAnimation()
      Step through the animation, called by the draw() method. Will also record a frame of video, if we're recording.

    • renderFrame

      public void renderFrame(int step)
      Renders a frame of animation: moving along the signal path, copies baseImage pixels to mapImage pixels, adjusting the index position of the copy using totalShift -- i.e. we don't actually rotate the pixels, we just shift the position they're copied to.
      Parameters:
      step - current animation step

    • handleDrawing

      public void handleDrawing()
      Handles user's drawing actions: draws previously recorded brushstrokes, tracks and generates interactive animation and audio events.

    • findHoverHit

      TutorialOne_04_Network.BrushHit findHoverHit()
      Returns:
      a reference to the brushstroke the mouse is over, or null if there's no brushstroke.

    • updateHover

      void updateHover()
      Update the hoverBrush and hoverIndex global variables.

    • writeToScreen

      public void writeToScreen(String msg, int x, int y, int weight, boolean isWhite)
      Displays a line of text to the screen, usually in the draw loop. Handy for debugging. typical call: writeToScreen("When does the mind stop and the world begin?", 64, 1000, 24, true);
      Parameters:
      msg - message to write
      x - x coordinate
      y - y coordinate
      weight - font weight
      isWhite - if true, white text, otherwise, black text

    • mousePressed

      public void mousePressed()
      The built-in mousePressed handler for Processing, clicks are handled in mouseClicked().
      Overrides:
      mousePressed in class processing.core.PApplet

    • mouseDragged

      public void mouseDragged()
      Overrides:
      mouseDragged in class processing.core.PApplet

    • mouseReleased

      public void mouseReleased()
      Overrides:
      mouseReleased in class processing.core.PApplet

    • mouseClicked

      public void mouseClicked()
      Overrides:
      mouseClicked in class processing.core.PApplet

    • keyPressed

      public void keyPressed()
      Built-in keyPressed handler, forwards events to parseKey.
      Overrides:
      keyPressed in class processing.core.PApplet

    • parseKey

      public void parseKey(char key, int keyCode)
      Handles key press events passed on by the built-in keyPressed method. By moving key event handling outside the built-in keyPressed method, we make it possible to post key commands without an actual key event. Methods and interfaces and even other threads can call parseKey(). This opens up many possibilities and a some risks, too.
      Specified by:
      parseKey in interface PANetworkClientINF
      Parameters:
      key -
      keyCode -

    • showHelp

      public void showHelp()
      to generate help output, run RegEx search/replace on parseKey case lines with: // case ('.'): // (.+) // println(" * Press $1 to $2.");

    • setAudioGain

      public void setAudioGain(float g)
      Sets audioOut.gain.
      Parameters:
      g - gain value for audioOut, in decibels

    • applyColor

      public int[] applyColor(int[] colorSource, int[] graySource, int[] lut)
      Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
      Parameters:
      colorSource - a source array of RGB data from which to obtain hue and saturation values
      graySource - an target array of RGB data from which to obtain brightness values
      lut - a lookup table, must be the same size as colorSource and graySource
      Returns:
      the graySource array of RGB values, with hue and saturation values changed
      Throws:
      IllegalArgumentException - if array arguments are null or if they are not the same length

    • applyColorShifted

      public int[] applyColorShifted(int[] colorSource, int[] graySource, int[] lut, int shift)
      Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing, taking into account any pixels that were shifted.
      Parameters:
      colorSource - a source array of RGB data from which to obtain hue and saturation values
      graySource - an target array of RGB data from which to obtain brightness values
      lut - a lookup table, must be the same size as colorSource and graySource
      Returns:
      the graySource array of RGB values, with hue and saturation values changed
      Throws:
      IllegalArgumentException - if array arguments are null or if they are not the same length

    • chooseColorImage

      public void chooseColorImage()
      Call to initiate process of opening an image file to get its color data.

    • colorFileSelected

      public void colorFileSelected(File selectedFile)
      callback method for chooseColorImage()
      Parameters:
      selectedFile - the File the user selected

    • applyImageColor

      public void applyImageColor(File imgFile, processing.core.PImage targetImage)
      Apply the hue and saturation of a chosen image file to the brightness channel of the display image.
      Parameters:
      imgFile - selected image file, source of hue and saturation values
      targetImage - target image where brightness will remain unchanged

    • chooseFile

      public void chooseFile()
      Wrapper method for Processing's selectInput command

    • fileSelected

      public void fileSelected(File selectedFile)
      callback method for chooseFile(), handles standard audio and image formats for Processing. If a file has been successfully selected, continues with a call to loadAudioFile() or loadImageFile().
      Parameters:
      selectedFile - the File the user selected

    • loadAudioFile

      public void loadAudioFile(File audFile)
      Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage. If you want to load the image file and audio file separately, comment out writeAudioToImage().
      Parameters:
      audFile - an audio file

    • blendInto

      public static void blendInto(ddf.minim.MultiChannelBuffer dest, ddf.minim.MultiChannelBuffer src, float weight, boolean normalize, float targetDB)
      Blends audio data from buffer "src" into buffer "dest" in place. The formula per sample is: dest[i] = weight * src[i] + (1 - weight) * dest[i]
      Parameters:
      dest - Destination buffer (will be modified)
      src - Source buffer to blend into dest
      weight - Blend ratio (0.0 = keep dest, 1.0 = replace with src)

    • normalize

      public static void normalize(float[] signal, float targetPeakDB)
      Normalizes a single-channel signal array to a target RMS level in dBFS (decibels relative to full scale). 0 is the maximum digital amplitude. -6.0 dB is 50% of the maximum level.
      Parameters:
      signal -
      targetPeakDB -

    • renderAudioToMapImage

      public void renderAudioToMapImage(PixelAudioMapper.ChannelNames chan, int shift)
      Transcodes audio data in audioSignal and writes it to color channel chan of mapImage.
      Parameters:
      chan - A color channel
      shift - number of index positions to shift the audio signal

    • writeAudioToImage

      public void writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan, int shift)
      Transcodes audio data in sig[] and writes it to color channel chan of img using the lookup tables in mapper to redirect indexing. Calls mapper.mapSigToImgShifted(), which will throw an IllegalArgumentException if sig.length != img.pixels.length or sig.length != mapper.getSize().
      Parameters:
      sig - an array of float, should be audio data in the range [-1.0, 1.0]
      mapper - a PixelAudioMapper
      img - a PImage
      chan - a color channel
      shift - the number of indices to shift when writing audio

    • loadImageFile

      public void loadImageFile(File imgFile)
      Attempts to load image data from a selected file into mapImage, then calls writeImageToAudio() to transcode HSB brightness channel to audio and writes it to playBuffer and audioSignal.
      Parameters:
      imgFile - an image file

    • setAlphaWithBlack

      public int setAlphaWithBlack(int argb, int alpha)
      Sets the alpha channel of an RGBA color, conditionally setting alpha = 0 if all other channels = 0.
      Parameters:
      argb - an RGBA color value
      alpha - the desired alpha value to apply to argb
      Returns:
      the argb color with changed alpha channel value

    • setAlpha

      public static int setAlpha(int argb, int alpha)
      Sets the alpha channel of an RGBA color.
      Parameters:
      argb - an RGBA color value
      alpha - the desired alpha value to apply to argb
      Returns:
      the argb color with changed alpha channel value

    • writeImageToAudio

      public void writeImageToAudio(processing.core.PImage img, PixelAudioMapper mapper, float[] sig, PixelAudioMapper.ChannelNames chan, int shift)
      This method writes a color channel from an image to playBuffer, fulfilling a central concept of the PixelAudio library: image is sound. Calls mapper.mapImgToSig(), which will throw an IllegalArgumentException if img.pixels.length != sig.length or img.width * img.height != mapper.getWidth() * mapper.getHeight(). Sets totalShift = 0 on completion: the image and audio are now in sync.
      Parameters:
      img - a PImage, a source of data
      mapper - a PixelAudioMapper, handles mapping between image and audio signal
      sig - an target array of float in audio format
      chan - a color channel
      shift - number of indices to shift

    • renderMapImageToAudio

      public void renderMapImageToAudio(PixelAudioMapper.ChannelNames chan)
      Writes a specified channel of mapImage to audioSignal.
      Parameters:
      chan - the selected color channel

    • commitMapImageToBaseImage

      public void commitMapImageToBaseImage()
      Writes the mapImage, which may change with animation, to the baseImage, a reference image that usually only changes when a new file is loaded.

    • commitNewBaseImage

      public void commitNewBaseImage(processing.core.PImage img)
      Copies the supplied PImage to mapImage and baseImage, sets totalShift to 0 (the images are identical).
      Parameters:
      img -

    • refreshMapImageFromBase

      public void refreshMapImageFromBase()
      Writes baseImage to mapImage with an index position offset of totalShift.

    • saveToAudio

      public void saveToAudio()
      Calls Processing's selectOutput method to start the process of saving the current audio signal to a .wav file.

    • audioFileSelectedWrite

      public void audioFileSelectedWrite(File selection)
      Parameters:
      selection - a File to write as audio

    • saveAudioToFile

      public void saveAudioToFile(float[] samples, float sampleRate, String fileName) throws IOException, UnsupportedAudioFileException
      Saves audio data to 16-bit integer PCM format, which Processing can also open. This same method can be called as a static method in AudioUtility.
      Parameters:
      samples - an array of floats in the audio range (-1.0f, 1.0f)
      sampleRate - audio sample rate for the file
      fileName - name of the file to save to
      Throws:
      IOException - an Exception you'll need to handle to call this method
      UnsupportedAudioFileException - another Exception

    • saveToImage

      public void saveToImage()
      Calls Processing's selectOutput method to start the process of saving the mapImage (the offscreen copy of the display image) to a .png file.

    • imageFileSelectedWrite

      public void imageFileSelectedWrite(File selection)

    • saveImageToFile

      public void saveImageToFile(processing.core.PImage img, String fileName)

    • initAudio

      public void initAudio()
      CALL THIS METHOD IN SETUP() Initializes Minim audio library and audio variables.

    • initGranularParams

      public void initGranularParams()
      Initializes global variables gParamsGesture and gParamsFixed, which provide basic settings for granular synthesis the follows gesture timing or fixed hop timing between grains.

    • audioMousePressed

      public void audioMousePressed(int x, int y)
      Handles mouse clicks that happen outside a brushstroke.
      Parameters:
      x - x-coordinate of mouse click
      y - y-coordinate of mouse click

    • handleClickOutsideBrush

      public int handleClickOutsideBrush(int x, int y)
      Parameters:
      x -
      y -

    • getSamplePos

      public int getSamplePos(int x, int y)
      Calculates the index of the image pixel within the signal path, taking the shifting of pixels and audioSignal into account. See MusicBoxBuffer for use of a windowed buffer in this calculation.
      Parameters:
      x - an x coordinate within mapImage and display bounds
      y - a y coordinate within mapImage and display bounds
      Returns:
      the index of the sample corresponding to (x,y) on the signal path

    • getCoordFromSignalPos

      public processing.core.PVector getCoordFromSignalPos(int pos)
      Calculates the display image coordinates corresponding to a specified audio sample index.
      Parameters:
      pos - an index into an audio signal, must be between 0 and width * height - 1.
      Returns:
      a PVector with the x and y coordinates

    • drawCircle

      public void drawCircle(int x, int y)
      Draws a circle at the location of an audio trigger (mouseDown event).
      Parameters:
      x - x coordinate of circle
      y - y coordinate of circle

    • playSample

      public int playSample(int samplePos, int samplelen, float amplitude, float pan)
      Plays an audio sample with default envelope and stereo pan.
      Parameters:
      samplePos - position of the sample in the audio buffer
      samplelen - length of the sample (will be adjusted)
      amplitude - amplitude of the sample on playback
      Returns:
      the calculated sample length in samples

    • playSample

      public int playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pan)
      Plays an audio sample with a custom envelope and stereo pan.
      Parameters:
      samplePos - position of the sample in the audio buffer
      samplelen - length of the sample (will be adjusted)
      amplitude - amplitude of the sample on playback
      env - an ADSR envelope for the sample
      pan - position of sound in the stereo audio field (-1.0 = left, 0.0 = center, 1.0 = right)
      Returns:
      the calculated sample length in samples

    • playSample

      public int playSample(int samplePos, int samplelen, float amplitude, ADSRParams env, float pitch, float pan)
      Plays an audio sample with with a custom envelope, pitch and stereo pan.
      Parameters:
      samplePos - position of the sample in the audio buffer
      samplelen - length of the sample (will be adjusted)
      amplitude - amplitude of the sample on playback
      env - an ADSR envelope for the sample
      pitch - pitch scaling as deviation from default (1.0), where 0.5 = octave lower, 2.0 = oactave higher
      pan - position of sound in the stereo audio field (-1.0 = left, 0.0 = center, 1.0 = right)
      Returns:

    • calcSampleLen

      public int calcSampleLen()
      Returns:
      a length in samples with some Gaussian variation

    • buildGranSynth

      public PAGranularInstrument buildGranSynth(ddf.minim.AudioOutput out, ADSRParams env, int numVoices)
      Initializes a new PAGranularSynth instance that you probably would pass to a PAGranularInstrumentDirector.
      Parameters:
      out - and AudioOutput, most likely the one used by this sketch
      env - an ADSRParams envelope
      numVoices - the number of voices to use for synthesizing simultaneous grains
      Returns:
      a PAGranularSynth instance

    • ensureSamplerReady

      void ensureSamplerReady()
      Ensures that all resources and variable necessary for the Sampler synth are ready to go.

    • ensureGranularReady

      void ensureGranularReady()
      Ensures that all resources and variable necessary for the Granular synth are ready to go.

    • updateAudioChain

      void updateAudioChain(float[] sig)
      Updates resources such as playBuffer and pool with a new signal, typcically when a new file is loaded.
      Parameters:
      sig - an audio signal as an array of float

    • playGranularGesture

      public void playGranularGesture(float[] buf, GestureSchedule sched, GestureGranularParams params)
      Calls PAGranularInstrumentDirector gDir to play a granular audio event.
      Parameters:
      buf - an audio signal as an array of flaot
      sched - an GestureSchedule with coordinate and timing information
      params - a bundle of control parameters for granular synthesis

    • calculateEnvelopeDb

      public ADSRParams calculateEnvelopeDb(float gainDb, int totalSamples, float sampleRate)
      Calculate an envelope of length totalSamples.
      Parameters:
      gainDb - desired gain in dB, currently ignored
      totalSamples - number of samples the envelope should cover
      sampleRate - sample rate of the audio buffer the envelope is applied to
      Returns:
      and ADSRParams envelope

    • calculateEnvelopeLinear

      public ADSRParams calculateEnvelopeLinear(float gainDb, float totalMs)
      Calculate an envelope of length totalSamples.
      Parameters:
      gainDb - desired gain in dB, currently ignored
      totalMs - desired duration of the envelope in milliseconds
      Returns:
      an ADSRParams envelope

    • debugIndexHeadroom

      static void debugIndexHeadroom(float[] buf, int[] startIndices, GestureGranularParams ggp)

    • debugTimesMs

      static void debugTimesMs(GestureSchedule s)

    • initAllPoints

      public void initAllPoints()
      Initializes allPoints and adds the current mouse location to it.

    • addPoint

      public void addPoint(int x, int y)
      While user is dragging the mouses and isDrawMode == true, accumulates new points to allPoints and event times to allTimes. Sets sampleX, sampleY and samplePos variables. We constrain points outside the bounds of the display window. An alternative approach is be to ignore them (isIgnoreOutsideBounds == true), which may give a more "natural" appearance for fast drawing.

    • clipToWidth

      public int clipToWidth(int i)
      Clips parameter i to the interval (0..width-1)
      Parameters:
      i -
      Returns:

    • clipToHeight

      public int clipToHeight(int i)
      Clips parameter i to the interval (0..width-1)
      Parameters:
      i -
      Returns:

    • jitterCoord

      public processing.core.PVector jitterCoord(int x, int y, int deviationPx)
      Parameters:
      x - x-coordinate
      y - y-coordinate
      deviationPx - distance deviation from mean
      Returns:
      a PVector with coordinates shifted by a Gaussing variable

    • generateJitterPitch

      float[] generateJitterPitch(int length, float deviationPitch)
      Generates an array of Gaussian values for shifting pitch, where 1.0 = no shift.
      Parameters:
      length - length of the returned array
      deviationPitch - expected average deviation of the pitch
      Returns:
      and array of Gaussian values centered on 1.0

    • initCurveMakerAndAddBrush

      public void initCurveMakerAndAddBrush()
      Initializes a PACurveMaker instance with allPoints as an argument to the factory method PACurveMaker.buildCurveMaker() and then fills in PACurveMaker instance variables from variables in the calling class (TutorialOneDrawing, here).

    • defaultPathModeFor

      TutorialOne_04_Network.PathMode defaultPathModeFor(TutorialOne_04_Network.BrushOutput out)
      Determines the path mode for a particular BrushOutput.
      Parameters:
      out - a BrushOutput (SAMPLER of GRANULAR)
      Returns:
      a PathMode (ALL_POINTS for Sampler instruments, CURVE_POINTS for Granular instruments)

    • drawBrushShapes

      public void drawBrushShapes()
      Enrty point for drawing brushstrokes on the screen.

    • drawBrushes

      public void drawBrushes(List<TutorialOne_04_Network.AudioBrushLite> list, int readyColor, int hoverColor, int selectedColor)
      Draw brushstrokes on the display image.
      Parameters:
      list - a list of all the brushstrokes (AudioBrushLite)
      readyColor - color for a selectable brush
      hoverColor - color for a brush when the mouse hovers over it
      selectedColor - color for a selected brush (click or spacebar selects)

    • setBrushEpsilon

      public void setBrushEpsilon(TutorialOne_04_Network.AudioBrushLite b, float e)
      Sets epsilon value for the PACurveMaker associated with an AudioBrushLite instance.
      Parameters:
      b - an AudioBrushLite instance
      e - desired epsilon value to control point reduction

    • getPathPoints

      ArrayList<processing.core.PVector> getPathPoints(TutorialOne_04_Network.AudioBrushLite b)
      Get the path points of a brushstroke, with the representation determined by the BrushConfig's path mode.
      Parameters:
      b - an AudioBrushLite instance
      Returns:
      an all points, reduced, or curve representation of the path points of an AudioBrushLite instance

    • getScheduleForBrush

      public GestureSchedule getScheduleForBrush(TutorialOne_04_Network.AudioBrushLite b)
      Get a GestureSchedule (points + timing) for an AudioBrushLite instance.
      Parameters:
      b - an AudioBrushLite instance
      Returns:
      GestureSchedule for the current pathMode of the brush

    • scheduleSamplerBrushClick

      void scheduleSamplerBrushClick(TutorialOne_04_Network.AudioBrushLite b)
      Schedule a Sampler brush audio / animation event.
      Parameters:
      b - an AudioBrushLite instance

    • storeSamplerCurveTL

      public void storeSamplerCurveTL(GestureSchedule sched, int startTime)
      Store scheduled sampler synth / animation events for future activation.
      Parameters:
      sched - a GestureSchedule (points + timing for a brush)
      startTime - time to start a series of events

    • runSamplerBrushEvents

      public void runSamplerBrushEvents()
      Execute audio / animation events for Sampler brushstrokes.

    • scheduleGranularBrushClick

      void scheduleGranularBrushClick(TutorialOne_04_Network.AudioBrushLite b)
      Schedule a Granular brush audio / animation event.
      Parameters:
      b - an AudioBrushLite instance

    • storeGranularCurveTL

      public void storeGranularCurveTL(GestureSchedule sched, int startTime, boolean isGesture)
      Store scheduled granular synth / animation events for future activation.
      Parameters:
      sched - a GestureSchedule (points + timing for a brush)
      startTime - time to start a series of events
      isGesture - is the schedule for a GESTURE or FIXED granular event (ignored)

    • runGrainEvents

      public void runGrainEvents()
      Tracks and runs TimedLocation events in the grainLocsArray list, which is associated with granular synthesis gestures.

    • runPointEvents

      public void runPointEvents()
      Tracks and runs TimedLocation events in the timeLocsArray list, which is associated with mouse clicks that trigger audio a the click point.

    • mouseInPoly

      public boolean mouseInPoly(ArrayList<processing.core.PVector> poly)
      Parameters:
      poly - a polygon described by an ArrayList of PVector
      Returns:
      true if the mouse is within the bounds of the polygon, false otherwise

    • getBrushIndex

      int getBrushIndex(TutorialOne_04_Network.AudioBrushLite brush)

    • reset

      @Deprecated public void reset()
      Deprecated.
      Reinitializes audio and clears event lists. -- TODO drop, this used to be the "emergency off" switch for runaway audio processing

    • removeHoveredOrOldestBrush

      public void removeHoveredOrOldestBrush()
      Removes the current active AudioBrushLite instance.

    • removeNewestBrush

      public void removeNewestBrush()
      Removes the most recent AudioBrushLite instance.

    • getPApplet

      public processing.core.PApplet getPApplet()
      Specified by:
      getPApplet in interface PANetworkClientINF

    • getMapper

      public PixelAudioMapper getMapper()
      Specified by:
      getMapper in interface PANetworkClientINF

    • controlMsg

      public void controlMsg(String control, float val)
      Specified by:
      controlMsg in interface PANetworkClientINF

    • playSample

      public int playSample(int samplePos)
      Specified by:
      playSample in interface PANetworkClientINF

    • playPoints

      public void playPoints(ArrayList<processing.core.PVector> pts)
      Specified by:
      playPoints in interface PANetworkClientINF