1. workflow

graph TD
  %% --- Sound Materials -----------------
  subgraph "Sound Materials"
    A["Ableton Live<br/>+ Max for Live<br/>(Original M4L Devices)<br/><br/>• Vector Synthesis<br/>• DC / AM / FM / PM<br/>• 3-D Rotator"]
    A -->|export| B((Sound Files))
    
  end

  %% --- Visual Engine -------------------
  CTRL["OSC Controller / Sequencer / TidalCycles / MIDI"] -->|OSC| C
  B -->|import| C["Processing 4 (Java)<br/> <br/><u>Generate</u>: Vector plot / Noise / Vec-ReScan / Image→Hatching<br><br/><u>Process</u>: Placement / Probabilities / Color / Animation / FX<br><br/><u>Export</u>: SVG ・ PNG"]

  %% --- Control -------------------------
  

  %% --- Outputs -------------------------
  subgraph "Outputs<br> <br> "
  A -->|audio output| OSC[Oscilloscope<br>LaserProjector]
    D["SVG Post Process<br/>Pen Plotter"]
    E["Digital Video / Still Image"]
    
  end

  C -->|SVG| D
  C -->|PNG| E

  

2.Sound Design & Audio Generation

Audio signal generation using Ableton Live and Max for Live devices is at the core of the work. While the sound can be listened to as audio, the emphasis is on being able to "see" the sound image using an oscilloscope or XY plotter.

2.1 Tools used

• Ableton Live (host environment)
• Max/MSP Max for Live (signal generation and control GUI)

2.2 Generation Techniques

• Vector Synthesis

  • Vector synthesis, used in the context of oscilloscope art, is a technique to draw a figure by assigning different audio signals to L (left) and R (right) of the stereo output and interpreting and displaying them as XY coordinates.
  • The waveforms (sine, triangle, noise, AM/FM modulation, etc.) given to each channel can be used to generate complex figures and continuous strokes in real time.

  0516.mp4

• Lissajous Figure

  • Lissajous figures are figures drawn on an oscilloscope or XY plotter when sine waves with different frequencies and phases are given to the X and Y axes, respectively.
  • Depending on the frequency ratio and phase difference, a circular, elliptical, figure-8, or complex shape appears, and has long been used as a method to visualize acoustic signals.
  • This project applies this principle and extends it to more dynamic and complex structures.

https://codepen.io/jh3y/pen/abObQNR

• VectorRescanning
  • Vector Rescanning is the process of transforming and redrawing a raster image into a vector linear scanning pattern based on its grayscale/contrast information.

VID_49420119_115134_377.mp4

• Combination of 3D rotation effects, DC offset, and AM/FM/PM (modulation system) signals

  Pseudo-3D rotation and various modulations by a self-made m4l device can be used to change the image and sound.

  Further process the signal by mixing multiple signals and applying normal audio effects (distortion, phaser, bit reduction, etc.).

  https://youtu.be/BU7-fNt0X14

2.3 Visual Monitoring with an Oscilloscope

• At the point of production, the signal is checked on an oscilloscope or XY plotter display, and production proceeds simultaneously with a sensory perception of the correspondence between sound = image.
• The generated sound material is reconstructed as vertex data in the Processing program described below.

The technology of projecting images on CRT tubes, such as oscilloscopes and vector monitors, was developed before liquid crystal displays (LCDs), but it still has a unique artistic value, and my project is inspired by this kind of analog video signal processing that is being decaying. In my projects, I am often inspired by such lost analog video signal processing, and I am trying to rediscover and extend it in digital domain using Processing4 as explained in the following section.