Painted Resonance

Painted Resonance is a multimedia ukulele and computer performance that makes use of programming in Max/MSP for real time interactive audiovisual effects. It was premiered at The Fidget Space in Philadelphia, PA in December 2024. 

About the performance

As seen in the video, a performer can play music and have the background color light up as they pluck notes. When they choose to record some notes into an audiovisual loop, the program will analyze what they recorded and generate visual bubbles associated with the notes they played. These bubbles then travel across the screen as they the associated loop's audio progresses, then finally replays from the beginning. When a loop's audio replays, so does its visual bubbles.  When an audiovisual loop is recorded, it's visual bubbles are placed in a random place across the screen; thus, the audio for the loop is then panned to follow the visual location of the loop. The camera of the field of view slowly rotates around, creating visual interest evolving over time. 

As the performance progresses, the performer has the option to start audiovisual noise. This makes their live ukulele playing be overdriven, and for the visuals to slightly distort guided by the sound of the overdriven ukulele. Musically, this decision was made to allow for the performer's soloing against the loops to timbrally stand out, as well as to provide an alternate sound to provide more interest & engagement with the texture, after attention fatigue from listening to the same type of sounds and looking at the same type of pictures for a few minutes.

The performance ends with a third option the performer gets to select on the computer. This selection slowly increases the reverb of the audio to let the audio dissolve into a wash of sound, has the visual distortion set itself into a feedback loop that doesn't reset creating noisy visuals, and slowly lowers the gain of the piece while the performer can play an ending. The performer can then further lower the gain as needed using the computer or their interface.

About the tech

Regarding programming, Painted Resonance was programmed in Max/MSP, a visual dataflow programming language that allows for real-time signal processing and programming with OpenGL. This program requires real time signal processing, trigonometry for animation, visual processing, significant data management, gpu/cpu optimization, and abstractions with hierarchical programming. Using Max/MSP allowed for me to program a completely generative system, where the only upper limit to how many audiovisual loops you can record and how many visual bubbles you can generate is up until your processor can't make anymore (which will take quite a while). 

The minimum hardware necessary during performance is a laptop, laptop charger, audio interface, microphone with mono cable, 2 TRS cables, stereo speakers, and 2 IEC cables for the speakers. 

About the musical performance

Elements of randomness are made use of at many points in the design of the audiovisual loops. This allows for dynamic and interactive performances. As the performer, no matter how many times you practice this with the same material, you are going to get a different audiovisual result. Imagine you started playing the same material I did in this video, but all your loops generated on the right side of the screen and on the right side of the visual field. Would you make the same decisions I did to continue a section or transition to the next one when I did? Would you make more recordings until the left and right sides of your audiovisual field were balanced? Would you embrace the lopsided nature of it and solo on top of it? These are some of the choices the performer gets to make when they use the system. By design, this program encourages improvisation on the part of the performer, allowing for interaction between the performer and all the recorded versions of the performer and creating a unique musical experience every time.

As a result, the performance of Painted Resonance is improvisation. While I composed a score for it, the score is heavily built upon improvisational sections. I started the piece in A minor pentatonic, to keep things always consonant while loops are being added on top of each other over and over to keep a sense of key and pleasant listening throughout. Then, as the audiovisual noise starts, I expand into the A minor blues scale, the tritone standing out so discordantly against the pentatonic scale we've been acustumed to, that a listener is able to use that to pick out my live playing from the recorded playing. 

The score is not a traditional score. It has blocks of music, switching between standard notation, chord names, and ukulele tablature, and using written word instructions. As a result, it is handwritten. Pictures of it are on the left. 

Inspiration of the Piece/Meaning

The inspiration behind this piece was based upon sensory friendly/stimulation videos. It's meant to gently capture your attention in a sensory way, rather than a thinking way, as a break against the constant digital attention fatigue. Having as many slowly moving and changing elements, allows for constant dishabituation in the audience, allowing for their attention to easily rest on the experience. Based on lived experience with ADHD, I have learned how to constantly aggressively control my attention, even for things I want to focus on- I can lose my focus when watching a tv show I like or playing a video game. By the end of the day, every day, it is exhausting. Part of the ideas with being a gentle attention holder, was as a way to rest. Rather than an audience memeber (with or without ADHD, but probably with) needing to aggressively & exhaustingly corral their attention, their attention will naturally and effortlessly be captured by what's going on in front of them. There has to be constant change and movement and opportunities to focus on all different parts of the performance so that as a mind naturally starts to wander, it will remain on the piece.