Explaining the witchcraft behind the Time Organ

I had originally planned on writing about the physical process of creating my Time Organ, but looking now at how things turned out, I think discussing the software behind it will be more interesting. (As far as the physical device, I'd say the most important takeaway is that these Rock Band Keyboards are fantastically cheap, have 25 keys w/ velocity, a touch sensor strip, a MIDI out port, and a ton of room inside for sensors and wires and junk, so you should all buy one to mess around with.)

First off: the code is all here on github for anyone curious – I tried my best to keep things well organized, although the commenting can be sparse in places. The majority of the logic is in Max (plus some amounts of arduino and javascript code), with the functionality split up into several main patchers, embedded in bpatchers here in main.maxpat:

The core features to think about are:

• A custom granular synth engine that 
• can draw grains from live recordings as well as pre-recorded samples
• reacts in real-time to parameter changes
• uses a Hanning window to smooth in and out of grains
• operates on the scale of milliseconds, rather than microseconds like many other gran-synth setups
• The gran-synth output is then piped through a poly~ object and pitch-shifted based on MIDI key input, so the sound can be played polyphonically.
• The pattrstorage object is used (even though it crashes Max every time) to save presets of buffer contents, grain windows, and other parameters, which can then be recalled by button presses from the arduino.
• A stutter~ object is constantly recording the last 5 seconds of audio from channel 2, and when the "freeze" button is pressed, that buffer is copied to the gran-synth engine's buffer.
• The patch actually doesn't require the physical device to mess around with – try flipping the toggle on near that lower keyboard, and fiddling with the keys and some other controls to see what happens.

I'll start by talking about the granular synth stuff. The patches in my actual project have gotten pretty complex for the sake of supporting things like stereo, bypassing certain effects, and loading default values, but the core concepts are pretty intuitive. Here are simplified versions of the patch "granvolution" and the patch "polygrain" inside of it, which is loaded inside the poly~ object:

So there are really just 3 main things going on: The polygrain~ patch plays the grain when it gets a bang, multiplies the output by the hanning window generated by the fadecurve patch I wrote (the details of which aren't very important – just think of it as a smoother adsr~), and then outside the poly~, the whole repeated stream of sounds gets multiplied by the average amplitude from my other mic to allow breath control of volume.

Then the "tailfx" patch you see at the end there handles the pitch shifting, compression, and reverb:

(The pitchshift, reverb, and compress patches are all other custom ones, but they do pretty much what you'd expect, so no need to dive into that.) 

The last main piece, but probably the one most relevant to other people in this class, is the patch inside controller.maxpat, called arduino-buttons.maxpat:

This uses Max's javascript support for parsing and sending serial data, which I think is a great choice for those who like to maintain their sanity, since Max doesn't technically even have the concept of a string type. I'll let anyone curious check out the scripts themselves on github, but the main idea is this: messages are sent using readable ASCII characters, instead of binary data (avoids the need for bitwise operations), and each message sent uses a single character at the beginning as an identifier (b for button, p for potentiometer, r for reverb, etc.) followed by some more characters encoding the data it wants to communicate. Something like "s 1 0" from the arduino to Max means that switch 1 was turned off. Or "r 65" from Max to arduino means that I set reverb to 65%, so it should print that to the LCD screen so the performer has that feedback.

parseSerial.js has a few functions named after these identifiers (b(), p(), s(), etc) which are automatically called when the object receives a message starting with that symbol. Then it cleans up the data and sends it out the outlets in a Max-friendly way. toSerial.js basically does the opposite, but in a much more general way – it will turn any list of characters into a list of bytes, perfect for sending to the serial object. Feel free to use these ideas or the code itself in your projects to make communicating over serial in Max a little easier!