My crazy WebGL music expierment, the Lysertron is live!

Check it out and Contribute!

Echotron - WebGL Semantic & Extensible Music Visualization

Been working on a really fun experiment.

I call it the Echotron. Check it out here (WebGL capable browser required)

spiral and rings

I know what you’re thinking. “Another music visualizer, like there isn’t enough already!” Well, I like to think this ones a little different.

Semantic

Most music visualizers visualize the waveform of the music, analyzing it in realtime, and animating some state based on that feedback. That can produce some cool visuals, but I’ve always found the correlation between song events and and what the visuals do somewhat disconnected. Simply making the visuals pulse to the steady beat of the song is very tricky with simple realtime waveform analysis. It’s extremely difficult to make a machine perceive music in the same way that the human brain does.

And then there’s Echonest

Echonest has a very cool API feature. It will analyze the song and give you back a huge JSON file with incredibly detailed song metadata. Every beat, measure, section, and note is listed, including when it happens, how long it goes for, and even what tones occur.

Here’s an weensy sample of what they provide:

With this info we can make the visualization respond to semantic music events. When a beat happens, we can make the visualization pulse or jump. On an individual segment, we can highlight part of some object according the pitches within that note. Also (data not shown above) it provides data on measures/bars and sections (long stretches of similar style, like say a verse, or a chorus).

This is much closer to the way the human brain perceives the music. We can now make a visualization “dance” based on similar cues that allows a human to dance. The result, hopefully, is that the visual feels more connected to the actual music that is playing.

You can read more about the output from their analyzer here.

Extensible

The other part that makes this difference is extensibility. At the moment, there are not a lot of different layer types, but it’s designed to be a very simple engine that provides an interface that a WebGL doodler can quickly conform to and get something on screen. To make your doodle dance, you simply implement a method like onBeat and modify whatever state you need to change on each beat.

Each scene is a combination of three “layers”: Background, Midground and Foreground.

  • Background: Renders first, paints entire frame.
  • Midground: Renders second, typically textures or adorns background.
  • Foreground: Renders last, typically some sort of object animating front and center

Each “scene” is composed of a single random layer of each type. So adding more complexity and variation to the system is as easy as adding a new layer type.

But what’s a layer’s code look like?

Since the engine sets up the context, and music event callbacks for you, it takes very little code to do something simple. It’s based around THREE.js so managing positions, rotations, materials and other things is pretty straightforward.

The following example simply spins a cube around, and changes it’s color on every beat.

(exciting, eh?)

It’s easy to see just how simply you can elaborate on that. You can keep track of more complex state in the instance, update it every frame in update() and kick your numbers on event handlers like onBeat, onBar and onSegment.

Contributing

If you are a WebGL doodler or technical artist and want to contribute your pretties, checkout the repo on GitHub here

For now, you will clone the repo, add layers, and ask me to merge it back in. Though in the future I have plans to build a better system around contributions, allowing each layer to be in it’s own remote repo and pushed to a central server.

And Chrome Apps look like a super-cool use case for this as well. Stay tuned for that :)

Coffeeson JSON + CoffeeScript

Coffeeson is a little syntax that compiles into JSON. Underneath all those awkward braces and commas, JSON has always had a gorgeous data format at its heart. Coffeeson is an attempt to expose the good parts of JSON in a more human maintainable way. ;)

Check it out on GitHub

I recently had a few projects that needed some config files sitting on disk for a Node.JS application. So I should use JSON, right?

JSON is an awesome data exchange format when computer generated, and computer parsed. But when a human has to maintain the raw JSON data, it’s not quite so awesome. Balanced braces, commas everywhere, and quoted key names make maintenance a bit tricky.

Coffeeson aims to solve that. JavaScript powered applications, like node.js servers and the like, tend to use json files as configuration. Humans write these files, but machines read them.

It is not meant to be a data exchange format, there is no way to save data as Coffeeson, only to convert it to JSON or parse it as JavaScript. This is intentional. You should still use JSON for machine -> machine data transmision.

Now you can use the Coffeeson npm module to generate raw JSON, or just get the result without going to JSON at all. It just allows you to use CoffeeScript notation in data files. And I think that’s pretty awesome.

Usage

Which would you rather maintain?

npm install coffeeson

Check it out on GitHub

TweenCurve.js

While working on MandalaTron I needed to distribute values along a curve. This helped achieve 2 goals.

  1. Animate shapes or properties over time. (Tween)
  2. Skew a random distribution. (Curve)

And sadly I could find no super simple JS library that provided a number of different curving strategies. So, naturally, I looked up the math and then rolled my own.

Meet TweenCurve!

TweenCurve

// Tweens!
Tween.easeMethod(now, beginValue, valueChange, duration); // tweened value
Tween.easeInSine(20, 100, 100, 30);                       // 150
Tween.easeInOutExpo(10, 50, 10, 20);                      // 55

// Curves!
Curve.easeMethod(normalizedFloat);                        // curved value
Curve.easeOutQuint(0.9);                                  // 0.99999
Curve.easeInCubic(0.1);                                   // 0.001

Easy to integrate and use, was the main goal here. Check out the example page if you need some tweens or curves in your web or node.js project.

accurateInterval()

Developing the MandalaTron presented a problem.  Syncing visual timings with music required accuracy.  And that was accuracy that setTimeout() did not have.

Enter accurateInterval(). It schedules each successive run by comparing the curent time with the expected time that it should run next, ensuring that it stays as in-sync permanently.

MandalaTron

I’ve been experimenting with using HTML Canvas to create procedural rhythmic art.  And I am pretty pleased with the result.

(Safari and Firefox seem to have the best performance, sadly Chrome seems to struggle to keep the same framerates)