Tag Archives: Media Installations

Geometric Landscapes | TouchDesigner

rolling landscapeWorking on a new piece to premiere in Mexico I spent a lot of time experimenting with creating landscapes and backgrounds. Searching for a way into this exploration I wanted to play with the idea of instancing objects in 3D space, and the illusion of moving and shifting planes in space. This is already a popular visual style, and I was wanted to try my hand at exploring what it might look like to make something like this in TouchDesigner.

I’ve talked about instancing before, and so this challenge seemed like something that would be both fun, and interesting to play with. I also wanted something that mixed material methods – shaded, flat, and wire frame in appearance. Let’s take a look at how we can making something interesting happen using these ideas as a starting point.

Rendering is going to make or break us when thinking about how to set up this project, with that in mind it’s important to remember that a typcially rendering set-up needs something to be rendered (some geometry), a perspective from which to draw the object(s) (a camera), and a light source (a light, we don’t always need a light but as a rule of thumb it’s good to think that we need one). Our Geometry, Light, and Camera are all components, while our render operator is a Texture Operator (TOP). As a point of reference, here’s what a generic rendering set-up looks like:

classic render setup

We can tell our Render TOP to look at multiple Geometry Components, in the same network, or we can nest our active surfaces inside of a single GEO. Much of this depends on what you’re looking to create. The most important thing to consider is that surfaces operators (SOPs) are computed on the CPU unless placed inside of Geometry COMP – placed inside of a Geo they’re computed on the GPU instead. This makes a huge difference in your performance, and as a best practice it’s good to place any rendered geometry inside of a Geo.

Now let’s take a look at what our rendering set-up is going to look like for making our geometric landscape:

rolling landscape render set-up

Here we can see that we have a similar set-up on the left – a light, a geo, a camera, and a Render TOP. On the right I’ve got the geometry viewer open so we can see a little more about the relationship in the scene of the camera, light, and geometry. We can see that the camera is set above our geometry looking down, we can also see that we have a cone light set-up with a wide angle and a wide delta.

Now that we have a general sense of what we’re making let’s dig-in and make something interesting.

Lets start with an empty network. Lets start by adding a Geo to our network.

geo

To get started we’ll need to dive inside of geo to start making some changes. You can do this by double clicking on the Geo, using the quick key “i” (shortcut for inside), or by scroll wheel zooming into your geo. Inside our Geo we’ll see a torus that has it’s render and display flag set (the small blue and purple circles on the surface operator).

Inside of our geo let’s start by frist deleting the Torus SOP. Next let’s add a Grid SOP. Our grid is going to act the key generative element for us inside of this network – it’s going to give our surface its wire texture, the shading on the surface, and the location of where our spheres get placed. Our Grid is the central piece of what we’re making, and we’ll how in just a bit. Once we’ve added a grid to our network, we need to make a few changes to some its properties. First we want to make sure that it’s set to be a Polygon for Primitive type; we want to make sure that our orientation is set to ZX Plane; finally we want to change the size to 20 x 20.

grid setup

Next we’ll connect our Gird SOP to a Noise SOP. We’re going to use the Noise SOP to drive some of the shifting locations of the points in our grid. Before we move on, let’s make one quick change, On the Transform page in the Noise SOP’s parameter’s we can see that the translate z parameter is set to change with the second count of our project – me.time.seconds. This is excellent, and it keeps our noise animated over time, but it also means that we’re only working with 600 samples (in a default TouchDesigner network) because me.time.seconds is locked to our timeline. If, instead, we want noise that doesn’t have a hiccup every 10 seconds, we can instead use the call me.time.absSeconds. This uses the absolute number of seconds that TouchDesigner has been running to drive the transformations in the noise SOP. It’s a small change, but makes for a nicer look (at least in my opinion).

noise SOP

Next we’ll add a Material SOP to the our network. Our material SOP is going to allow us to assign a material to our grid. We’ll do this by also adding a Wireframe Material to our network. Before we assign our wireframe to our material we should see something like this:

before assignment

To assign the wireframe to the material, we’re just going to drag and drop the MAT onto the SOP.

assign MAT

Finally, we’re going to end this string by adding a Null SOP, making sure to turn on the render and display flags.

wire null

At this point we’ve made the Wireframe outlined elements of our grid. We’re now going to use the same data stream that we’ve already programmed to help us create a another layer of texture, and to create the locations for our spheres. Let’s start by adding our spheres to the network. To do this we’re going to do some instancing. When we’re instancing we need some location information for where to generate the copies of our source geometry. To get this information we’re going to use a SOP to CHOP to convert our SOP information into CHOP data.

SOPtoNow before we move on we need to change gears for just a moment. What we’re going to do next is to add another Geo inside of our current Geo – in Russian Nesting Doll Fashion. Why? Well, we’re going to do this in order to take advantage of the Geo’s ability to instance. Why not use the Copy SOP? I love me some Copy SOP action, but in this case the use of instancing is more efficient for this particular activity.

So, let’s add another Geo to our network:

new geo

Next we’re going to replace the torus inside of this Geo with a sphere. We’re also going to add a material inside of the geo. Easy, right? I’m going to set my sphere to be pretty small ( 0.09, 0.09, 0.09 ), I’m also going to make sure that I connect my sphere to a Null (in case I want to make any other changes), and then turn on the Null’s display and render flags. Finally I’m going to add a constant Material. When we’re all said and done you should have something like this:

inside the sphere

Excellent. Now we we back out of this nested Geo we should see just a single sphere. What?! Well, now we can set the Geo to instance – my favorite part.

small sphere

Let’s start by taking a look at the parameters of our Geo. Specifically, we want to look at the Instance page. Here we first need to turn on Instancing. Next we’re going to tell our Geo to look at the CHOP called sopto1. Finally we’ll set the TX, TY, and TZ parameters to correspond to the channels called tx, ty, and tz. If this seems like crazy talk, that’s okay – check out the picture below and it should make more sense:

instance page

We also want to head to Render Page of the Geo, and set this geo to use the material ./constant1 (this means, use the material inside of this geo called constant1 – ./ is a directory pointer indicating where to look for the thing in question, in this case a constant).

constant

Alright, now we can finally see some of our handy work – you should now see a sphere instanced at each of the vertices of the grid that we’re transforming with noise.

Spheres

Now let’s kick it up a notch. Now we’re going to add another Geo to our network.

last geo

We’re going to treat this Geo slightly differently. Inside let’s add an In SOP and a Phong Material. On our SOP In let’s make sure that the display and render flags are turned on, and for your SOP choose a nice dark color – I’m choosing a deep crimson.

geo3

The In SOP allows us to pass in the geometry that we’ve already made, acting as a kind of short-cut for us. When we go back to our Geo we just need to make sure that we’ve set our Render material to be ./phong1 – like with our constant this is the pathway to and the name of the material we want to use.

geo3mat

Alright, now you should have a network that looks something like this:

complete geo network

Now we’re ready to move out of our Geo and get ready to render our scene. Zooming out of our Geo we should see a network that looks something like this:

work space

In order to render our scene we need to revisit what we talked about at the beginning of the post – we need to add a Camera, a Light, and a Render TOP. Let’s go ahead and add these to our network.

rendersetupGeo

What gives?! This doesn’t look right at all. Well, part of what’s happening is that we don’t have our camera and light positioned correctly to render the scene correctly. To make this easier to understand, let’s change up our work space so we can use the geometry viewer (one of my favorite tools). Let’s start by dividing the workspace into two windows. We can do this by using the split work space icon in the menu bar:

menu bar

I like the vertical split, but you can choose whatever arrangement works for you. When you initially click on this split you’ll see two views of your current network location:

split step1

In order to see the geometry viewer we need to change the pane type selection for one of our windows. Clicking on the small drop down triangle will reveal a menu of network views. Let’s select Geometry Viewer from the list.

pane type

You should now see your network on one side, and the geometry from our Geo comp on the other side.

geo viewer

Now we’re cooking with gas. Alright, let’s make our lives just a little easier and change the scale of our light and camera to make them easier to see. Click on your light COMP, if your parameters window has disappeared you can bring it back by pressing “p” on your keyboard.” In the scale parameter, let’s turn that up to 10, 10, 10.

light scale

Great, now we can see part of the reason that our scene isn’t rendering the way we want it to. Our light is currently set up as a point light that’s positioned at the edge of our geometry. Let’s make some changes to our light’s properties so we get something closer to what we want. Let’s start by changing the location of our light, I’m going to place mine at 0, 10, 0.

lightplacement

Now let’s take a look at the Light page of the parameter’s window. Here I’m going to change my light to a Cone, and alter the cone size, delta, and rolloff. Experiment with different settings here to find something that you like.

light properties

Before you get too much experimenting done, however, you’ll probably notice that the cone of our light isn’t pointing towards the surface of our geometry. We can fix this by heading back to the Xform page of the parameters window, and setting the rotation values to -90, 0, 0.

light ROT

With our light starting to take shape, let’s get our camera in order. Over at our camera, let’s make the same initial change we made to the light and turn the scale up to 10, 10, 10 – this is going to make it much easier for us to find our camera.

cameraScale

With the scale turned up we can see that our camera needs to be translated back, up and rotated downwards so it’s looking at the geometry. After a little bit of adjusting I think I like my camera at:

TX    0
TY    6.2
TZ    18.9
RX    -16.8
RY    0
RZ    0

cameraTrans

Boom! Alright, let’s close our geometry viewer and take a closer look at our Render TOP to see what we’ve made.

finalRender

Nice work. This is a good looking start, and now that you know how it’s made you can start to really have fun – camera placement, light placement, noise amplitude, you name it, go crazy, make something fun or weird, or just silly.

Sending and Receiving OSC Values with TouchDesigner

The other day I posted look at how to send and receive TouchOSC data with Troikatronix’s Isadora. In an effort to diversify my programming background I’m often curious about how to translate a given task from one programing environment to another. To this end, I was curious about repeating this same action in Derivative’s TouchDesigner. If you’re one of a growing number of artists interested in visual programming, VJing, media design, interactive system design, media installation, or media for live performance than it’s well worth your time to look at both Isadora and TouchDesigner.

That said, I initially started thinking about control panel interfaces for TouchOSC when I saw Graham Thorne’s instructional videos about how to transmit data from Isadora to Touch OSC (If you have a chance take a look at his posts here: Part 1 & Part 2). Last year I used TouchOSC accelerometer data to drive an installation at Bragg’s Pie Factory in Downtown Phoenix, but I didn’t do much with actual control panels. Receiving data from TouchOSC in TouchDesigner is a piece of cake, but sending it back out again didn’t work the first time that I tried it. To be fair, I did only spend about 3 minutes trying to solve this particular question as it was a rather fleeting thought while working on another project.

Getting this to working Isadora, however, gave me a renewed sense of focus and determination, so with a little bit of experimenting I finally got this working the way that I wanted. While there are any number of things you might do with TouchDesigner and TouchOSC, I wanted to start with a simple deliverable: I wanted the position of Slider 1 to inversely change the position of Slider 2, and vise versa. In this way, moving Slider 1 up moves Slider 2 down, and moving Slider 2 up moves Slider 1 down. In performance setting it’s unlikely that I’d need something this simple, but for the sake of testing an idea this seemed like it would give me the kind of information that I might need.

For the most part this is pretty easy, but there are a few things to watch for in order to get the results that you want.

Here’s the network stream of Chanel Operators (CHOPs):

OSC In – Select – Math – Rename – OSC Out

OSC In

Derivative Documentation on the OSC In CHOP 

The OSC in CHOP is pretty excellent out the gate. We need to do a little configuring to get this up and running, but after that we’re truly off to the races. First off it’s important to make sure that your TouchOSC device is connected to your network and broadcasting to your TouchDesigner machine. If this process is new to you it’s worth taking a little time to read about out to set this all up – start on this page if this is new to you. Once your TouchOSC device is talking to your TouchDesigner network we’re ready to start making magic happen. One of the first things that I did was to activate (just adjust up or down) the two faders that I wanted to work with. This made sure that their IDs were sent to TouchDesigner, making it much easier for me to determine how I was going to rename in them in the future.

Screenshot_100913_094345_PMSelect

Derivative Documentation on the Select CHOP

When you’re working with the OSC In CHOP, all of your TouchOSC data comes through a single pipe. One of the first things to do here is to use the select CHOP to single out the signal that you’re looking for (as a note, you could also export the Channel in question to a Null, or handle this a variety of other ways – in my case a Select CHOP was just the first method I tried). This operator allows you to pull out a single Channel from your OSC In.

Screenshot_100913_094428_PM

Math

Derivative Documentation on the Math CHOP

Next I wanted to remap the values of my incoming signal to be inverted. A Math CHOP lets us quickly remap the range for our data by changing a few values in the Range Tab of that parameters window. You’ll notice that you first need to specify the from range (what’s the range of incoming values), and then specify the to range (what’s the range of outgoing values). In my case, the from values are 0 and 1, and the to values are 1 and 0.

Screenshot_100913_094528_PM

Rename

Derivative Documentation on the Rename CHOP

The Rename CHOP is one of the most important steps in your whole network. When communicating with TouchOSC you need to know exactly what object you’re trying to drive with your data. To accomplish this you need to know the UDP address of your device, the port number that you’re board casting to, and the name of the slider or button that you’re wanting to change. While we can specify the UPD address and port number with another CHOP, we need to use the Rename CHOP (a quick aside – there are, of course, other ways to do this – this just happens to be the method that I got working after a bit of trial and error) to specify the name of the object that we’re driving. In my case I wanted fader 1 to drive fader 2. You’ll notice that formatting of the fader is important. In this particular OSC layout there are multiple tabs. We can see this as evidenced by the “1/” preceding the name of our fader. In changing the name of this input we need to enter the precise name of the fader that we want to change – this is why we activated both faders initially, this gave us the names of both objects in the TouchOSC panel. You’ll notice that changed the name of my fader to “1/fader2”.

Screenshot_100913_094606_PM

OSC Out

Derivative Documentation on the OSC Out CHOP 

Now that we’ve processed and named our signal we’re ready to send it back out to TouchOSC. In this CHOP we need to input the UDP address of the TouchOSC device (you can find this in the info tab of the TouchOSC app) as well as the port that we’re sending to. We also want to make sure that we’re sending this as a sample, and finally we want to turn off “send events every cook.” This last step is important because it ensures that we’re only sending values when they change. If we send messages for every cook we won’t be able to see the inverse relationship between our two sliders. Correction, we’ll be able to see the relationship so long as we don’t set up slider 2 to drive slider 1. In order to create an inverse relationship with both sliders we only want to transmit data as it’s changed (otherwise we’ll find that we’re fighting with the constantly streamed position data from our slider that isn’t being activated).

Screenshot_100913_094705_PM

The last step in this process is to copy the whole string and set up fader 2 to transmit to fader 1, this last step allows both faders to drive one another in turn.

There you have it. You’ve now successfully configured your TouchDesigner network to receive and send data to a mobile device running TouchOSC.

TouchDesigner | Sculpture

In the ever growing list of tools that I’m experimenting with Derivative’s TouchDesigner is a tool that time and again keeps coming up as something that’s worth learning, experimenting with, and developing competencies around it’s work flow. TD is a nodal environmental called a network. Inside of the network nodes can be directly connected by by exporting parameters. 

Nodes, also called Ops (Operations) are split into families specific to the characteristics of their behavior: CHOPS (Channel Operators)TOPS (Texture Operators)SOPS (Surface Operators)MATS (Materials), and DATS (Data Operators). Nodes from within the same families can pass data directly to one another through patch cords (similar to MaxMSP and Isadora). The output of nearly every node can be passed into other nodes by exporting parameter values. This process divides the process of passing data values into two distinct processes, one that’s centered around like to like processes and one that’s about moving from like to different. 

TouchDesigner’s nodes are the most powerful when they’re connected. Like Max single nodes do little by themselves, and are the most powerful when they’re connected. Also like Max the flexibility of TD is it’s ability to build nearly anything, and with that comes the fact that little is already built. Similar to Isadora is the native ability to build user interfaces as a part of the very fabric of building a program / user experience. 

One of the projects that I’m working on this semester is for a sculpture course. This course, called New Systems, is intended to address the link between media and sculpture. One of the areas that I’m interested in exploring is about collecting data from a circus apparatus and using that to drive a visualization in performance. I’m most interested in the direct link between how an apparatus is behaving and how that data can be interpreted in other ways. To that end this semester I set to the work of building an apparatus and determining how to parse that data. In my case I decided to use this opportunity to experiment with TouchDesigner as a means of driving the media. While I was successful in welding together a square from stainless steel, after some consultation from my peers in my sculpture course it was determined that this structure was probably not safe to perform on. Originally I had planned to use a contact mic to capture some data from my interaction with the apparatus, and after a little bit of thinking and consultation with my adviser (Jake Pinholster) I decided that gyroscope data might be more useful.

My current plan is to move away from this being a performance apparatus and instead think of it as installed sculptural piece that serves as a projection surface. For data I’ll be using an iPod Touch running Hexler’s touch OSC. Touch OSC passes data using UDP packets to communicate over a wired or wireless network using Open Sound Control (OSC). One of the many things that Touch OSC can do is pass the accelerometer data from an iOS device out to other applications. In my case Touch OSC is passing this information to TouchDesigner. TD is then used to pull this information and drive some of the media.

One of the challenges that my adviser posed in this process was to create three scenes that the  media moved through. For the sake of experimentation I applied this challenge to the idea of working with containers in TouchDesigner. Containers are a method of encapsulation in TD, they’re a generic kind of object that can hold just about any kind of system. In my case I have three containers that are equivalent to different scenes. The time line moves the viewer through the different containers by cross-fading between them. Each container holds it’s own 3D environment that’s rendered in real time and linked to the live OSC inputs coming from an iPod touch. 

The best way to detail the process of programming this installation is to divide it up into the component pieces that make it all work. The structure of this network is defined by three hierarchical levels: The Container, Control, and Final Output; the individual composited scene, the underlying geometry.


Want to work your way through these ideas one chunk at a time? Visit their individual Posts here:

The Underlying Geometry

The Individual Composited Scene

The Container, Control, and Final Ouput

Want to work you way through the whole process, keep on scrolling.


The Underlying Geometry

One of the benefits of working with TouchDesigner is the ability to work in 3D. 3D objects are in the family of operators called SOPs – Surface Operators. One of the aesthetic directions that I wanted to explore was the feeling of looking into a long box. The world inside of this box would be characterized by examining artifacts as either particles or waves with a vaguely dual-slit kind of suggestion. With that as a starting point I headed into making the container for these worlds of particles and waves.

Before making any 3D content it’s important to know how TouchDesigner processes these objects in order to display them. On their own, Surface Operators can’t be displayed as a rendered texture. In TouchDesigner’s idiom textures are two-dimension surfaces, and it follows that the objects that live in that category are called TOPs, Texture Operators. Operators from different families can’t be directly connected with patch chords. In order to pass the information from a SOP to a TOP one must use a TOP called a Render. The Render TOP must be connected to three COMPs (Compositions) in order to create an image that can be displayed. The render TOP requires a Geometry COMP (something to be rendered), a Light COMP (something to illuminate the scene), and a Camera COMP (the perspective from which the object is to rendered). In this respect TD pulls from conventions familiar to anyone who has worked with Adobe’s After Effects. 

Knowing the component pieces required in order to successfully render a 3D object it’s easier to understand how I started to create the underlying geometry. The Geometry COMP is essentially a container object (with some special attributes) that holds the SOPs responsible for passing a surface to the Render TOP. The default Geometry COMP contains a torus as a geometry. 

We can learn a little about how the COMP is working by taking a look inside of the Geometry object. 

Here the things to pay close attention to are the two flags on the torus object. You’ll notice in the bottom right corner there is a purple and a blue circle that are illuminated. The purple circle is a “Render Flag” and tells TouchDesigner to render the object, and the blue circle is a “Display Flag” which tells TouchDesigner that this is the object that should be displayed in the Geometry COMP.

Let’s take a look at the network that I created.

Now let’s dissect how my geometry network is actually working. At first glance we can see that multiple objects are being combined into a single piece of geometry that’s ultimately being passed out of this Geometry COMP. 

If we look closer we’ll see that here that the SOP network looks like this:

Grid – Noise – Transform – Alpha Noise (here the bypass flag is turned on)

Grid creates a plane that’s created out of polygons. This is different from a rectangle that’s only composed four points. In order to create a surface that can deform I needed a SOP points in the middle of it. The grid is attached to a Noise SOP that’s animating the surface. Noise is attached to a transform SOP that allows me to change the position of this individual plane. The last stop in this chain is another Noise SOP. Originally I was experimenting with varying the transparency of the surface. Ultimately, I decided to move away from this look. Rather than cutting this out of the chain, I simply turned on the Bypass Flag which turns off this single SOP. This whole chain is repeated eight times (for a total of eight grids). 

These Nine planes are then connected so that the rest of the network looks like this:

Merge – Transform – Facet – Texture – Null – Out

Merge takes all of the inputs and puts them together into a single piece of geometry. Transform allows me to move object as a whole in space. Facet is a handy operator that allows you to compute the normals’ of a geometry, which is useful for creating some more dynamic shading. Texture was useful for another direction that I was exploring, ultimately  ended up turning on the bypass flag for this SOP. A null, like in other environments, is really just a place holder kind of object. In the idiomatic structure of TouchDesigner, the Null is operationally an object that one places at the end of operation string. This is considered a best practice for a number of reasons. High on the list of reasons to end a string in a Null is because this allows easy access for making changes to a string. TouchDesigner allows the programmer to insert operations between objects. By always ending a string in a Null it becomes very easy to make changes to the stream without having to worry about re-exporting parameters. Finally all of this ends in an Out. While the Out isn’t necessary for this string, at one point I wasn’t sure if I was going to pass this geometry into another component. Ending in the Out ensured that I would have that flexibility if I needed it.


The Individual Composited Scene

There are always large questions to answer when thinking about creating an interactive work: Who is it for? What does it look likes? What are you trying to communicate? How much instruction do you provide, how little instruction do you provide? And on and on.  As I started to think about how this piece was going to work as an installation rather than as a performance apparatus, I started by thinking about what kind of data I could use to drive the visual elements of this work. One of the sensors that I knew I could easily incorporate into my current sculptural configuration was a an iPod Touch. The Touch has an on-board gyroscope and accelerometer. After a conversation with my adviser (Jake Pinholster) we decided that this would be a direction of exploration worth pulling apart, and from there I went back to TouchDesigner to start thinking about how I wanted to incorporate live data into the piece I was making.

When dealing with a challenge like building an interactive sculptural system that has at least three different visualizations, it can be challenging to think about where to start. Different programmers are bound to have different approaches to addressing this question. My approach was to start by thinking about what kind of input data I had to work with. Because I was dealing with a sensor that relayed spatial information, this also help me think about how to represent that data. Next I thought about what different kinds of ways I wanted to present this information, and finally I addressed how to playback this experience for users. Some of my more esoteric and existential questions (why am I making this? what does it mean? what does it represent?) were addressed through the methodical programming process, and others were sussed out over contemplative cups of coffee. As much as I wish that these projects could have a straight line of execution, a checklist even, I’m discovering more and more that the act of creating and programming is often a winding path with happy (and unhappy) discoveries along the way.

My first step on this journey, however, was to address what kind of inputs I had to use. Hexler has an excellent app for sending UDP messages over wireless connections called Touch OSC. OSC, or Open Sound Control, is a communications protocol that uses UDP messages to send data over wired and wireless networks. It’s functionally similar to MIDI and has some additional flexibilities and constraints. In the case of touch OSC, one of the parameters that you can enable from your iOS device to send xyz data from the accelerometer. Getting Touch OSC up and running does require a few steps to get the ball rolling. First both the computer that’s receiving and the device that’s broadcasting need to be on the same network. Your broadcasting device will need the IP address of the receiving computer, and a specified port to send the data to (how to find your IP address on a Mac, and on a PC). Once this information is set on your broadcasting device, it’s time to add a Channel Operator to your TouchDesigner network.

In TouchDesigner, there is a CHOP called “OSC in.” This CHOP will allow you to receive OSC data over a wireless network. Once you’ve added the CHOP to your TD network you’ll have to specify the port that Touch OSC is broadcasting to, and then you should be in business. In my case once this was set up I could instantly see a stream of accelerometer data coming from my iPod Touch. In order to use these values, however, I needed to take some additional steps. The raw OSC data from Touch OSC comes in as a range of data from -1 to 1. Additionally, the data comes in from one CHOP. My flow of operators looks like:

OSC In – Select – Lag – Math – Null

OSC In is the data input. The CHOP Select allows you to select a single channel out of a bundle of channels. In this case I used this to separate my X, Y, and Z inputs into different streams. The Lag chop helps to smooth out the attack and decay rates of input data. In my case this ensured that my final values used to control another object where kept from being too jittery. The Math CHOP is tremendously powerful, in my case I wanted to be able to map the values of my raw data [ -1 to 1 ], to a larger range of values, say 0 to 200. Finally I ended my string in a Null. A null in this case is very useful in case I need to add any other operators into my string.

Before thinking about how to use these values, it’s important to take a moment to revisit how geometry is rendered in TouchDesigner. The geometry COMPs that are used to create the objects to be displayed can’t be visualized without using a render TOP. The render TOP requires three components in order to generate an image that can be seen. Render requires a source geometry, light, and camera. The Geometry COMP provides the location of surfaces, the light provides the necessary information about how the object is being lit. The camera COMP controls the perspective that the object is being rendered from. This is similar to an approach that one might use when creating 3D content in After Effects – an object to be rendered, a light so the object can be seen, and a camera to control the perspective the audience sees of the object. Because we need to think of rendering by combining multiple COMPs, that can inform how we use live data. 

With some scaled values process and ready to export I was ready to think about how these values could influence the viewers perspective of the geometry. One of my initial thoughts was to render a cube that a user could look inside of. As the observer changed the orientation of the sensor, the virtual environment would also change in kind. While it’s possible to do this by rotating and translating the geometry itself, I instead decided to focus on the orientation of the camera instead. This has a few advantages. One important advantage is the ability to tell a camera to look directly at a specified geometry. This means that in translating the camera (left or right, up or down, in or out) the camera stays focused on the center of the target geometry. This makes changing perspective much simpler.  

Initially I was thinking of rendering the entire 3D scene as a single geometry. In doing this, however, I was experiencing some challenges when thinking about the placement of lights and the overall organization of the geometry, and in applying texture to the surfaces. By using a Phong shader one can apply texture maps to the 3D geometry COMPs that have been created. By separating the interior and exterior pieces of the geometry and then compositing them after rendering I was able to apply different shaders to each geometry.

The portion of my network responsible for compositing the geometry looks like this:

Render 1, Render 2, Constant (black solid) – Composit – Transform – Null – Out

 Render 1, Render 2, and the Constant are the three source surfaces. Render 1 is the box, Render 2 is the merged set of waves, and the Constant is a black background. Another approach to this would be to set one of the camera background’s as black. These three flow into a Composit COMP. Next is the Transition COMP (this allowed for some small adjustments that needed to be made in order to help align the projection with the sculpture. Originally I made this string with a Null as the final output of this Component. I would eventually find that I needed an Out to pass this scene into another display module. 

I used the same techniques as above for the other two scenes – starting with establishing my data stream, generating the geometry, rendering out layers to be composited and then passed out to the visual stream.

Are these pictures too small? You can see higher quality versions by looking at this Flickr Gallery: Graduate School Documentation


The Container, Control, and Final Output

In thinking about how to meet the objectives that I had for this piece, one of my central questions was how to make sure that I could move through three cued scenes – either with manual or automatic triggers. I knew that I had three different aesthetic environments that I wanted to move through. I explored several different options, and the one that ultimately made sense to me given my current level of proficiency (at this point I had only been programming in Touch for a total of three weeks) in TouchDesigner was to use a cross fading approach. Here’s what the whole network looks like:

In thinking about how to ensure that I was being efficient I decided to encapsulate my three different scenes in their own respective containers. You’ll notice on the left hand side that there are three containers – each holding it’s own 3D environment. These are joined through Corss fading TOPs though a final composite (for a mask) until ending in a Null that was used as the display canvas. 

I spent a lot of time thinking about how this piece was going to be both interactive and autonomous. It needed to be interactive in that the user was able to see how their interaction with an object was driving the visual media; it needed to be autonomous in its ability to transition between scenes and then loop back to the beginning of the network. I don’t think I’ve totally cracked the nut that is the right balance of interactivity and self-directed programming, but it feels like I did make strides towards addressing this question. My solution to these issues was to allow the interaction with the projection to be centered around the control of perspective, but to drive the transitions through the scenes with time-line triggers.

Unlike some other interactive programming environments, TouchDesigner has a timeline built into the fabric of the control system. The Timeline is based in frames, and the programmer can specify the number of frames per second as well as the total number of frames for a given project. My Timeline triggering system was the following string of CHOPs:

Timeline – Trigger – Null

Timeline reports out the current frame number. The trigger CHOP can be set to trigger at a given threshold (or in my case a frame number). This in turn is passed to a null and exported to a Corssfade TOP as a rate for crossfade. The Crossfades are daisy-chained together before finally being attached to the null that’s output to the projector.

With the system working I also needed to make a mask for the final projection to ensure that I wasn’t displaying any empty grid onto the floor of the gallery where this was being installed. I would typically make a mask for something like this in Photoshop, but decided to try making this all in the TouchDesigner programming environment. My TOP operator string for this looked like:

Contsant – Transform – Blur – Composite

I started by creating a black constant that’s then passed to a transform so that it can be positioned into place. This is then passed to a blur to soften the edges, and finally to a composite to create a mask that contains a left, right, top, and bottom side. In hindsight I realize that I could use a single constant passed to four transform TOPs, to be a little more tidy. The mask as a composited object is then composited with final render stream before being passed to the Null that’s connected to the projector. 

In the end I’m fairly happy with this project. t’s been a steep learning curve, but well worth the hassle, angst, and late nights. It’s no small thing to have made a piece of interactive media driven sculpture in a programming environment where that’s totally new to me. For as hard as all of this work has proven to be, I have to remind myself that I’m actively doing the work that I came to Graduate School to do. Everyday I realize that I’ve been changed by my time in the desert, and by my time with the gifted and brilliant artists and friends that I’ve found here. 

Are these pictures too small? You can see larger versions of them here:

TouchDesigner | The Container, Control, and Final Ouput

In thinking about how to meet the objectives that I had for this piece, one of my central questions was how to make sure that I could move through three cued scenes – either with manual or automatic triggers. I knew that I had three different aesthetic environments that I wanted to move through. I explored several different options, and the one that ultimately made sense to me given my current level of proficiency (at this point I had only been programming in Touch for a total of three weeks) in TouchDesigner was to use a cross fading approach. Here’s what the whole network looks like:

In thinking about how to ensure that I was being efficient I decided to encapsulate my three different scenes in their own respective containers. You’ll notice on the left hand side that there are three containers – each holding it’s own 3D environment. These are joined through Corss fading TOPs though a final composite (for a mask) until ending in a Null that was used as the display canvas. 

I spent a lot of time thinking about how this piece was going to be both interactive and autonomous. It needed to be interactive in that the user was able to see how their interaction with an object was driving the visual media; it needed to be autonomous in its ability to transition between scenes and then loop back to the beginning of the network. I don’t think I’ve totally cracked the nut that is the right balance of interactivity and self-directed programming, but it feels like I did make strides towards addressing this question. My solution to these issues was to allow the interaction with the projection to be centered around the control of perspective, but to drive the transitions through the scenes with time-line triggers.

Unlike some other interactive programming environments, TouchDesigner has a timeline built into the fabric of the control system. The Timeline is based in frames, and the programmer can specify the number of frames per second as well as the total number of frames for a given project. My Timeline triggering system was the following string of CHOPs:

Timeline – Trigger – Null

Timeline reports out the current frame number. The trigger CHOP can be set to trigger at a given threshold (or in my case a frame number). This in turn is passed to a null and exported to a Corssfade TOP as a rate for crossfade. The Crossfades are daisy-chained together before finally being attached to the null that’s output to the projector.

With the system working I also needed to make a mask for the final projection to ensure that I wasn’t displaying any empty grid onto the floor of the gallery where this was being installed. I would typically make a mask for something like this in Photoshop, but decided to try making this all in the TouchDesigner programming environment. My TOP operator string for this looked like:

Contsant – Transform – Blur – Composite

I started by creating a black constant that’s then passed to a transform so that it can be positioned into place. This is then passed to a blur to soften the edges, and finally to a composite to create a mask that contains a left, right, top, and bottom side. In hindsight I realize that I could use a single constant passed to four transform TOPs, to be a little more tidy. The mask as a composited object is then composited with final render stream before being passed to the Null that’s connected to the projector. 

 In the end I’m fairly happy with this project. t’s been a steep learning curve, but well worth the hassle, angst, and late nights. It’s no small thing to have made a piece of interactive media driven sculpture in a programming environment where that’s totally new to me. For as hard as all of this work has proven to be, I have to remind myself that I’m actively doing the work that I came to Graduate School to do. Everyday I realize that I’ve been changed by my time in the desert, and by my time with the gifted and brilliant artists and friends that I’ve found here. 

 Are these pictures too small? You can see larger versions of them here:

TouchDesigner | The Individual Composited Scene

There are always large questions to answer when thinking about creating an interactive work: Who is it for? What does it look likes? What are you trying to communicate? How much instruction do you provide, how little instruction do you provide? And on and on.  As I started to think about how this piece was going to work as an installation rather than as a performance apparatus, I started by thinking about what kind of data I could use to drive the visual elements of this work. One of the sensors that I knew I could easily incorporate into my current sculptural configuration was a an iPod Touch. The Touch has an on-board gyroscope and accelerometer. After a conversation with my adviser (Jake Pinholster) we decided that this would be a direction of exploration worth pulling apart, and from there I went back to TouchDesigner to start thinking about how I wanted to incorporate live data into the piece I was making.

When dealing with a challenge like building an interactive sculptural system that has at least three different visualizations, it can be challenging to think about where to start. Different programmers are bound to have different approaches to addressing this question. My approach was to start by thinking about what kind of input data I had to work with. Because I was dealing with a sensor that relayed spatial information, this also help me think about how to represent that data. Next I thought about what different kinds of ways I wanted to present this information, and finally I addressed how to playback this experience for users. Some of my more esoteric and existential questions (why am I making this? what does it mean? what does it represent?) were addressed through the methodical programming process, and others were sussed out over contemplative cups of coffee. As much as I wish that these projects could have a straight line of execution, a checklist even, I’m discovering more and more that the act of creating and programming is often a winding path with happy (and unhappy) discoveries along the way.

My first step on this journey, however, was to address what kind of inputs I had to use. Hexler has an excellent app for sending UDP messages over wireless connections called Touch OSC. OSC, or Open Sound Control, is a communications protocol that uses UDP messages to send data over wired and wireless networks. It’s functionally similar to MIDI and has some additional flexibilities and constraints. In the case of touch OSC, one of the parameters that you can enable from your iOS device to send xyz data from the accelerometer. Getting Touch OSC up and running does require a few steps to get the ball rolling. First both the computer that’s receiving and the device that’s broadcasting need to be on the same network. Your broadcasting device will need the IP address of the receiving computer, and a specified port to send the data to (how to find your IP address on a Mac, and on a PC). Once this information is set on your broadcasting device, it’s time to add a Channel Operator to your TouchDesigner network.

In TouchDesigner, there is a CHOP called “OSC in.” This CHOP will allow you to receive OSC data over a wireless network. Once you’ve added the CHOP to your TD network you’ll have to specify the port that Touch OSC is broadcasting to, and then you should be in business. In my case once this was set up I could instantly see a stream of accelerometer data coming from my iPod Touch. In order to use these values, however, I needed to take some additional steps. The raw OSC data from Touch OSC comes in as a range of data from -1 to 1. Additionally, the data comes in from one CHOP. My flow of operators looks like:

OSC In – Select – Lag – Math – Null

OSC In is the data input. The CHOP Select allows you to select a single channel out of a bundle of channels. In this case I used this to separate my X, Y, and Z inputs into different streams. The Lag chop helps to smooth out the attack and decay rates of input data. In my case this ensured that my final values used to control another object where kept from being too jittery. The Math CHOP is tremendously powerful, in my case I wanted to be able to map the values of my raw data [ -1 to 1 ], to a larger range of values, say 0 to 200. Finally I ended my string in a Null. A null in this case is very useful in case I need to add any other operators into my string.

Before thinking about how to use these values, it’s important to take a moment to revisit how geometry is rendered in TouchDesigner. The geometry COMPs that are used to create the objects to be displayed can’t be visualized without using a render TOP. The render TOP requires three components in order to generate an image that can be seen. Render requires a source geometry, light, and camera. The Geometry COMP provides the location of surfaces, the light provides the necessary information about how the object is being lit. The camera COMP controls the perspective that the object is being rendered from. This is similar to an approach that one might use when creating 3D content in After Effects – an object to be rendered, a light so the object can be seen, and a camera to control the perspective the audience sees of the object. Because we need to think of rendering by combining multiple COMPs, that can inform how we use live data. 

With some scaled values process and ready to export I was ready to think about how these values could influence the viewers perspective of the geometry. One of my initial thoughts was to render a cube that a user could look inside of. As the observer changed the orientation of the sensor, the virtual environment would also change in kind. While it’s possible to do this by rotating and translating the geometry itself, I instead decided to focus on the orientation of the camera instead. This has a few advantages. One important advantage is the ability to tell a camera to look directly at a specified geometry. This means that in translating the camera (left or right, up or down, in or out) the camera stays focused on the center of the target geometry. This makes changing perspective much simpler.  

Initially I was thinking of rendering the entire 3D scene as a single geometry. In doing this, however, I was experiencing some challenges when thinking about the placement of lights and the overall organization of the geometry, and in applying texture to the surfaces. By using a Phong shader one can apply texture maps to the 3D geometry COMPs that have been created. By separating the interior and exterior pieces of the geometry and then compositing them after rendering I was able to apply different shaders to each geometry.

The portion of my network responsible for compositing the geometry looks like this:

Render 1, Render 2, Constant (black solid) – Composit – Transform – Null – Out

 Render 1, Render 2, and the Constant are the three source surfaces. Render 1 is the box, Render 2 is the merged set of waves, and the Constant is a black background. Another approach to this would be to set one of the camera background’s as black. These three flow into a Composit COMP. Next is the Transition COMP (this allowed for some small adjustments that needed to be made in order to help align the projection with the sculpture. Originally I made this string with a Null as the final output of this Component. I would eventually find that I needed an Out to pass this scene into another display module. 

I used the same techniques as above for the other two scenes – starting with establishing my data stream, generating the geometry, rendering out layers to be composited and then passed out to the visual stream.

Are these pictures too small? You can see higher quality versions by looking at this Flickr Gallery: Graduate School Documentation

TouchDesigner | The Underlying Geometry

One of the benefits of working with TouchDesigner is the ability to work in 3D. 3D objects are in the family of operators called SOPs – Surface Operators. One of the aesthetic directions that I wanted to explore was the feeling of looking into a long box. The world inside of this box would be characterized by examining artifacts as either particles or waves with a vaguely dual-slit kind of suggestion. With that as a starting point I headed into making the container for these worlds of particles and waves.


Before making any 3D content it’s important to know how TouchDesigner processes these objects in order to display them. On their own, Surface Operators can’t be displayed as a rendered texture. In TouchDesigner’s idiom textures are two-dimension surfaces, and it follows that the objects that live in that category are called TOPs, Texture Operators. Operators from different families can’t be directly connected with patch chords. In order to pass the information from a SOP to a TOP one must use a TOP called a Render. The Render TOP must be connected to three COMPs (Compositions) in order to create an image that can be displayed. The render TOP requires a Geometry COMP (something to be rendered), a Light COMP (something to illuminate the scene), and a Camera COMP (the perspective from which the object is to rendered). In this respect TD pulls from conventions familiar to anyone who has worked with Adobe’s After Effects. 

Knowing the component pieces required in order to successfully render a 3D object it’s easier to understand how I started to create the underlying geometry. The Geometry COMP is essentially a container object (with some special attributes) that holds the SOPs responsible for passing a surface to the Render TOP. The default Geometry COMP contains a torus as a geometry. 

We can learn a little about how the COMP is working by taking a look inside of the Geometry object. 


Here the things to pay close attention to are the two flags on the torus object. You’ll notice in the bottom right corner there is a purple and a blue circle that are illuminated. The purple circle is a “Render Flag” and tells TouchDesigner to render the object, and the blue circle is a “Display Flag” which tells TouchDesigner that this is the object that should be displayed in the Geometry COMP.

Let’s take a look at the network that I created.

Now let’s dissect how my geometry network is actually working. At first glance we can see that multiple objects are being combined into a single piece of geometry that’s ultimately being passed out of this Geometry COMP. 

If we look closer we’ll see that here that the SOP network looks like this:

Grid – Noise – Transform – Alpha Noise (here the bypass flag is turned on)

Grid creates a plane that’s created out of polygons. This is different from a rectangle that’s only composed four points. In order to create a surface that can deform I needed a SOP points in the middle of it. The grid is attached to a Noise SOP that’s animating the surface. Noise is attached to a transform SOP that allows me to change the position of this individual plane. The last stop in this chain is another Noise SOP. Originally I was experimenting with varying the transparency of the surface. Ultimately, I decided to move away from this look. Rather than cutting this out of the chain, I simply turned on the Bypass Flag which turns off this single SOP. This whole chain is repeated eight times (for a total of eight grids). 

These Nine planes are then connected so that the rest of the network looks like this:

Merge – Transform – Facet – Texture – Null – Out
Merge takes all of the inputs and puts them together into a single piece of geometry. Transform allows me to move object as a whole in space. Facet is a handy operator that allows you to compute the normals’ of a geometry, which is useful for creating some more dynamic shading. Texture was useful for another direction that I was exploring, ultimately  ended up turning on the bypass flag for this SOP. A null, like in other environments, is really just a place holder kind of object. In the idiomatic structure of TouchDesigner, the Null is operationally an object that one places at the end of operation string. This is considered a best practice for a number of reasons. High on the list of reasons to end a string in a Null is because this allows easy access for making changes to a string. TouchDesigner allows the programmer to insert operations between objects. By always ending a string in a Null it becomes very easy to make changes to the stream without having to worry about re-exporting parameters. Finally all of this ends in an Out. While the Out isn’t necessary for this string, at one point I wasn’t sure if I was going to pass this geometry into another component. Ending in the Out ensured that I would have that flexibility if I needed it.

TouchDesigner | Time Trigger

Ultimately I don’t know how practical this method is, but at the moment it feels like a victory. Creating a time based trigger in TouchDesigner can be accomplished a ton of ways, the method I’m using uses the following CHOPS: LFO, Constant, Count, Trigger. 

The set to the same sample rate as the project will cross 0 once per second. The constant indicates the rate at which you wish to count. Count will use the input from the LFO and the Constant to count at the rate specified by the constant every time the LFO crosses 0. This is attached to a trigger CHOP. The trigger’s threshold can e set to to the time (in seconds) appropriate for the transition. Bang. A time based triggering method using four CHOPs.



It’s also worth nothing that another method for achieving this same  triggering method would be to use just two CHOPs: Timeline and Trigger. The Timeline chop will report out the current position as a frame number. Provided that you know the time (and therefore frame number) you can use this to set the threshold for a trigger value.