Tag Archives: Grad School

Inspired by Rutt Etra | TouchDesigner

Fall of the House of EscherBack when I was working on The Fall of the House of Escher I became obsessed with live z-displacement in the media. That particular show was wrestling with questions of quantum physics, time, reality, and all manner of perceptual problems and so it made a lot of sense to be love the look and suggested meaning inherent in the displacement of an image in a third dimension. This particular technique is perhaps most well known to us because of the Rutt Etra Video Synthesizer. This video synth was an attempt to begin manipulating video in same manner as sound was being manipulated in the 1970s, and was truly a ground breaking examination of our relationship to video – live video especially. One of the most famous elements of the Rutt Etra Synth was z displacement, the change in the depth of an image based on its luminance.

You can get a better sense of what this kind of effect looks like by playing with the Rutt Etra-izer online. This simple tool lets you play with one slice of what the original video synth did. Additionally, if you’re a mac user you might want to check out what v002 had to share when it comes to plug-ins, as well as some thoughts from Bill Etra. You can find more from v002 here: Rutt Etra v002

So that’s all well and good, but what am I after? Well, in the pursuit of better understanding how to program with TouchDesigner, as well as how to explore some of the interesting ideas from the 1970s, I wanted to know how to replicated this kind of look in a TouchDesigner network. There are plenty of other people who have done this already, and that’s awesome. I, however, happen to subscribe to the kind of art philosophy asks students to copy the work of others – to practice their hands at someone else’s technique, to take what works and leave what doesn’t. Art schools have often required students to copy the work of masters in order to foster a better appreciation and understanding of a particular form, and I think there’s a lot too that in programming. So today, we’ll look at how to make this kind of effect in TouchDesigner and then ask how we might manipulate this idea in ways that differ from how the original method was intended to  work.

Rutt Etra almostTo begin, this idea started when I was looking through the wiki page about Generative Design. Specifically, the sample network talking about image manipulation really got my attention. Buried in this example network is something that’s Rutt Etra flavored, and it’s from this example that I started to pull out some ideas to play with. As we work our way through this example it’ll be easy to get lost, frustrated, or confused. Before that happens to you, take a moment to think about what we’re trying to do. Ultimately, we want to take an image (video later, but for now an image) and add together the RGBA values of an individual pixel and use that number to transform said pixel in a z dimension. In other words, we’re really after creating a faux depth map out of an image. It’s also important to remember that using an image that’s 1920 x 1080 means we’re talking about 2,073,600 pixels. That’s a lot of points, so we’re going to start by creating a grid that simplifies those dimensions – partially for our own sanity, partially for the sake of the processing involved, and partially to remain true the aesthetic of the original idea. Once we replicate the original, then we can start to talk about where we can start to play. That’s enough disclaimers for us to get started, so let’s do some programming.

Let’s start by looking at the whole network:

From this vantage point we can see that we’re going to make a network that uses a little bit of everything – some texture operators (TOPs), some channel operators (CHOPs), and some surface operators (SOPs).

Starting with Texture Operators

First things first, let’s make a new container and dive inside. To our empty container let’s start by adding an In TOP as well as a Movie In TOP. We’re going to start by connecting the Movie In to the second input on the In TOP. Why? The Movie In TOP is going to allow us to pass a video stream into this container. We may, however, want some image to show up while we’re working or when we don’t have a video stream coming in; this is where that second input comes in handy. This gives us a default image to use when we don’t have a stream coming into the container. Here’s what you should have so far:

in

Next, we want to down sample some of this image. This step is actually helping us plan ahead, we’re going to use some expressions to set the parameters of some of our other operators, and this step is going to help us with that. To do this down-sampling, we’re going to use the Resolution TOP. After that we’ll end our TOP string in a Null TOP. All of that should look like this:

Rutt-TOPs

Before we move on, let’s make on change to the resolution TOP in our string. Open up the parameters window for your resolution TOP and on the common page change the Output Resolution parameter to Quarter:

resolution

 

Channel Operators

There are a number of different transformations that we’ll need to do with the data from our image, we’ll start this process by first moving to Channel operators. If we think back to where we started this process with an intent to transform luminance into depth, then we know that one of the operations we need to complete is to add together the RGBA values of our pixels in order to have a single number we can use to manipulate a surface. To get us started we first need to convert our image into channel data that we can manipulate. To do this we’re going to add a TOP to CHOP to our network. This operator is going to allow us to look at our TOP as if it were all Channel data. We’ll see what that looks like in a moment. First, however, let’s make a few changes to our operator. In the Image page for the TOP to make sure that you’re set to “Next Frame” as the download type. On the Crop page you’ll also want to make sure that you’re set to the full image:

Next we need to assign a TOP so we have a texture that we’re converting into Channel data. You can do this by dragging the TOP onto this CHOP, or you can enter in the name of the target TOP on the image page. Your TOP to CHOP, should now look something like this:

top to chop

This viewer on this operator can be taxing on your system, so at this point I’d recommend that you click the small bulls-eye in the upper left hand corner of your CHOP to turn off the viewer. This will save you some system resources as you’re working in your network and keep you from seeing too much lag while you’re editing this network.

The next series of channel operators is going to allow us to make some changes to our data. First we’ll use a Shuffle CHOP to reorganize our samples into sets of channels, then we’ll use some Math CHOPs to add together our pixel values and to give us some control over the z-displacement, and finally we’ll use a Rename CHOP to make it easy to use this data.

Let’s start this process by connecting our Top to CHOP to a Shuffle CHOP and setting our method to be Sequence Channels by Name:

shuffle

Next we’ll add our Math CHOP and set the combine channel operation to Add, let’s also set the multiply value to 1 /3:

math

Next we’re going to add a slider, and another Math CHOP. Why? Great question. For starters, I want to be able to control the strength of this effect with a slider. At some point I’m going to want to be able to drive the strength of this effect, and a slider is a great way for us to do that. Why another Math CHOP? Another excellent question. While we could just use one Math CHOP and apply our slider there, that also means that there’s no way to isolate the effect of the slider. There’s some redundancy here, but there’s also a little more flexibility in the isolation of applying different alterations to our data set. Is this the best way to code a final component, maybe not, but it is a fine way to work with something that’s still being developed. Alright, let’s add our slider and second Math CHOP:

slider math

Next we need to write a simple expression in our math2 operator in order to be able to use the slider as an input method. On the Multi-Add page we’re going to use the out value from the slider as the value that the incoming channel information is multiplied by – if we think about the structure of our slider’s output we’ll remember that it’s a normalized value ranging from 0 – 1, and we can think of this as being the same as 0 – 100%. Alright, here’s our simple reference expression:

op( ‘slider1/out1’ ) [ ‘v1’ ]

In plain English this expression reads: give me the number value of the channel called ‘v1’ from operator called ‘out1’ that’s inside of ‘slider1’. If you click on the + sign on the bottom right of your slider (making it viewer active) you can now move it left and right to see the change in the number value of your math2 CHOP.

simple referecne expression

Before we move away from this portion of our network we need to add one final operator, a Rename CHOP. The Rename CHOP allows us to rename the channels inside of an operator. Later we’ll want to be able to use this number to replace an value from a surface operator chain – in order to do that easily we need to rename this value. In the to field of the rename CHOP type tz:

rename

 

Surface Operators

Now that we have started the process of converting our Texture into a channel data, we need to think about what we’re going to do with that information. We’re going to start by adding a Grid SOP to our network. We’d like this operator to pull some of its dimensions from the our video – this will make sure that we’re dealing with a surface that’s the correct aspect ratio. In our Grid SOP we’re going to use some simple expressions to pull some information from our Null1 TOP (remember that our Null1 is at the end of our TOP chain). First we’ll use the height and width of our Texture to set the number of rows and columns – we can use the dot call method to ask for the height and width of our TOP with the following expressions for rows and cols:

op( ‘null1’ ).width
op( ‘null1’ ).height

Next we can use internally call the number of rows and columns to for our height and width with the expressions:

me.par.cols
me.par.rows

We’ll also want to make sure that we’ve set our grid to be a Polygon with Rows as the connectivity type.

Now we’ll need to get ready for our next step by adding a CHOP to SOP. This operator is going to allow us to create some geometry out our CHOP data.

chopto

What gives? Well, in order for this operator to work properly, we need to have some feed it some CHOP data.

 

Channel Operators Again

Now we’re finally making progress, even if it doesn’t feel like it just yet. For starters, we have our Texture Operators converted into channel data, we have a piece of geometry that we can alter based on the dimensions of the input Texture, and we’re ready to combine our Channel data with our Surface data, we just have some final house keeping to do.

Let’s start by first converting our Grid SOP to a CHOP with a SOP to CHOP.

sopto

Like we did before we’re going to use a Shuffle CHOP, but this time we’re going to set it to Sequence all Channels:

shuffle2

Next we’ll use a Math CHOP to find the absolute value of our shuffle – we can do this by setting the Channel Pre Op to be Positive:

absolute value

Next we’ll use an Analyze CHOP to find the maximum value coming out of our Math CHOP.

analyze

Now we’re going to normalize our data against itself. We’re going to add another Math CHOP to our network. We’ll connect it’s input to our sopto1 CHOP and in this case we’ll use the Multi-Add page and use 1 / the maxim from our analyze CHOP. We can do this with a simple reference expression:

1 / op( ‘analyze1’ ) [ ‘tx’ ]

All of that should look something like this:

MORE math

We can finally start putting all of this together with one more Math CHOP. This final Math CHOP is going to combine our SOP to string, and our TOP to string. You’ll want to make sure that the SOP string is in the first place on the Math CHOP, with the TOP string coming in underneath. Next make sure that Combine Channels is set to Add, and that Match By is set to Channel Name.

SOP TOP

Now let’s end this set of operations in a Null, and move back to where we left off with our surface operators. Back in our Chop to SOP, let’s set our CHOP reference to be our last CHOP null (in my case it’s Null2). All of that should finally look like this:

chop to 1

At long last we’ve finally transformed a grid in the z direction with the information from an input Texture. Ba da bing bang boom:

grid transformed

 

 

Rendering

Dear flying spaghetti monster help us… this is has been a lot of work, but so far it’s not very fancy. What gives? Well, if we really want to make something interesting out of this, we need to render it – that is, we need to change it from being just some geometric information back into some pixels. If we think back to what we learned while we were playing with Instancing, rendering isn’t too hard, we just need a few things to make it work (some geometry, a camera, and a light source… this time we’ll skip that last one).

Let’s start by adding a camera, and a geometry component to our network.

geo camera

For our render to work properly, we need our chopto SOP to be inside of our Geo. At this point you can use your favorite piece of profanity and get get ready to remake this network OR you can HOLD YOUR HORSES and think about how we might solve this problem. My favorite way to address this kind of issue is to jump inside of the geo component, add an In SOP and set it to render and display. This means that we can pass our geometry into this component without needing to encapsulate all of the geometry inside.

insop

Now let’s connect our chopto to our inlet on the Geo comp:

geo connect

Ideally we want the image from our original texture to be used when rendering our z transformation. To do this, let’s jump back inside of our Geo COMP and add a constant Material. A Constant, unlike a phong, isn’t shaded. In other words, it doesn’t render with shadows. While this isn’t great in some respects, the pay off is that it’s much cheaper to render. While we’re getting started, this is a fine material to start with. We’ll also want to make sure that our Constant is using our original TOP as a color Map. In the Color Map field we can tell this operator to look for the operator called in1 in the directory above with the following call:

../in1

constant mat

Next we need to apply this material to our Geo. To do that let’s jump out of the Geo comp and head to the render page of the parameters window. We can tell our Geo to look for the material called constant1 that’s inside of the geo comp like this:

./constant1

geoconstant

Holy macaroni, we’re almost there. The last step we need to take is to add a Render TOP to our network:

renderTOP

At long last we have finally replicated the z-translation aesthetic that we set out to emulate. From here you might consider changing the orientation of the geo comp, or the camera comp for a more interesting angle on your work.

 

Play Time

That’s all well and good, but how can we turn it up a little? Well, now that we have a portion of this built we can start to think about what the video stream going into this operation looks like, as well as how we modify the video coming out of it. Let’s look at  one example of what we can do with the video coming out.

I’ve made a few changes to our stream above (some simple moving animation, and changed the resolution op), to have something a little more interesting to play with but it’s still not quite right. One thing I want to look at is giving the lines a little more of a neon kind of look and feel. To do this I’m going to start by adding a Blur TOP to my network:

blur

Next I’m going to add an Add TOP and plug both of my TOPs into it, adding the blur back to the original render:

addblur

Finally, I’m going to add a Constant TOP set to black, and a Composite TOP. I’ll composite my Add TOP and my Constant TOP together to end with a final composition:

composite

Now it’s your turn to play. What happens when you play with the signal processing after your render, what happens when you alter the video stream heading into this component. Also, don’t forget that we built a slider that controls the strength of the effect – play and make something fun.

rutrut

 

Looking to take a closer look at what makes this process work? Download the tox file and see what makes this thing tick: rut

Instant Instancing | TouchDesigner

Instant Instance“What if I want ten-thousand. Literally. Ten Thousand.” Back when Wonder Dome was in full swing I had the great honor of meeting some of the folks from Concordia’s Topological Media Lab. While some of their folks were in residence with ASU’s school of Arts Engineering and Media. They’ve spent a significant chunk of time building a system that they call Ozone, which is largely powered by Max MSP and Jitter. While they were visiting with us at Wonder Dome, they had lots of questions about TouchDesigner, how it works and how it was working for us. We talked bout a number of different things that day, but my favorite moment came over lunch. We were talking about making surfaces, and the flexibility and challenges involved. After we had added a sphere to our network, one of the team members asked me “what if I want to make a lot of them. What if I want to make ten-thousand. Literally. Ten Thousand.” There are lots of ways you might do this, but one of my favorite ways to think about making multiple copies of the same object is with instancing.

Say what now?

βατόμουρο / Raspberries by Fir0002/Flagstaffotos

βατόμουρο / Raspberries by Fir0002/Flagstaffotos image by flickr user Duncan Hall

Instancing is a fun way to make lots of copies of a piece of geometry and to place those copies very specifically. On the face of it, that sounds a bit academic. Let’s look at a specific example to unwrap that idea a little bit. Let’s look at a raspberry. We can think of a raspberry in a few different ways. On the one hand it’s a small oval like berry. At the same time if we look closer we can think of this single object as an array of spheres tucked next to one another in a patterned way. Instancing, as a method, allows us to do something like this relatively easily and quickly. On top of that, it’s a ton of fun, and unlocks a whole new set of possibilities when it comes to creating geometry in TouchDesigner.

So where do we start?! Well, we need to start by taking some time to do a little bit of thinking about how instancing works. For starters, instancing is something that you do to a Geometry COMP. As a component, we can do things with Geo’s that we can’t do with regular surface operators. A Geometry COMP is also one of the main ingredients when it comes to rendering surfaces. First we’ll look at how we can work with the Geometry COMP, then we’ll talk about what that means when it comes to rendering.

Okay. Let’s start in an empty network and create a new Geometry COMP.

geo

We could just work with the torus that’s in our Geo to get started, but let’s make a few changes so we have something that’s a little more fun to play with. Dive into your geo, and let’s get situated.

inside the geo

Inside our geo you should only see a single surface operator, a Torus SOP. You should also notice that there’s a little purple dot, and a little blue dot that are in the bottom right hand corner. These are two of the Operator’s flags – the purple dot is the render flag, and the blue dot is the display flag. Flags let you change the behavior of your operators in some interesting ways. The render flag, like it’s name, tells you if something is being rendered or not. Why not render something?! Well, you might find yourself in a situation where you need a SOP to help control the movement of another piece of geometry, but you don’t necessarily want that guiding element to be rendered. In this case you’d want to make sure the render flag is off. The display flag allows a SOP to be seen in the Geometry viewer. Like our other example, there might be a time when you don’t want to display a particular SOP while you’re making a change, in this case you’d want to make sure the display flag is off. There’s lots more to say about Flags, but that should be enough to make you dangerous.

Alright, now that we know a little more about what’s happening in our Geo let’s make some changes. For starters, let’s delete the Torus, and add a Sphere SOP and a Null SOP. Let’s connect these two operators, and make sure that we turn on the display and render flags on the bottom right corner of our Null. While we’re here let’s change the name of our null to DISP for Display.

sphere

Let’s also change the size of our sphere. In the radius parameter field lets change the three values to 0.15.

smaller sphere

Now when we back out of our Geo we should see our sphere instead of a torus.

sphere geo

Alright. Now we’re going to add another sphere to our network, but this time at the same level as our Geo. Let’s also connect that Sphere to a null.

sphere null geo

The next part is gonna start to get funky, so let’s help ourselves out before the world starts to turn upside down on us. Head over to your sphere, and make it viewer active (hit the small + in the bottom right corner), next hit the letter w on your keyboard. What you should now see is the wire frame of your Sphere SOP.

wire sphere

Okay. Let’s just leave that be for now, but know that we’re going to come back to it so we can better understand what’s coming next. Now we’re going to add a SOP to CHOP to our network.

sop to

The SOP to CHOP is a special kind of operator that allows us to change one type of operator to another. In our case, we’re going to change our null1 into CHOP data. To do this, we’re going to grab our SOP null1 and drag it on top of our CHOP sopto1.

soptochop

What manner of witch-craft is this?! Alright, what we’re looking at is the channel information from our SOP. Let’s think back to our wire frame Sphere for a moment. Each one of those intersecting lines represents a point. Each point has three values associated with it in terms of position – x, y, and z. Our SOP to CHOP is giving us a graph that represents all of those points. That’s awesome.

Why is that so awesome? Because now we can instance like there’s no tomorrow. Let’s head over to our Geo COMP and take a look at a few things. If we look at the parameters of this COMP, we’ll notice a page called Instance. Click that bad boy. Here we want to do a few things. First we want to turn on Instancing. Next we’re going to tell our Geo that we’d like to use our CHOP called sopto1. The last thing we need to do here is to help our Geo understand what to do with our CHOP data. In our case we’re going to tell the Geo to use tx for tx, ty for ty, and tz for tz – we’re matching up the location of our points from our Sphere to the points where we’d like instances of our geo. If that doesn’t make sense, that’s okay – for now follow along and this should make more sense soon.

instance

Now if we look at our network we can see some of the magic that’s happening with instancing.

networkinstance

To get a sense of how powerful this really is, let’s go back to our Sphere here in our parent network. In the parameters page, let’s change this sphere from being drawn as a mesh to being drawn as polygons.

polygons

We can now see that our instances are happening at each of the vertices of our polygon. Let’s try one more change to see how this powerful this is. For starters, let’s turn up the frequency count – I’m going to turn mine up to 10. Next let’s add a Noise SOP to our chain of operators between the Sphere and the Null.

noise instance

Now we’re starting to see some magic happen. At this point we’re now transforming the position of our instanced spheres with a noise algorithm that’s being applied to our original geometry.

An Aside About Efficiency 
As a quick note, it’s important to consider that this is not the most efficient way we could have programmed this. In TouchDesigner SOPs are a CPU operation, unless they’re encapsulated inside of a Geo COMP. What does this mean? Well, it means that the noise SOP in our network is a CPU operation, and not a GPU operation and depending on your machine for this example your mileage may vary.

Alright, so now we’ve made something awesome… but how do I use it?! Well, in order to actually use something like this in a project we need to render this geometry. Rendering requires a few different things to be in place, we need something to render (a Geometry COMP), a perspective to render it from (a Camera COMP), and a light (a Light COMP, we don’t always need a light but for now lets add one anyways). Let’s start by adding those COMPs to the network.

render network

Alright, now we can add our final ingredient to our network, a Render TOP. Our render TOP is going to use the three COMPs in our network to actually draw the pixel values of our scene. At this point your network should look something like this:

whole network

Nice work. You’ve not instanced your first set of geometry and rendered it. Now for extra credit, go back and consider changing some of your source SOPs… what happens if you change what’s in your Geo into a box or torus? What happens if you change your Sphere in the parent network into a grid instead? Happy instancing.

noise cubes

2D Sliders | TouchDesigner

slider arrayThe other day I took a moment to write down how to change a horizontal slider into a vertical one in TouchDesigner. This got me thinking, and I realized that in addition to a good ol vertical slider, a 2D slider (or an XY slider) would be another handy tool to have your disposal. Why make these yourself when there are a ton of tools already made in the TUIK tool set? Good question – there are a ton of wonderful pre-made control tools at our disposal there, but if we want to take some time to better understand how those tools work and and the programming logic behind them then it’s worth taking some time to make our own versions. It’s part mental exercise, part challenge, all fun. Maybe.

For starters, why use a 2D slider anyway? There are lots of things you can do with these kinds of interface objects. These are especially handy for corner pinning objects – this control surface reports out an x and y value, which you can then scale and apply to the corner of an image. With four of these 2D sliders you now can control the four vertices of an image. More generally, these kinds of controls are useful when you want two values to come out of a single control surface. That’s all well and good, but lets get to making one, shall we?

We’re going to start this process with a regular horizontal slider. Let’s begin by adding a Slider COMP to your network.

slider

Eventually, we’re going to make some changes to the parent object for our slider, but for now we’re going to dive inside and re-arrange the guts of our component just a little. When you get inside of the Slider you should see a familiar set-up: a Panel CHOP, an Out CHOP, and a Container called Knob.

slider insides

In another post I talked about how we can decode what’ s happening in the expression in the x parameter field of our Knob. In summary we can decode the expression this way. The expression:

me.parent().panel.u.val*me.parent().par.panelw-me.par.panelw/2

In English this might read – my parent’s panel u value (.5) * my parents width (100) – my panel’s width / 2. Great. Why? Well by multiplying the u value (a normalized value) with the width of the panel I get my position in terms of pixels – almost. The last thing I need to do is to take into consideration the width of the knob (divided by 2 so you split it’s value equally left and right).

This is all excellent news, but how does it help us with a 2D slider? First we need to think about what’s happening in a 1D slider vs. a 2D slider. In our regular horizontal (or vertical) slider we’re just tracking changes in a single direction – left and right, or up and down. In a 2D slider, we want to track both of those changes – up and down and left and right at the same time. To do this, we’re going to take our existing expression, and the expression that we used in our vertical slider and use both of them.

We already have an expression that allows us to track horizontal change, so how do we change that to track vertical change? In the expression above we’re going to make the following changes – first we want to use the value “v” instead of “u” that’s coming out of our panel (If you just had a moment when you said – “u, v, what now?!” take a break and read the Wikipedia page about uv mapping). We’ll also need to look at panelh (panel height) instead of panelw (panel width). This means our new expression for watching the change in a vertical dimension is:

me.parent().panel.v.val*me.parent().par.panelh-me.par.panelh/2

So far so good? Okay, next we’re going to use this expression in the Y parameter field of the knob component. This means that we have the two following expressions that are in the x and y parameters of our knob component:

x = me.parent().panel.u.val*me.parent().par.panelw-me.par.panelw/2
y = me.parent().panel.v.val*me.parent().par.panelh-me.par.panelh/2

knob expressions

Okay, now we’re going to make one more change to our knob’s parameters while we’re here. Instead of this control object being a rectangle, let’s make it a square. We can notice that its current width and height are 5 and 20 respectively. Let’s change that to 10 and 10. You certainly don’t have to work with a square knob, but I like the way this looks for this type of control. We also need to make some changes to our Panel CHOP.

The panel chop gives us all sorts of useful information about what’s happening in a panel at any given moment. In our case we can use this CHOP to tell us the horizontal and vertical position of the knob that for our slider. If we look at how our panel is currently set up we can see that it’s selecting the u value and renaming it “v1.” We want our slider to send out both u and v information, so let’s change the select to read “u v” – separating our variables with spaces tells TouchDesigner that we’d like both to be selected. We can also rename this values while we’re here. I choose to rename them xPos and yPos (x Position, Y Position). Choose whatever name makes sense for you, and that it’s too long – if you end up needing to call these values in another expression having a shorter name is helpful.

panel change

Now let’s take a moment and head back up to our parent container. Here in our parent let’s change the width and height to be 100 and 100. We should now see a square panel with a square inside of it, that we can drag about.

slider no lines

If we connect a Null CHOP or a Trail CHOP we can see the values reported out of our new tool. Sweet.

null and trail gif

Alright, this is pretty cool, and I’m mostly happy with it… but it’s missing something. One of the things you’ll see with other 2D sliders are guild lines that pass through the middle of the central knob – these help you visually maintain a sense of where your control object is pointed, and let’s face it they just look cooler.

Okay, so let’s add some guild lines, where do we start?

We can begin by diving back into our Slider component. Let’s work the easy way, and take our knob component, and make two copies of it. We can call these two new additions xWire and yWire. Instead of hard coding in the dimensions of these wires, we are instead going to use some expressions to define what they look like. Why? Well, while we certainly could do this by hard coding the numbers it also means that if we make any changes to the parent component, it also means we need to make changes several pieces inside of the component. This works just fine if you’re only making something you want to use once, but if you want to make a component that you can use and reuse then using some expressions is going to save you a ton of time – and as a bonus you’ll learn more about using python expressions in TouchDesigner. Enough of my soap box, let’s do this.

Le’ts start with our xWire component. I’d like my guild lines to be one pixel tall (or wide for the yWire), and the same width as the parent component. To make this happen let’s use the following expressions in the Width field:

me.parent().par.panelw

In plain English this reads – what is my parent component’s panel width parameter. By using this expression we know that the width of this object will always match the width of our parent. Perfect. Before you celebrate too soon we need to add one more expression. We’d like the line that we’re drawing to stay aligned with our central knob (no really, we do want that). For this we need to keep in mind that our xWire that stretches the width of the parent component needs to move vertically – this may seem counter intuitive, but it’ll make perfect sense here in one moment. How do we do that? Well, luckily we were just practicing ways to make a knob stay aligned to a position when we made a vertical slider. We’re going to use some of the same ideas from that expression here. In the Y paramter field we’re going to use this expression:

op(‘panel1’) [ ‘yPos’ ] * me.parent().par.panelh

Say what now?! In plain English we’re saying – look at the operator called “panel1” and find the value called “xPos”, multiply that by the panel height of my parent. Now we should have a working xWire component. Before jump up to look at our 2D slider, here’s what you should have so far:

xWire

If jump out of our 2D slider and take a look at what we have so far, we should see something like this:

xWire

 

This is almost what we want, right? Now we just need to repeat this process to add our yWire guideline. Let’s dive back inside of our slider to finish up. For our yWire we’re going to set the width of our component to 1 and use an expression to change the height:

me.parent().par.panelh

We’re also going to use an expression to change the X position of our component:

op(‘panel1’) [ ‘xPos’ ] * me.parent().par.panelw

These are like our other expressions just calling different values. At this point you should have something that looks like this:

yWire

Now we’re finally ready to back out of our 2D slider, and admire our hard work.

xySider

Nice.

Alright, the last thing to do after you’ve done all of this hard work is to save this component as a .tox file that you can reuse. If you’ve never done that before, you can read how that works here.

Shuffling Words Around | Isadora

About a month ago I was playing about in Isadora and discovered the Text/ure actor. Unlike some of the other text display actors, this one hides a secret. This actor lets you copy and paste into a whole block of text that you can then display one line at a time. Why do that? Well, that’s a fine question, and at the time I didn’t have a good reason to use this technique, but it seemed interesting and I tucked it into the back of my mind. Fast forward a few months, and today on the Facebook group – Isadora User Group (London) – I see the following call for help:

Isadora_User_Group__London_

And that’s when I remembered the secret power of our friend the Text/ure actor. Taking Raphael’s case as an example let’s look at how we might solve this problem in Izzy.

First off we need to start by formatting our list of words. For the sake of simplicity I’m going to use a list of 10 words instead of 100 – the method is the same for any size list, but 10 will be easier for us to work with in this example. Start off by firing up your favorite plain text editing program. I’m going to use TextWrangler as my tool of choice on a Mac, if you’re on a PC I’d suggest looking into Notepad++.

In TextWrangler I’m going to make a quick list of words, making sure that there is a carriage return between each one – in other words I want each word to live on its own line. So far my sample text looks like this:

untitled_text

Boring, I know, but we’re still just getting started.
Next I’m going to open up Isadora and create a new patch. To my programming space I’m going to add the Text/ure actor:

Untitled

So far this looks like any other actor with inlets on the left, and outputs on the right. If we look closely, however, we’ll see a parameter called “line” that should catch our attention. Now for the magic. If we double click on the actor in the blue space to the right of our inlets, we suddenly get a pop up window where we can edit text.

 

Edit_Text_and_Untitled

Next let’s copy and past our words into this pop up window. Once all of your text has been added, click “OK.”

Edit_Text_and_Untitled 2

Great. Now we have our text loaded into the Text/ure actor, but we can’t see anything yet. Before we move on, let’s connect this actor to a projector and turn on a preview so we can get a sense of what’s happening. To do this start by adding a Projector actor, then connecting the video outlet of the Text/ure actor to the video inlet of the Projector.

Untitled 2

Next show stages – you can do this from the menu bar, or you can use the keyboard shortcut Command G. If you’re already connected to another display then your image should show up on your other display device. If you’d like to only see a preview window you can force preview with the keyboard shortcut Command-Shift F.

Untitled___Stage_1_and_Untitled

Alright, now we’re getting somewhere. If we want to change what text is showing up we change the line number on the Text/ure actor.

changing-text

Alright. So now to the question of shuffling through these different words. In Raphael’s original post, he was looking to not only be able to select different words, but also to have a shuffling method (and I’m going to assume that he doesn’t want to repeat). To crack this nut we’re going to use the shuffle actor, and some logic.

Let’s start by adding a shuffle actor to our patch, and let’s take a moment to look at how it works.

Untitled 3

Our Shuffling actor has a few parameters that are going to be especially important for us – min, max, shuffle, and next. Min, like the label says is the lowest value in the shuffle stack; Max is the highest value. Shuffle will reset the our counter, and reshuffle our stack. The next trigger will give us the next number in the stack. On the outlet side of our actor we see Remaining and Value. Value, is the shuffled number that we’re working with; Remaining is how many numbers are left. If we think of this as a deck of cards then we can start to imagine what’s happening here. On the left, shuffle is like actually shuffling the deck. Next is the same as dealing the next card. On the right the Value would be the face value of the card dealt, while remaining is how many cards are left in the deck.

Alright already, why is this important?! Well, it’s important because once we get to the end of our shuffled stack we can’t deal any more cards until we re-shuffle the deck. We can avoid this problem by adding a comparator actor to our patch. The comparator is a logical operation that compares two values, and then tells you when the result is a match (True) and when it doesn’t (False).

Untitled 4

To get our logic working the way we want let’s start by connecting the Shuffle’s Remaining value to the value2 of the Comparator. Next we’ll connect the true trigger from the Comparator back to the Shuffle inlet on the Shuffle actor.

Untitled 5

Great, now we’ve made a small feedback loop that automatically reshuffles our deck when we have used all of the values in our range. Now we can connect the Value outlet of the Shuffle Actor to the Line input of the Text/ure actor:

Untitled 6There we have it. Now the logic of our shuffle and comparator will allow us to keep running through our list of words that are in turn sent to our projector.

shuffle-text

Vertical Sliders Please | TouchDesigner

vertical sliderIf you’ve done any interface building in TouchDesigner, you’ve surely noticed that there is a button and a slider Comp at your disposal. These are outstanding building blocks to use when you’re creating interfaces, and give you a ton of flexibility and dependability. In the past I’ve taken some time to talk about making some changes to the Button Comp, but what about the slider Comp? Won’t someone think of the slider comps?! Perhaps most importantly, what about making a vertical slider – the standard slider comp is horizontal, and wouldn’t it be nice to have one that worked up and down instead of left to right? Well, today is your lucky day – let’s look at how to make a few changes to our stock horizontal slider and turn it into a vertical slider instead.

Let’s start by first looking at what’s inside of our slider, and what makes it tick so to speak.

slider insides

Unlike the button comp, the slider looks much simpler – after all it’s only made up of a container (the knob) a Panel CHOP and an Out CHOP. That seems pretty bare bones, but if we take a closer look at the knob we can see some interesting things at work here. Of the most interest to us is the expression in the x parameter of the knob. Here you should see an expression that reads:

me.parent().panel.u.val*me.parent().par.panelw-me.par.panelw/2

What on earth does that mean?! Well let’s take a closer look at this by first experimenting with a Constant CHOP that will let us see these numbers individually. In our Constant I’m going to break this longer expression into its component parts so we can see what’s happening.

me.parent().panel.u.val

me.parent().par.panelw

me.par.panelw

At this point you should be seeing an error message. What gives?

error

The answer to that questions lies in the last expression that we’re using – me.par.panelw. A constant, doesn’t have a panel that’s associated with it, so by asking for this value we’re getting an error message telling us that we are in fact asking for the impossible. If we want the panel width of our knob, we need to change this expression slightly so instead it reads: op(‘knob’).par.panelw. Once you make that change, you should now see something like this in your Constant.

fixed

 

Okay, so what are all of these values.

Our first expression (me.parent().panel.u.val) is looking for the u value from the parent’s control panel. We can confirm this value by looking at the Panel CHOP in this slider. At the middle position of the container, the panel u value is .5 – all the way to the right is 1, all the way to the left is 0. So far so good.

samesies

Our second expression (me.parent().par.panelw) is looking for the width parameter of the parent operator. If we look at our parent’s panel values we can see that that parameter panelw is in fact 100. That’s two values down, one to go.

sliderparent

Our third expression (me.par.panelw) is asking for the width parameter of the current operator. We can verify this by looking at our width parameter of the knob, and can see that they match.

panelw

Okay. Now let’s look at what is happening in the full expression:

me.parent().panel.u.val*me.parent().par.panelw-me.par.panelw/2

In English this might read – my parent’s panel u value (.5) * my parents width (100) – my panel’s width / 2. Great. Why? Well by multiplying the u value (a normalized value) with the width of the panel I get my position in terms of pixels – almost. The last thing I need to do is to take into consideration the width of the knob (divided by 2 so you split it’s value equally left and right). So what can we do with this new information?

Let’s make a vertical slider.

First lets start by taking our Slider COMP and swap the width and height values:

setp1vertslider

Next, let’s dive into our slider, and make a few changes. Inside of the slider, on the knob copy the expression from the x parameter and paste it into the y parameter. Let’s also swap the width and height values:

xtoy

Hang on to your boots, because now we need to pay close attention to some details. We did some swapping around with our expressions, so before this is working correctly we need to make sure we change the expressions so they’ll work they way we expect them to. First up we need to change our u value to v, and then we need to look at panelh instead of panelw – remember that we’ve swapped our directions, instead of moving left and right we’re moving up and down and our expression needs to reflect that change. Here’s what your expression should look like:

me.parent().panel.v.val*me.parent().par.panelh-me.par.panelh/2

newknob

Finally, we need to change the Panel CHOP to output the v value instead of the u value:

utov

Congratulations, you should now have a vertical slider that works just like the horizontal ones.

vsliders

 

OSC Remote Control | TouchDesigner

13771584713_fb81838217_mThere’s a lot to love about the internet, really. But I think one of my favorite things is how it connects people, how it flattens old hierarchies (not really, but let me wax idealistic for the sake of this intro) and connects people. In starting to program with TouchDesigner, I did the thing that any smart n00b would do – I joined the forum. The TouchDesigner forum is a great place to ask questions, find answers, learn from some of the best, and to offer help. We’ve all been stuck on a problem, and a commons like this one is a great place to ask questions, and keep tabs on what others are doing. To that end I shared a technique for sending and receiving OSC data with TouchDesigner back in October of 2013. I also shared this on the forum, because this happened to be something that I figured that others might want to know more about. My post was a simple example, but often it’s the simple examples that help move towards complex projects. As it turns out, someone else was fighting the same battle, and had some questions about how to make some headway – specifically they wanted to look at how to create an interface that could be controlled remotely with TouchOSC or from the TouchDesigner control panel itself. Ideally, each interface’s changes would be reflected in the other – changes on a smartphone would show up in the TouchDesigner control panel, and vice versa. I caught the first part of the exchange, and then I got swallowed by the theatre. First there was The Fall of the Hose of Escher, then Before You Ruin It took over my life, and then I spent almost a month solid in Wonder Dome. Long story short, I missed responding to a question, and finally made up for my bad Karma by responding, even if belatedly. It then occurred to me that I might as well write down the process of solving this problem. If you want to see the whole exchange you can read the thread here.

Enough jibber-jabber, let’s start programming.

For the sake of our sanity, I’m going to focus on just working with two sliders and two toggle buttons. The concepts we’ll cover here can then be applied to any other kind of TouchOSC widget, but let’s start small first. One more disclaimer, this one creates a messy network. If we took a little more time we could clean it up, but for now we’re going to let it be a little bit sloppy  – as much as it pains me to do so.

First things first, if you’re new to using TouchOSC you should take some time to get used to the basics and start by reading through this overview on using TouchOSC. Open Sound Control (OSC) messages are messages that are sent over a network. OSC can be used locally on a computer to allow for communication between programs, and it can also be used between multiple machines in order to allow them to communicate with one another. Interface tools like TouchOSC allow users to create custom interfaces that control some aspect of a program. That’s a very simplistic way of looking at OSC, but it’s a good start. The important takeaway here is that when we use OSC we’re actually relying on some network protocols to facilitate the communication between computers.

To get started I selected a simple interface on an iPod Touch that came with TouchOSC. Again, I wanted something with two toggles and two sliders to serve as our example. We can set-up our mobile device by starting TouchOSC and hitting the circular option / configuration button in the corner.

Once we do that we’ll see a screen with a few different options. We want to select the OSC connection tab.

oBGZO3662ohWeEYLEld_chDzyCFrrH1KFjmrRzwl0gA

From here we need to configure a few settings. First off you’ll want to set add the IP address of your computer into the “Host” field. To find your computer’s IP address you can use the ipconfig command to quickly find your IP address (if this sounds like another language, check out this youTube video to see what I’m talking about). I also want to take note of the IP address of the device – this is the “Local IP address.” I also want to take a close look at the outgoing and incoming ports, I’ll need to use these numbers in TouchDesigner to make sure that I can talk and listen to this device.

eAPW1uy3r24Z1X5Xba6Y6CRsU8g623zOkAU_BJIKXtY

Alright, now that we know a few things about our mobile device now we can head back to TouchDesigner.

First off, let’s take a look at how to listen to the incoming data. In a new network at an OSC In CHOP.

OSC_In

In the parameters box let’s make sure that we’re listening to the same port that we’re broadcasting on – in my case it’s port 10000. Now you should be able to start hitting buttons and moving sliders to see some new channels appear in your CHOP. Here’s it’s important to take a closer look at the naming convention for our channels. Let’s look at 1/fader1 first. TouchOSC has a great utility for creating your layouts, and if you let it do the naming for you this is the kind of format that you’ll see. The semantics of this are page/widgetNameWidgetNumber. So by looking at 1/fader1 we can read this to mean that this is on page one, it’s a fader, and it’s number (the order it was created) is one. This naming convention is going to be important to take note of, and will save you a lot of headaches if you take some time to really wrap your head around how these widgets are named.

Before we start building an interface to control let’s take a moment to get a few more ducks in a row. I’m going to connect my OSC In CHOP to four different select CHOPS – each control is going to be routed to a button or slider, and I want to make sure that I’m only dealing with one channel to do this.

4selectsFor my own sanity I’ve named each of the selects with a name that corresponds to what channel they’re carrying. You can choose any naming convention that you’d like, but definitely choose a naming convention. This kind of patching can get messy quickly, and a solid method for naming your operators will server you well.

Before we move on let’s take a closer look at one of the selects to see how we can pull out a particular channel.

fader1

You’ll notice that in the Channel Names I’ve used the name *fader1 rather than 1/fader1. What gives?! Either of those names is going to give me the same result in this case. I’ve elected to use the asterisk modifier to save myself some time and because I’m only using page one of this particular interface. What does the asterisk modifier do? I’m so glad you asked – this particular modifier will give results that match anything after the asterisk. If, for example, I had 1/fader1, 2/fader1, and 3/toggle1 as incoming channels, the asterisk would pull out the 1/fader1 and 2/fader1 channels. Naming and patterning isn’t important for this particular example, but it’s a technique to keep your back pocket for a rainy day.

Okay, now that we’ve got our OSC In data ready, let’s build a quick interface. In this instance I’m going to use two vertical sliders from the TUIK presets in the pallet browser. You can find them here:

TUIK

I’m also going to add two Button COMPs  and a Container COMP to my network.

interfaceSetup

Before we move forward, I need to make a few quick changes to by buttons. You’ll notice that the sliders both have CHOP inlets on their left-hand side, but my buttons do not. This means that my sliders are all ready set-up to receive a channel stream to change them. My buttons aren’t ready yet, so let’s take a look at the changes we need to make. Let’s start by diving into our button.

buttonCOMP

If this is your first time taking a look inside of button comp take a moment to read through a quick overview about working with buttons in TouchDesigner. We’re going to want to add a CHOP In to this component, and we’re going to also want the changes from the Touch interface to drive how our button’s color changes. Here’s what that’s going to look like:

new buttonHere I’ve added an In CHOP that feeds to a Math CHOP that I’ve renamed to “i” that’s finally connected to our Out CHOP. So why a Math, and why rename it? If you look closely at the Text TOP  you’ll see that it’s driven by several expressions allowing it to change color based on its active state – this is part of what’s happening in the expression CHOP originally called “i”. Here our Math CHOP is set to add our in and the “ii” CHOP (formerly just “i”) together. By changing the name of the operators, I’ve avoided re-writing the scripts to save myself some time.

With our buttons and sliders finally ready we can start connecting our interface elements. There are lots of ways to build interface elements in TouchDesigner, but today I’m just going to use the ability to wire components vertically to do this. First I’m going to layout my buttons and sliders in an approximate location that matches my TouchOSC layout just to help me get organized initially. Next I’m going to connect them through their vertical inlets to my container COMP. In the end, if you’re following along, it should look something like this:

vierticalwires

Finally, you’ll want to take some time to adjust the placement of your buttons and sliders, as well as adjusting their color or other elements of their appearance. My final set up looks like this:

finalcontrol

If you’re still with me, this is where the real magic starts to happen. First up we’re going to connect the our corresponding OSC In values to our control panel elements that we want to control. Fader 1 to fader 1, toggle 1 to toggle 1 and so on. Next we’re going to connect all four of our control panel elements to a single Merge CHOP.

almsot there

Next we’ll need to do a little renaming. To make things easier I’m going to use a Rename CHOP and a Constant CHOP. Here our Constant CHOP holds all of the names that we want to apply to the channels that are coming into our Rename CHOP. Here’s where all of that funny business about naming our channels becomes important. To make sure that I’m feeding back data to TouchOSC in a way that properly associates changes to my sliders I need to follow the naming conventions exactly the way they’re coming into TouchDesigner. 1/fader1 that’s since become v1 needs to have it’s name changed back to 1/fader1. You can see what I mean by taking a closer look at the network below:

reanme

Last but not least we’re going to connect our rename CHOP to an OSC Out CHOP. When we set out our OSC out we need to know the IP address of the device that we want to communicate with, and we need to know the port that we’re broadcasting to. I also like to change my OSC Out to be samples, and to turn off “Send every cook.” Sending with every cook is going to create a lot more network traffic, and while it’s not an issue for TouchOSC, if you’re working with someone using MaxMSP having this trick up your sleeve is going to make them much happier. Here’s what our OSC Out operator should look like:

OSCOUT

Whew! Alright gang, at this point you should be ready to start making the magic happen. If you’ve got everything set-up correctly you should now be able to drive your control panel in TouchDesigner either from the panel we created, or from a TouchOSC setup. As a bonus (why we did all of this hard work) you should also be able to see your changes in either environment reflected in the other.

simplefeedback

Inside Wonder Dome | TouchDesigner

first test gridIn approaching some of the many challenges of Wonder Dome one of the most pressing and intimidating was how to approach programming media playback for a show with a constant media presence. One of the challenges we had embraced as a team for this project was using Derivative’s TouchDesigner as our primary programming environment for show-control. TouchDesigner, like most programming environments, has very few limitations in terms of what you can make and do, but it also requires that you know what it is that you want to make and to do. Another challenge was the fact that while our team was full of bright and talented designers, I was the person with the broadest TouchDesigner experience. One of the hard conversations that Dan and I had during a planning meeting centered around our choices of programming environments and approaches for Wonder Dome. I told Dan that I was concerned that I would end up building an interface / patch that no one else knew how to use, fix, or program. This is one of the central challenges of a media designer – how to do you make sure that you’re building something that can be used / operated by another person. I wish there were an easy answer to this question, but sadly this is one situation that doesn’t have simple answers. The solution we came to was for me to do the programming and development – start to finish. For a larger implementation I think we could have developed an approach that would have divided some of the workload, but for this project there just wasn’t enough time for me to both teach the other designers how to use / program in TouchDesigner and to do the programming needed to ensure that we could run the show. Dan pointed out in his thesis paper on this project that our timeline shook out to just 26 days from when we started building the content of the show until we opened.

The question that follows, then, is – how did we do it? How did we manage to pull of this herculean feat in less than a month, what did we learn along the way, and what was an approach that, at the end of the process, gave us results that we used?

Organization

organizationMake a plan and stay organized. I really can’t emphasize this enough. Wonder Dome’s process lived and died in our organization as a team, and as individuals. One of the many hurdles that I approached was what our cuing system needed to be, and how it was going to relate to the script. With three people working on media, our cue sheet was a bit of a disaster at times. This meant that in our first days working together we weren’t always on the same page in terms of what cue corresponded to what moment in the play. We also knew that we were going to run into times when we needed to cut cues, re-arrange them, or re order them. For a 90 minute show with 20 media cues this is a hassle, but not an impossibility. Our 25 minute long kids show had, at the beginning, over 90 media cues.

In beginning to think about how to face this task I needed an approach that could be flexible, and responsive – fast fast fast. The solution that I approached here was to think about using a replicator to build a large portion of the interface. Replicators can be a little intimidating to use, but they are easily one of the most powerful tools that you can use in TouchDesigner. Here the principle is that you set up a model operator that you’d like subsequent copies to look like / behave like. You then use a table to drive the copies that you make – one copy operator per row in the table. If you change the table, you’ve changed / remade your operators. In the same way if you change your template operator – this is called your “Master Operator” – then you change all of the operators at once. For those reasons alone it’s easy to see how truly powerful this component is, but it’s also means that a change in your table might render your control panel suddenly un-usable.

button replicator set-up

Getting started here I began by first formatting my cue sheet in a way that made the most sense for TouchDesigner. This is a great time to practice your Excel skills and to use whatever spreadsheet application / service that you prefer to do as much formatting as possible for you. In my case I used the following as my header rows:

  • Cue Number – what was the number / name for the cue. Specifically this is what the stage manager was calling for over headset. This is also the same name / number for the cue that was in the media designer script. When anyone on the team was talking about M35 I wanted to make sure that we were all talking about the same thing.
  • Button Type – Different cues sometimes need different kinds of buttons. Rather than going through each button and making changes during tech, I wanted to be able to update the master cue sheet for the replicator, and for the properties specified to show up in the button. Do I want a momentary button, a toggle, a toggle down, etc. These things mattered, and by putting these details in the master table It was one less adjustment that I needed to make by hand.
  • Puppet – Wonder Dome had several different types of cues. Two classifications came to make a huge difference for us during the tech process. Puppet entrances / exits, and puppet movements. Ultimately, we started to treat puppet entrances and exits as a different classification of cue (rather than letters and numbers we just called for “Leo On” and “Leo Off”, this simplified the process of using digital puppets in a huge way for us), but we still had puppet movements that were cued in TouchDesigner. During the tech process we quickly found out that being able to differentiate between what cues were puppet movements and what cues were not was very important to us. By adding this column I could make sure that these buttons were a different color – and therefore differentiated from other types of cues.

Here I also took a programming precaution. I knew that invariably I was going to want to make changes to the table, but might not want those changes to be implemented immediately – like in the middle of a run for example. To solve this problem I used a simple copy script to make sure that I could copy the changed table to an active table when we were in a position to make changes to the show. By the end of the process I was probably fast enough to make changes on the fly and for them to be correctly formatted, but at the beginning of the process I wasn’t sure this was going to be the case. The last thing I wanted to do was to break the show control system, and then need 25 minutes to trouble shoot the misplacement of a 1 or 0. At the end of the day, this just made me feel better, and even if we didn’t need it in place I felt better knowing that I wasn’t going to break anything if I was thinking on my feet.

replicator in action

Above you can see a replicator in action – looking at an example like this I think helps to communicate just how useful this approach was. Method, like organization, is just a way to ensure that you’re working in a way that’s meaningful and thoughtful. I’m sure there are other methods that would have given us the same results, or even better results, but this approach helped me find a way to think about being able to quickly implement cue sheet changes into our show control environment. It also mean that we standardized our control system. With all of the buttons based on the same Master Operator it gave the interface a clean and purposed look – staring down the barrel of a 25 show run, I wanted something that I didn’t might looking at.

Thinking more broadly when it comes to organization, beyond just the use of replicators for making buttons I also took the approach that the show should be modular and organized as possible. This meant using base and container components to hold various parts of the show. Communication to lighting and sound each had their own module, as did our puppets. For the sake of performance I also ended up placing each of the locations in their own base as well. This had the added bonus of allowing for some scripting to turn cooking on and off for environments that we were using or not using at any given point in the show. We had a beast of a media server, but system resources were still important to manage to ensure smooth performance.

notThatStory_fullMap

If you want to learn more about replicators you can read through this post about getting started using them.

Show Control

Show control, however, is about more than just programming buttons. Driving Wonder Dome meant that we needed a few additional features at our fingertips during the show. Our show control system had two preview screens – one for the whole composite, and one for puppets only. One of the interesting features of working in a dome is how limited your vision becomes. The immersive quality of the projection swallows observers, which is downright awesome. This also means that it’s difficult to see where all of the media is at any given point. This is one of the reasons that we needed a solid preview monitor – just to be able to see the whole composition in one place. We also needed to be able to see the puppets separately at times – partially to locate them in space, but also to be able to understand what they looked like before being deformed and mapped onto the curved surface of the dome.

show_control

The central panel of our control system had our cues, our puppet actions, our preview monitors, and a performance monitor. During the show there were a number of moments when we had a dome transformation that was happening while nearly simultaneously a puppet was entering or exiting. While originally I was trying to drive all of this with a single mouse, I quickly abandoned that idea. Instead I created a simple TouchOSC interface to use on an iPad with another hand. This allowed me to take a double handed approach to diving the media added some challenge, but paid itself back ten fold with a bit of practice. This additional control panel also allowed me to drive the glitch effects that were a part of the show. Finally it also made for an easy place to reset many of the parameters of various scenes. In the change over between shows many elements needed to be reset, and but assigning a button on my second interface for this task I was able to move through the restore process much faster.

2014-04-09 14.44.46

If you’d like to learn more about using TouchOSC with TouchDesigner there a few pages that you might take a glance at here:

TouchOSC | Serious Show Control
Sending and Receiving OSC Values
Visualizing OSC Data

Cues

Beyond creating a system for interacting with TouchDesigner, a big question for me was how to actually think about the process of triggering changes within my network. Like so many things, this seems self evident on the face of it – this button with do that thing. But when you start to address the question of “how” then the process becomes a little more complicated. Given the unstable nature of our cue sheet, I knew that I needed a name-based approach that I called from a central location. Similar to my module based approach for building the master cue sheet, I used the same idea when building a master reference sheet.

With a little push and guidance from the fabulous Mary Franck, I used an evaluate DAT to report out the state of all of the buttons from the control panel, and name them in a way that allowed for easy calling – specifically I made sure that each cue maintained it’s name letter and number convention from our cue sheet.

master ref

 

On the face of this it seems like that’s an awful lot of scripts to write – it is, but like all things there are easier and harder ways to solve any problem. My approach to here was to let google spreadsheets do some work for me. Since cue sheet was already set-up as a spread sheet, writing some simple formulas to do the formatting for me was a quick and easy way to tackle this. It also meant that with a little bit of planning my tables for TouchDesigner were formatted quickly and easily.

excel script formattingIt was also here that I settled on using a series of Execute DATs to drive the cooking states of the various modules to control our playback performance. I think these DATs were some of the hardest for me to wrap my head around – partially because this involved a lot of considered monitoring of our system’s overall performance, and the decisions and stacking necessary to ensure that we were seeing smooth video as frequently as possible. While this certainly felt like a headache, by the time the show was running we rarely dropped below 28 frames per second.

cooking on and off

If you want to read a little more about some of the DAT work that went into Wonder Dome you can start here:

Evaluate DAT Magic
These are the DATs You’ve Been Looking For

Communication

All of the designers on the Wonder Dome team had wrestled with the challenges of communication between departments when it comes to making magic happen in the theatre. To this end, Adam, Steve, and I set out from the beginning to make sure that we had a system for lights, media, and sound to all be able to talk with one another without any headache. What kind’s of data did we need to share? To create as seamless a world as possible we wanted any data that might be relevant for another department to be easily accessible. This looked like different things for each of us, but talking about it from the beginning ensured that we built networks and modules that could easily communicate.

Screenshot_032314_115125_AM

In talking with lighting, one of our thoughts was about passing information relative to the color of the environment that we found ourselves in at any given point. To achieve this I cropped the render to a representative area, then took the average of the pixel values in that area, then converted the texture data to channel data and streamed lighting the RGBA values over OSC. We also made a simple crossfader in our stream for the times when we wanted the lighting in the scene to be different from the average of the render.

WD_AdamThis technique was hardly revolutionary, but it did create very powerful transitions in the show and allowed media to drive lighting for the general washes that filled the space. This had the added benefit of offloading some programming responsibility from lighting. While I had done a lot of work in the past to coordinate with sound, I hadn’t done much work coordinating with lights. In fact, this particular solution was one that we came up with one afternoon while we were asking questions like “what if…” about various parts of the show. We knew this was possible, but we didn’t expect to solve this problem so quickly and for it to be so immediately powerful. Through the end of the run we continued to consistently get positive audience response with this technique. Part of the reason this solution was so important was be cause Adam was busy building a control system that ultimately allowed him to control two moving lights with two wacom tablets – keeping the washing lighting driven by media kept both of his hands free to operate the moving lights.

Screenshot_032314_115026_AM

The approach to working with sound was, of course, very different from working with lights. Knowing that we wanted to use spatialized sound for this show Stephen Christensen built an incredible Max patch that allowed him to place sound anywhere he wanted in the dome. Part of our conversation from the beginning was making sure that media could send location data bout puppets or assets – we wanted the voice of the puppeteers to always be able to follow the movement of the puppets across the dome. This meant that created an OSC stream for sound that carried the location of the puppets, as well as any other go or value changes for moments where sound and media needed to be paired together.

Screenshot_032314_114946_AM

Communicating with sound wasn’t just a one way street though. Every day the Wonder Dome had a 90 minute block of free time when festival visitors were allowed to explore the dome and interact with some of the technology outside of the framework of the show. One of the components that we built for this was a 3D environment that responded to sound, animating the color and distribution of objects based on the highs, mids, and lows from the music that was being played. Here sound did the high, mid, low processing on its end, and then passed me a stream of OSC messages. to get a smoother feel from the data I used a Lag CHOP before using this to drive any parameters in my network.

Components and Reuse

Perhaps the most important lesson to be learned from this project was the importance of developing solid reusable components. This, again, isn’t anything revolutionary but it is worth remembering whenever working on a new project. The components that you build to use and reuse can make or break your efficiency and the speed of your workflow. One example of this would be a tool that we created to make placing content on the dome. Our simple tool for moving images and video around the dome would be used time and again throughout the project, and if I hadn’t take the time early on to create something that I intended to reuse, I would have instead spent a lot of time re-inventing the wheel every time we needed to solve that problem.

Screenshot_032314_114332_AM

In addition to using this placement tool for various pieces of media in the show, this is also how we placed the puppets. During the development phase of this tool I thought we might want to be able to drive the placement of content from a iPad or another computer during tech. To make this easier, I made sure that there was a mechanism embedded in the tool to allow for easy control from multiple inputs. This meant that when we finally decided to adapt this tool for use with the puppets, we already had a method for changing their location during the show. There are, of course, limits to how much anyone can plan ahead on any project but I would argue that taking the time to really think about what a component needs to be do before developing it makes good sense. I also made use of local variables when working with components in order to make it easier to enable or disable various pieces of the tool.

Screenshot_032314_114451_AM

You can read more about some of this process here:

3D Solutions for a 2D World
Container Display

Documentation and Comments

comment exampleI nearly forgot to mention one of the most critical parts of this process. Documentation and commenting. If I hadn’t commented my networks I would have been lost time after time. One of the most important practices to develop and to continue is good commenting. Whenever I was working on something that I couldn’t understand immediately by just looking at it, I added a comment. I know that some programmers use the ability to insert comments with individual operators, but I haven’t had as much success with that method. Personally, I find that inserting a text DAT is the best way for me to comment. I typically write in a text editor using manual carriage returns. I also make sure that I date my comments, so if I make a change I can leave the initial comments and then append the comment with new information. I can’t say enough about the importance of commenting – especially if you’re working with another programmer. Several times during the process I would help lighting solve a problem, and good commenting helped ensure that I could communicate important details about what was happening in the network to the other programmer.

I think it’s also important to consider how you document your work. This blog often functions as my method of documentation. If I learning something that I want to hold onto, or something that I think will be useful to other programmers then I write it down. It doesn’t do me any good to solve the same problem over and over again – writing down your thoughts and process help you organize your approach. There have been several times when I find shortcuts or new efficiency in a process only when I’m writing about it – the act of taking it all a apart to see how the pieces connect make you question what you did the first time and if there’s a better way. At times it can certainly feel tedious, but I’ve also been served time and again by the ability to return to what I’ve written down.

 

 

TouchOSC | Serious Show Control

TouchOSC BuffetI know. You love your iDevice / Android. You love your phone, your tablet, your phablet, your you name it. You love them. Better yet, you’ve discovered Hexler’s TouchOSC and the thought of controlling your show / set / performance set you on fire – literally. You were beside yourself with glee and quickly set yourself to the task of triggering things remotely. Let’s be honest, it’s awesome. It’s beyond awesome, in some respects there are few things cooler than being able to build a second interface for a programming environment that works on your favorite touch screen device. But before we get too proud of ourselves, let’s have a moment of honesty. True honesty. You’ve dabbled and scrambled, but have you ever really sat down to fully understand all of the different kinds of buttons and switches in TouchOSC? I mean really looked at them and thought about what they’re good for? I know, because I hadn’t either. But it’s time, and it’s time in a big way.

TouchOSC_Editor_-_Untitled_1_touchoscFirst things first, make sure you download a copy of the TouchOSC editor for your OS of choice. If you happen to be using windows, save yourself a huge hassle and make sure that you download the version of the editor (32 or 64 bit) that corresponds to the version of Java that you’re running – otherwise you’ll find yourself unable to open the editor and be grumpy. Great. Now in the editor create a new control setup. Make sure you’re configured to work on the device that you’re planning on playing with / use the most. In my case I’m working with an iPad layout. I prefer to use my iPad in a horizontal orientation most of the time for show control. I’m also only working with one page for now, and happy to leave it as named “1”. You’ll notice for now that the box next to auto for OSC is checked – we’ll leave that for now. Alright, now we’re going to do something that most of us have never done.

In your empty control panel, right click and add one of every different kind of object. Yep. Add one of each so we can look at how they all work. You might choose to skip the repetition of vertical and horizontal sliders – that’s a-okay, but make sure you pick one of them. By the end you should have something that looks like this:

TouchOSC_Editor_-_Untitled_1_touchosc

That’s certainly not sexy, but it’s going to teach you more than you can imagine. Next upload that interface to your device of choice. If you’ve never uploaded a new interface to TouchOSC you’ll need to know the IP address of your computer. If you’re using a windows machine you can use the command prompt and ipconfig to find this information, on a mac use the network pane in System Preferences. Next you’ll want to transfer this layout to your device, Helxer has a wonderful piece of documentation about do get this done, and you can read it here.

Next take a moment to read through what all of those lovely widgets do and how they talk to your programming environment. After that the real fun starts. Start listening to your network with your programming environment of choice, and look at what kinds of messages you get from all of these different kinds of buttons and sliders.

Isadora

If you’re working with Troikatronix Isadora you can start to see the signal flow coming from your layout by first going to the communications drop down menu, and then selecting “Stream Setup.”

Communications_and_Menubar

Next you’ll want to select “auto detect input.”

Menubar_and_Untitled

Now you can start moving sliders and toggling buttons so you can record their address. Make sure that you select “Renumber Ports” and click “OK” on the bottom of the page.

Menubar_and_Untitled

Now you’re ready to use the OSC listener actor to use those inputs to drive any number of elements inside of your scene. You’ll notice that the “channel number” on the listener actor corresponds to the port number on the Stream-Setup dialog.

Menubar_and_Untitled

 

Now you’re off to the races. Do something exciting, or not, but play with all of the buttons, sliders, and gizmos so you know how they all work. For extra credit, remember that you can also send messages back to your layout. To do this you need to use the OSC Transmit actor. You’ll also need to know the IP address of your device, and the port that you’re transmitting to – you can find all of this information on the same page in TouchOSC where you initially set your target IP address.

Quartz Composer

If you’re working with Apple’s Quartz Composer you have to do a little more work to get your OSC stream up and running. For starters, you’ll need to know a little bit more about your OSC messages. You’ll notice in the TouchOSC editor that each object has an address that’s associated with it. For example, my fader is ” /1/fader1 “. To Touch OSC this reads as Fader 1 on Page 1 of your layout. In order to read information from this slider in Quartz, we’ll need to know this address, along with the type of information that we’re sending. In the case of a slider we’re sending a float (a number with fractional parts). To get started let’s open up a new Quartz Composer patch, and add an “OSC Receiver” from the library.

Menubar_and_Untitled_-_Editor

Now let’s take a closer look at that node. If we look at the inspector, and at the settings page we can see lots of useful information.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

Here we can see what port we’re set to receive information on by default, as well as an example of how we need to format the key’s from our layout so Quartz can properly receive the signals coming over the network. Let’s set up our fader to be properly decoded by Quartz. First we’ll need to remove the test key. Next we’ll want to add a new key by typing in the dialog box /1/fader1 and designating this as a float. Finally, click the plus sign to add this key the receiver. I usually add something else to my Quartz patch to make sure that I’m passing values through my network, this is optional.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

There you go. Now, to add the other buttons and sliders from your layout, you’ll need to similarly add keys for each of the buttons, sliders, and gizmos. Remember that you can find this information in your TouchOSC editor for this layout:

TouchOSC_Editor_-_Untitled_1_touchosc

Now you’re cooking with gas. Experiment with your layout, and how you can use this buttons and sliders to drive your Quartz Composer creations. For extra credit remember that you can also transmit back to your device. In this case you’ll need to use the “OSC Sender” object in Quartz. You’ll need to know the IP address of your target device, as well as the port that you’re communicating on. Have fun, and making something interesting… or if nothing else, learn something along the way.

TouchDesigner

If you’re working with Derivative’s TouchDesigner you have two options for receiving OSC information – CHOPs or DATS. Both of these methods work well but might have different uses for different situations.

Screenshot_032514_083023_PMLet’s start by looking at the OSC in CHOP. First we need to specify what port we want to listen to. Take a look at the TouchOSC layout to make sure the ports and addresses match. Then start moving sliders and buttons to see them appear in TouchDesigner.

Screenshot_032514_083345_PM

To use these to drive various parts of your network you can either use a select CHOP or reference these channels with expressions.

Next let’s look at what we see coming from the OSC In DAT.

Screenshot_032514_083545_PMHere instead of a changing single we see table with a message header and then with our float values coming in as static messages. Depending on the circumstance one or the other of these methods is going to help you drive your network.

For extra credit use the OSC out Chop to push values back to your TouchOSC layout – remember that you’ll need the IP address of your device, the port you’re receiving messages on, and you’ll need to format the header of the message correctly so that it corresponds to the right slider or button. Have fun learning and playing with your layout of all of the different kinds of controls.

TouchDesigner | Evaluate DAT Magic

13185491723_10766c1574_oLet’s say you’re an ambitions person, maybe too ambitious, and all of a sudden you have a control panel with 68 buttons that need to trigger various scenes, effects, and transitions. Let’s also say that your TouchDesigner network is  vast enough that you need (and I mean in a big way) a simple method for getting information about what’s active in your network. If you find yourself programming the media for a live theatrical event with these same kinds of concerns, the Evaluate DAT might just be the best friend that you didn’t know that you needed.

My example above might sound a little far fetched, so let’s instead look at a more concrete example. Let’s imagine that we have a digital puppet that we’d like to trigger to enter and exit with a button set to toggle. Just to make this more interesting, let’s say that you actually have three puppets and three visual effects that you’d like to all control with buttons from a single control panel. If all of your puppets and effects are all located in a similar place this might not be a huge hassle for you, if however, you’re building a complicated network it will soon become very important to be able to call the states of these buttons across your network.

Here’s my example set-up for us – a control panel that has all of our controls, a base component where we’re going to store the status of our buttons, and a scene where our puppet needs to make an entrance and an exit based on our button’s active state.

Screenshot_031614_012640_AM

I’m going to assume that you’re already familiar with working with control panels in TouchDesigner, but if that’s not the case you can learn a little more here before you keep reading. Lets start by taking a look inside of our “trigger_ref” base component where we are going to store the status of our buttons to use other places in our network.

Screenshot_031614_013152_AM

This should look pretty impressively boring at first glance. To really get a feel for what’s happening here, let’s take a closer took how how this is being driven by our control panel.

evaluate DAT in action

Alright, so here we can see that Puppet 1, 2, and 3 all act as Toggle switches, VFX 1 acts as a momentary trigger, and VFX2 and VFX3 both act as toggles. Before we start to look at why this is important, let’s first see how this is working.

To really make our Evaluate DAT sing we need to start by feeding a table with some instructions about what information we would like it to evaluate. In my case I’ve made a table with a name for the item that I’d like to call in one column, and the pathway to the operator and channel that’s being evaluated in the other column.

Screenshot_031614_013937_AM

If we were to write these instructions in English they might read something like: “TouchDesigner, please watch operator ‘button0’ in ‘ctrl’ and tell me when it’s v1 changes, and give that the name ‘Puppet1’. It’s important note that the formatting and syntax of your request to the Evaluate DAT matters. In my case I needed to put the names in the first column in quotations. I also made sure to tell the evaluate DAT that I wanted an integer (you’ll notice that in front of the operator’s pathway I’ve int() to specify that I want an integer returned to me). Next in the Evaluate DAT I need to check a few settings in the operators parameters.

Screenshot_031614_014828_AM

The most important parameter to take note of here is that I’ve told the Evaluate DAT that I’d like the output in Expressions. This ensures that my input table is being evaluated.

Okay, but why is this interesting / important / worth your attention? While it might be tempting to avoid an organization method like this, it means that when you’re working with a large control panel, or a large cue stack, that you can refer to a button by its name rather than by a complicated network pathway. Better yet, once you write the expression to call this value from our Evaluate DAT it becomes easy to copy and paste that expression and only need to change the name of the value that you’re calling. Let’s go back to our puppet example.

Screenshot_031614_020356_AMHere I have my video that I’d like to change with my puppet button. I’ve already set up my network with a few operators. First I have a constant CHOP, and a trigger CHOP. Next I have a Movie in TOP, a Level TOP, and a Null TOP. I’d like to use my trigger CHOP to change the opacity of my Movie.

To start let’s make sure that our constant CHOP is set up with a channel that we’ll name “Puppet1” .

Screenshot_031614_021044_AM

Let’s also make sure that our trigger is set to have a sustain value of 1, and let’s make sure that we’re happy with how the trigger is going to fire and release. For now I’m only interested in making sure that my sustain is changed.

Screenshot_031614_020704_AM Next we’re going to write a quick expression that will allow our level TOP to reference the trigger CHOP. In your level top we’re going to use the expression on the opacity Parameter on the Post page of Level TOP:

op( “trigger1” )[ “Puppet1” ]

Screenshot_031614_021154_AM

Now we’re going to go back to our constant CHOP to see how this really makes our network programming fun. Here for the value of the Puppet1 channel we’re going to write the following expression:

op( “../trigger_ref/buttons” ) [ “Puppet1” , 1 ]

Screenshot_031614_021552_AM

Now, let’s finally see what this means when we use this in conjunction with our control panel.

puppet inout

Great, so now we toggle the opacity of a video on and off with a button. What’s important here is that what if you suddenly decide that you want this action to happen with the Puppet 2 button instead? Now, rather than having to look up where your original button was, along with it’s reference you can instead just change the expression in the Constant CHOP to be:

op( “../trigger_ref/buttons” ) [ “Puppet2” , 1 ]

Right? We’ve changed all of those button values into a table that we can centrally reference by name. If you’re just getting started with TouchDesigner this might not seem like a huge revelation, but once you start to build more complicated networks the ability to call something by a name (maybe even a name that’s in your cue sheet) becomes hugely important.

TouchDesigner | Container Display

10620433406_3bcbc8f36e_oA quick one today that addresses problems I didn’t know that I had until I decided that I wanted to show off. In Wonder Dome I’m finding that sometimes the best way to build a scene is to use methods encapsulated inside of a container that make some change to some source imagery. This approach gives me a quick access to the parameters that I’m the most likely to use in a given method, without requiring that I always dive into the container to make changes. For example, I’ve build a handy tool for moving video assets around in the dome. This is fantastic, but if I drop this container into another container that needs its own set of buttons I quickly end up with a mess. Here’s an example of what I’m talking about:

Screenshot_030614_010956_PM

Looking at this container from the root folder I can the control panels for this network, and for it’s child. There are lots of ways to address this. I might, for example, just dive into the container find the child container and turn off its display parameter.

Screenshot_030614_012024_PM

That’s a fine approach if I’m only working with a couple of different scenes, but what happens when I suddenly have 10 or 20 scenes in a given network, each with a number of containers with control panels inside of them? Then I have a headache on my hands as I have to methodically go through and carefully disable each of the containers that I don’t want displayed.

We can solve this problem by using a python expression to conjure some TouchDesigner black magic. I’m going to solve this problem by asking my encapsulated containers to look at their parent operator and to complete a simple logical operation. When I’m ready to lock down a scene in my network, I add a capital “P” to the name of the container – for me this means it’s “P”erformance ready.

Now, instead of toggling that display parameter on my encapsulated container instead I’m going to use this expression:

0 if me.parent().name[0] == “P” else 1

“What on earth does this mean?!”
We can call for the name of our parent with the following expression:

me.parent().name

Better yet, we can ask for a specific character in from that name. If we ask for the character in the 0 position (remember that in programming languages 0 is often the first number in a list), we get the first letter from the name of our parent operator. Putting these two ideas together we get the expression:

me.parent().name[0]

Alright, now that’t we have the fist character from our parent operator’s name, let’s do a simple logical operation. Using an if else statement we can set one of two values. In this case I’m toggling between the values 0 or 1 (off or on). The syntax of this starts with the value if your logical operation is true, and then specifies what value to use if that statement is false. With this in mind, the whole expression is:

0 if me.parent().name[0] == “P” else 1

Alright, so if we were to look at our expression as a sentence it might read something like: “Hey TouchDesigner, if my parent’s name starts with the letter ‘P’ set this value to 0, otherwise leave it as 1.”

Screenshot_030614_012941_PM

 

Alright, now that we’ve written our expression in the right place, let’s see what it’s doing.

button expressions

 

Now we have a quick way to turn child control panels on and off without needing to dive into the container and hunt for our container display parameters.