Tag Archives: TouchOSC

OSC Remote Control | TouchDesigner

13771584713_fb81838217_mThere’s a lot to love about the internet, really. But I think one of my favorite things is how it connects people, how it flattens old hierarchies (not really, but let me wax idealistic for the sake of this intro) and connects people. In starting to program with TouchDesigner, I did the thing that any smart n00b would do – I joined the forum. The TouchDesigner forum is a great place to ask questions, find answers, learn from some of the best, and to offer help. We’ve all been stuck on a problem, and a commons like this one is a great place to ask questions, and keep tabs on what others are doing. To that end I shared a technique for sending and receiving OSC data with TouchDesigner back in October of 2013. I also shared this on the forum, because this happened to be something that I figured that others might want to know more about. My post was a simple example, but often it’s the simple examples that help move towards complex projects. As it turns out, someone else was fighting the same battle, and had some questions about how to make some headway – specifically they wanted to look at how to create an interface that could be controlled remotely with TouchOSC or from the TouchDesigner control panel itself. Ideally, each interface’s changes would be reflected in the other – changes on a smartphone would show up in the TouchDesigner control panel, and vice versa. I caught the first part of the exchange, and then I got swallowed by the theatre. First there was The Fall of the Hose of Escher, then Before You Ruin It took over my life, and then I spent almost a month solid in Wonder Dome. Long story short, I missed responding to a question, and finally made up for my bad Karma by responding, even if belatedly. It then occurred to me that I might as well write down the process of solving this problem. If you want to see the whole exchange you can read the thread here.

Enough jibber-jabber, let’s start programming.

For the sake of our sanity, I’m going to focus on just working with two sliders and two toggle buttons. The concepts we’ll cover here can then be applied to any other kind of TouchOSC widget, but let’s start small first. One more disclaimer, this one creates a messy network. If we took a little more time we could clean it up, but for now we’re going to let it be a little bit sloppy  – as much as it pains me to do so.

First things first, if you’re new to using TouchOSC you should take some time to get used to the basics and start by reading through this overview on using TouchOSC. Open Sound Control (OSC) messages are messages that are sent over a network. OSC can be used locally on a computer to allow for communication between programs, and it can also be used between multiple machines in order to allow them to communicate with one another. Interface tools like TouchOSC allow users to create custom interfaces that control some aspect of a program. That’s a very simplistic way of looking at OSC, but it’s a good start. The important takeaway here is that when we use OSC we’re actually relying on some network protocols to facilitate the communication between computers.

To get started I selected a simple interface on an iPod Touch that came with TouchOSC. Again, I wanted something with two toggles and two sliders to serve as our example. We can set-up our mobile device by starting TouchOSC and hitting the circular option / configuration button in the corner.

Once we do that we’ll see a screen with a few different options. We want to select the OSC connection tab.

oBGZO3662ohWeEYLEld_chDzyCFrrH1KFjmrRzwl0gA

From here we need to configure a few settings. First off you’ll want to set add the IP address of your computer into the “Host” field. To find your computer’s IP address you can use the ipconfig command to quickly find your IP address (if this sounds like another language, check out this youTube video to see what I’m talking about). I also want to take note of the IP address of the device – this is the “Local IP address.” I also want to take a close look at the outgoing and incoming ports, I’ll need to use these numbers in TouchDesigner to make sure that I can talk and listen to this device.

eAPW1uy3r24Z1X5Xba6Y6CRsU8g623zOkAU_BJIKXtY

Alright, now that we know a few things about our mobile device now we can head back to TouchDesigner.

First off, let’s take a look at how to listen to the incoming data. In a new network at an OSC In CHOP.

OSC_In

In the parameters box let’s make sure that we’re listening to the same port that we’re broadcasting on – in my case it’s port 10000. Now you should be able to start hitting buttons and moving sliders to see some new channels appear in your CHOP. Here’s it’s important to take a closer look at the naming convention for our channels. Let’s look at 1/fader1 first. TouchOSC has a great utility for creating your layouts, and if you let it do the naming for you this is the kind of format that you’ll see. The semantics of this are page/widgetNameWidgetNumber. So by looking at 1/fader1 we can read this to mean that this is on page one, it’s a fader, and it’s number (the order it was created) is one. This naming convention is going to be important to take note of, and will save you a lot of headaches if you take some time to really wrap your head around how these widgets are named.

Before we start building an interface to control let’s take a moment to get a few more ducks in a row. I’m going to connect my OSC In CHOP to four different select CHOPS – each control is going to be routed to a button or slider, and I want to make sure that I’m only dealing with one channel to do this.

4selectsFor my own sanity I’ve named each of the selects with a name that corresponds to what channel they’re carrying. You can choose any naming convention that you’d like, but definitely choose a naming convention. This kind of patching can get messy quickly, and a solid method for naming your operators will server you well.

Before we move on let’s take a closer look at one of the selects to see how we can pull out a particular channel.

fader1

You’ll notice that in the Channel Names I’ve used the name *fader1 rather than 1/fader1. What gives?! Either of those names is going to give me the same result in this case. I’ve elected to use the asterisk modifier to save myself some time and because I’m only using page one of this particular interface. What does the asterisk modifier do? I’m so glad you asked – this particular modifier will give results that match anything after the asterisk. If, for example, I had 1/fader1, 2/fader1, and 3/toggle1 as incoming channels, the asterisk would pull out the 1/fader1 and 2/fader1 channels. Naming and patterning isn’t important for this particular example, but it’s a technique to keep your back pocket for a rainy day.

Okay, now that we’ve got our OSC In data ready, let’s build a quick interface. In this instance I’m going to use two vertical sliders from the TUIK presets in the pallet browser. You can find them here:

TUIK

I’m also going to add two Button COMPs  and a Container COMP to my network.

interfaceSetup

Before we move forward, I need to make a few quick changes to by buttons. You’ll notice that the sliders both have CHOP inlets on their left-hand side, but my buttons do not. This means that my sliders are all ready set-up to receive a channel stream to change them. My buttons aren’t ready yet, so let’s take a look at the changes we need to make. Let’s start by diving into our button.

buttonCOMP

If this is your first time taking a look inside of button comp take a moment to read through a quick overview about working with buttons in TouchDesigner. We’re going to want to add a CHOP In to this component, and we’re going to also want the changes from the Touch interface to drive how our button’s color changes. Here’s what that’s going to look like:

new buttonHere I’ve added an In CHOP that feeds to a Math CHOP that I’ve renamed to “i” that’s finally connected to our Out CHOP. So why a Math, and why rename it? If you look closely at the Text TOP  you’ll see that it’s driven by several expressions allowing it to change color based on its active state – this is part of what’s happening in the expression CHOP originally called “i”. Here our Math CHOP is set to add our in and the “ii” CHOP (formerly just “i”) together. By changing the name of the operators, I’ve avoided re-writing the scripts to save myself some time.

With our buttons and sliders finally ready we can start connecting our interface elements. There are lots of ways to build interface elements in TouchDesigner, but today I’m just going to use the ability to wire components vertically to do this. First I’m going to layout my buttons and sliders in an approximate location that matches my TouchOSC layout just to help me get organized initially. Next I’m going to connect them through their vertical inlets to my container COMP. In the end, if you’re following along, it should look something like this:

vierticalwires

Finally, you’ll want to take some time to adjust the placement of your buttons and sliders, as well as adjusting their color or other elements of their appearance. My final set up looks like this:

finalcontrol

If you’re still with me, this is where the real magic starts to happen. First up we’re going to connect the our corresponding OSC In values to our control panel elements that we want to control. Fader 1 to fader 1, toggle 1 to toggle 1 and so on. Next we’re going to connect all four of our control panel elements to a single Merge CHOP.

almsot there

Next we’ll need to do a little renaming. To make things easier I’m going to use a Rename CHOP and a Constant CHOP. Here our Constant CHOP holds all of the names that we want to apply to the channels that are coming into our Rename CHOP. Here’s where all of that funny business about naming our channels becomes important. To make sure that I’m feeding back data to TouchOSC in a way that properly associates changes to my sliders I need to follow the naming conventions exactly the way they’re coming into TouchDesigner. 1/fader1 that’s since become v1 needs to have it’s name changed back to 1/fader1. You can see what I mean by taking a closer look at the network below:

reanme

Last but not least we’re going to connect our rename CHOP to an OSC Out CHOP. When we set out our OSC out we need to know the IP address of the device that we want to communicate with, and we need to know the port that we’re broadcasting to. I also like to change my OSC Out to be samples, and to turn off “Send every cook.” Sending with every cook is going to create a lot more network traffic, and while it’s not an issue for TouchOSC, if you’re working with someone using MaxMSP having this trick up your sleeve is going to make them much happier. Here’s what our OSC Out operator should look like:

OSCOUT

Whew! Alright gang, at this point you should be ready to start making the magic happen. If you’ve got everything set-up correctly you should now be able to drive your control panel in TouchDesigner either from the panel we created, or from a TouchOSC setup. As a bonus (why we did all of this hard work) you should also be able to see your changes in either environment reflected in the other.

simplefeedback

TouchOSC | Serious Show Control

TouchOSC BuffetI know. You love your iDevice / Android. You love your phone, your tablet, your phablet, your you name it. You love them. Better yet, you’ve discovered Hexler’s TouchOSC and the thought of controlling your show / set / performance set you on fire – literally. You were beside yourself with glee and quickly set yourself to the task of triggering things remotely. Let’s be honest, it’s awesome. It’s beyond awesome, in some respects there are few things cooler than being able to build a second interface for a programming environment that works on your favorite touch screen device. But before we get too proud of ourselves, let’s have a moment of honesty. True honesty. You’ve dabbled and scrambled, but have you ever really sat down to fully understand all of the different kinds of buttons and switches in TouchOSC? I mean really looked at them and thought about what they’re good for? I know, because I hadn’t either. But it’s time, and it’s time in a big way.

TouchOSC_Editor_-_Untitled_1_touchoscFirst things first, make sure you download a copy of the TouchOSC editor for your OS of choice. If you happen to be using windows, save yourself a huge hassle and make sure that you download the version of the editor (32 or 64 bit) that corresponds to the version of Java that you’re running – otherwise you’ll find yourself unable to open the editor and be grumpy. Great. Now in the editor create a new control setup. Make sure you’re configured to work on the device that you’re planning on playing with / use the most. In my case I’m working with an iPad layout. I prefer to use my iPad in a horizontal orientation most of the time for show control. I’m also only working with one page for now, and happy to leave it as named “1”. You’ll notice for now that the box next to auto for OSC is checked – we’ll leave that for now. Alright, now we’re going to do something that most of us have never done.

In your empty control panel, right click and add one of every different kind of object. Yep. Add one of each so we can look at how they all work. You might choose to skip the repetition of vertical and horizontal sliders – that’s a-okay, but make sure you pick one of them. By the end you should have something that looks like this:

TouchOSC_Editor_-_Untitled_1_touchosc

That’s certainly not sexy, but it’s going to teach you more than you can imagine. Next upload that interface to your device of choice. If you’ve never uploaded a new interface to TouchOSC you’ll need to know the IP address of your computer. If you’re using a windows machine you can use the command prompt and ipconfig to find this information, on a mac use the network pane in System Preferences. Next you’ll want to transfer this layout to your device, Helxer has a wonderful piece of documentation about do get this done, and you can read it here.

Next take a moment to read through what all of those lovely widgets do and how they talk to your programming environment. After that the real fun starts. Start listening to your network with your programming environment of choice, and look at what kinds of messages you get from all of these different kinds of buttons and sliders.

Isadora

If you’re working with Troikatronix Isadora you can start to see the signal flow coming from your layout by first going to the communications drop down menu, and then selecting “Stream Setup.”

Communications_and_Menubar

Next you’ll want to select “auto detect input.”

Menubar_and_Untitled

Now you can start moving sliders and toggling buttons so you can record their address. Make sure that you select “Renumber Ports” and click “OK” on the bottom of the page.

Menubar_and_Untitled

Now you’re ready to use the OSC listener actor to use those inputs to drive any number of elements inside of your scene. You’ll notice that the “channel number” on the listener actor corresponds to the port number on the Stream-Setup dialog.

Menubar_and_Untitled

 

Now you’re off to the races. Do something exciting, or not, but play with all of the buttons, sliders, and gizmos so you know how they all work. For extra credit, remember that you can also send messages back to your layout. To do this you need to use the OSC Transmit actor. You’ll also need to know the IP address of your device, and the port that you’re transmitting to – you can find all of this information on the same page in TouchOSC where you initially set your target IP address.

Quartz Composer

If you’re working with Apple’s Quartz Composer you have to do a little more work to get your OSC stream up and running. For starters, you’ll need to know a little bit more about your OSC messages. You’ll notice in the TouchOSC editor that each object has an address that’s associated with it. For example, my fader is ” /1/fader1 “. To Touch OSC this reads as Fader 1 on Page 1 of your layout. In order to read information from this slider in Quartz, we’ll need to know this address, along with the type of information that we’re sending. In the case of a slider we’re sending a float (a number with fractional parts). To get started let’s open up a new Quartz Composer patch, and add an “OSC Receiver” from the library.

Menubar_and_Untitled_-_Editor

Now let’s take a closer look at that node. If we look at the inspector, and at the settings page we can see lots of useful information.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

Here we can see what port we’re set to receive information on by default, as well as an example of how we need to format the key’s from our layout so Quartz can properly receive the signals coming over the network. Let’s set up our fader to be properly decoded by Quartz. First we’ll need to remove the test key. Next we’ll want to add a new key by typing in the dialog box /1/fader1 and designating this as a float. Finally, click the plus sign to add this key the receiver. I usually add something else to my Quartz patch to make sure that I’m passing values through my network, this is optional.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

There you go. Now, to add the other buttons and sliders from your layout, you’ll need to similarly add keys for each of the buttons, sliders, and gizmos. Remember that you can find this information in your TouchOSC editor for this layout:

TouchOSC_Editor_-_Untitled_1_touchosc

Now you’re cooking with gas. Experiment with your layout, and how you can use this buttons and sliders to drive your Quartz Composer creations. For extra credit remember that you can also transmit back to your device. In this case you’ll need to use the “OSC Sender” object in Quartz. You’ll need to know the IP address of your target device, as well as the port that you’re communicating on. Have fun, and making something interesting… or if nothing else, learn something along the way.

TouchDesigner

If you’re working with Derivative’s TouchDesigner you have two options for receiving OSC information – CHOPs or DATS. Both of these methods work well but might have different uses for different situations.

Screenshot_032514_083023_PMLet’s start by looking at the OSC in CHOP. First we need to specify what port we want to listen to. Take a look at the TouchOSC layout to make sure the ports and addresses match. Then start moving sliders and buttons to see them appear in TouchDesigner.

Screenshot_032514_083345_PM

To use these to drive various parts of your network you can either use a select CHOP or reference these channels with expressions.

Next let’s look at what we see coming from the OSC In DAT.

Screenshot_032514_083545_PMHere instead of a changing single we see table with a message header and then with our float values coming in as static messages. Depending on the circumstance one or the other of these methods is going to help you drive your network.

For extra credit use the OSC out Chop to push values back to your TouchOSC layout – remember that you’ll need the IP address of your device, the port you’re receiving messages on, and you’ll need to format the header of the message correctly so that it corresponds to the right slider or button. Have fun learning and playing with your layout of all of the different kinds of controls.

Sending and Receiving OSC Values with TouchDesigner

The other day I posted look at how to send and receive TouchOSC data with Troikatronix’s Isadora. In an effort to diversify my programming background I’m often curious about how to translate a given task from one programing environment to another. To this end, I was curious about repeating this same action in Derivative’s TouchDesigner. If you’re one of a growing number of artists interested in visual programming, VJing, media design, interactive system design, media installation, or media for live performance than it’s well worth your time to look at both Isadora and TouchDesigner.

That said, I initially started thinking about control panel interfaces for TouchOSC when I saw Graham Thorne’s instructional videos about how to transmit data from Isadora to Touch OSC (If you have a chance take a look at his posts here: Part 1 & Part 2). Last year I used TouchOSC accelerometer data to drive an installation at Bragg’s Pie Factory in Downtown Phoenix, but I didn’t do much with actual control panels. Receiving data from TouchOSC in TouchDesigner is a piece of cake, but sending it back out again didn’t work the first time that I tried it. To be fair, I did only spend about 3 minutes trying to solve this particular question as it was a rather fleeting thought while working on another project.

Getting this to working Isadora, however, gave me a renewed sense of focus and determination, so with a little bit of experimenting I finally got this working the way that I wanted. While there are any number of things you might do with TouchDesigner and TouchOSC, I wanted to start with a simple deliverable: I wanted the position of Slider 1 to inversely change the position of Slider 2, and vise versa. In this way, moving Slider 1 up moves Slider 2 down, and moving Slider 2 up moves Slider 1 down. In performance setting it’s unlikely that I’d need something this simple, but for the sake of testing an idea this seemed like it would give me the kind of information that I might need.

For the most part this is pretty easy, but there are a few things to watch for in order to get the results that you want.

Here’s the network stream of Chanel Operators (CHOPs):

OSC In – Select – Math – Rename – OSC Out

OSC In

Derivative Documentation on the OSC In CHOP 

The OSC in CHOP is pretty excellent out the gate. We need to do a little configuring to get this up and running, but after that we’re truly off to the races. First off it’s important to make sure that your TouchOSC device is connected to your network and broadcasting to your TouchDesigner machine. If this process is new to you it’s worth taking a little time to read about out to set this all up – start on this page if this is new to you. Once your TouchOSC device is talking to your TouchDesigner network we’re ready to start making magic happen. One of the first things that I did was to activate (just adjust up or down) the two faders that I wanted to work with. This made sure that their IDs were sent to TouchDesigner, making it much easier for me to determine how I was going to rename in them in the future.

Screenshot_100913_094345_PMSelect

Derivative Documentation on the Select CHOP

When you’re working with the OSC In CHOP, all of your TouchOSC data comes through a single pipe. One of the first things to do here is to use the select CHOP to single out the signal that you’re looking for (as a note, you could also export the Channel in question to a Null, or handle this a variety of other ways – in my case a Select CHOP was just the first method I tried). This operator allows you to pull out a single Channel from your OSC In.

Screenshot_100913_094428_PM

Math

Derivative Documentation on the Math CHOP

Next I wanted to remap the values of my incoming signal to be inverted. A Math CHOP lets us quickly remap the range for our data by changing a few values in the Range Tab of that parameters window. You’ll notice that you first need to specify the from range (what’s the range of incoming values), and then specify the to range (what’s the range of outgoing values). In my case, the from values are 0 and 1, and the to values are 1 and 0.

Screenshot_100913_094528_PM

Rename

Derivative Documentation on the Rename CHOP

The Rename CHOP is one of the most important steps in your whole network. When communicating with TouchOSC you need to know exactly what object you’re trying to drive with your data. To accomplish this you need to know the UDP address of your device, the port number that you’re board casting to, and the name of the slider or button that you’re wanting to change. While we can specify the UPD address and port number with another CHOP, we need to use the Rename CHOP (a quick aside – there are, of course, other ways to do this – this just happens to be the method that I got working after a bit of trial and error) to specify the name of the object that we’re driving. In my case I wanted fader 1 to drive fader 2. You’ll notice that formatting of the fader is important. In this particular OSC layout there are multiple tabs. We can see this as evidenced by the “1/” preceding the name of our fader. In changing the name of this input we need to enter the precise name of the fader that we want to change – this is why we activated both faders initially, this gave us the names of both objects in the TouchOSC panel. You’ll notice that changed the name of my fader to “1/fader2”.

Screenshot_100913_094606_PM

OSC Out

Derivative Documentation on the OSC Out CHOP 

Now that we’ve processed and named our signal we’re ready to send it back out to TouchOSC. In this CHOP we need to input the UDP address of the TouchOSC device (you can find this in the info tab of the TouchOSC app) as well as the port that we’re sending to. We also want to make sure that we’re sending this as a sample, and finally we want to turn off “send events every cook.” This last step is important because it ensures that we’re only sending values when they change. If we send messages for every cook we won’t be able to see the inverse relationship between our two sliders. Correction, we’ll be able to see the relationship so long as we don’t set up slider 2 to drive slider 1. In order to create an inverse relationship with both sliders we only want to transmit data as it’s changed (otherwise we’ll find that we’re fighting with the constantly streamed position data from our slider that isn’t being activated).

Screenshot_100913_094705_PM

The last step in this process is to copy the whole string and set up fader 2 to transmit to fader 1, this last step allows both faders to drive one another in turn.

There you have it. You’ve now successfully configured your TouchDesigner network to receive and send data to a mobile device running TouchOSC.

Sending and Receiving TouchOSC Values in Isadora

Sometime last year (in 2012) I came across Graham Thorne’s instructional videos about how to transmit data from Isadora to Touch OSC. Here’s Part 1 and Part 2 – if this is something that interests you, I’d highly recommend by that you watch these two videos first.

While I didn’t have a TouchOSC project at the time that I was working on, it got me thinking about how interfaces communicate information about what’s happening in a patch, and how that information communicates to the operator or user of a given system. This year I’m working on the thesis project of Daniel Fine (an MFA student here at ASU), and one of the many challenges we’re bound to face is how to visualize and interact with a system that’s spread across multiple computers, operating systems, and controlling a variety of different systems in the installation / performance space.

To that end, I thought that it would be important to start thinking about how to to send control data to a TouchOSC interface, and how to then ensure that we can see relationships between different control values in a given control panel. That all sounds well and good, but it’s awfully vague. A concrete exploration of this kind of concept was what I needed to start planning, in order to more fully wrap my head around how idea could be more fully exploited in a performance setting.

In order to do this I decided that I wanted to accomplish a simple task with a TouchOSC control panel. On a simple panel layout I wanted the position of Slider 1 to inversely change the position of Slider 2, and vise versa. In this way, moving Slider 1 up moves Slider 2 down, and moving Slider 2 up moves Slider 1 down. In performance setting it’s unlikely that I’d need something this simple, but for the sake of testing an idea this seemed like it would give me the kind of information that I might need.

First let’s look at the whole patch:

The Whole Patch

The set up for this starts by configuring Isadora to receive data from TouchOSC. If you’re new to this process start by reading this post (or at least though the Stream Set-Up section) to learn about how to start receiving data in Isadora from TouchOSC. Next we’re going to use a few simple actors to make this happen. We’re going to use the following actors:

  • OSC Listener
  • Limit Scale Value (there are other ways to scale values, I just like this method as a way to clearly see what values you’re changing and in what way)
  • OSC Transmitter

OSC ListenerOnce you have your connections between TouchOSC and Isadora set up you’ll want to make sure that you’ve isolated a single slider. We can do this by using the OSC Lister Actor. The OSC Listener reports the data coming from a given channel that you specific in the input inlet on the actor. The listener then sends out the transmitted values from the value outlet.

Limit Scale ValueWe have two sliders that we’re working with – Channel 1 and 2 respectively (they also have names, but we’ll get to that later). We first want to look at Channel 1. We’re going to set the OSC Listener actor to channel 1 and then connect the value output to the value inlet on a Limit-Scale Value Actor. The Limit-Scale Value Actor allows you to change, scale, or otherwise remap the floats or integers to a new range of values. This is important because TouchOSC uses normalized values (values from 0-1) for the range of the sliders in its control panels. In order to create an inverse relationship between two sliders we want to remap 0-1 to output as values from 1-0. We can do that by enter the following values in the inlets on the actor:

  • limit min: 0
  • limit max: 1
  • out min: 1
  • out max: 0

OSC TransmitThe remaining step is to connect the output from our Limit-Scale Value Actor to an OSC Transmit Actor. The OSC Transmit Actor, like its name suggests, transits data wrapped in the OSC protocol. In order to fully understand how this data is being transmitted we need to know a few things about this actor. In looking at its components we can see that it is made up of the following inlets:

  • UDP Addr – UDP address. This is the IP address of the computer or device that you’re taking to. You can determine what this address is in TouchOSC by looking at the info panel. Imagine that this is the street name for a house that you’re sending a letter to.
  • Port – this is the port on the device that you’re sending data to. It’s important that you know what port you’re trying to talk to on a given device so that your message can be parsed. If the UDP Address is the street name, the port number is akin to the house number that you’re trying to send a letter to.
  • Address – The address in the case of TouchOSC is the individual target / name of an asset that you want to change. Each of the sliders and buttons on a TouchOSC panel have a name (for example – /1/fader1), the address is how you tell Isadora what slider you are wanting to change. You can determine these names by looking closely at your Stream Set-up when you’re connecting your device to Isadora. To follow with our letter sending metaphor above, the Address is the name of the person you’re sending the letter to.
  • Use Type – this allows us to toggle the sending mechanism on and off.
  • Value – this is the value that we’re transmitting to our other device.

To use the OSC Transmit actor we need to fill in all of the appropriate fields with the information from our mobile device. You’ll need to specify the UPD Address, the Port number, the Address, and connect the value out form our Scale-Value actor to the value inlet of the OSC Transmit Actor.

In this test I started by having fader1 drive fader2. Once I got this working, I then repeated all of the steps above, for the other fader – if you look closely at the full patch this will make more sense. The resulting interaction can be seen in the gifs below.

 

Visualizing OSC Data | TouchDesigner

After looking at how to work with accelerometer data in Isadora in an earlier post, I thought it might also be worth looking at how to approach the same challenge working with Derivative’s TouchDesigner. In the Spring of 2013, for an installation piece, I used TouchDesigner to create a sculpture with a reactive projection component. While I learned a lot about working with TouchOSC in the process, I didn’t spend much time really digging into understanding what kind of data I was getting out of my sensor – in this case I’m using an iPod Touch running TouchOSC, broadcasting data over a wireless network. This type of sensor is one that I hope to use in furutre live performances, and so spending some time really digging into what the detects is an area of interest for me.

TouchOSC to TouchDesigner

To start we need to first capture the OSC data with TouchDesigner. I’m using TouchOSC for this process. With TouchOSC installed on your mobile device you first need to make sure that’ve identified your IP address, and chosen a port number that corresponds to the port number you’ve selected in TouchDesigner. Next we need to ensure that TouchOSC is broadcasting the accelerometer data. You can do this by first tapping on the “Settings” icon in the upper corner of the TouchOSC panel. From here selection “Options” Under options we need to make sure that “Accelerometer (/xyz)” is enabled. Back in TouchDesigner we’ll need to do a little work to get started. You’ll need to create a new network, and toss out all of the template OPs so we can get started fresh. You can choose to create a new container or to use the one that’s present in the template file. Once we’re in this first container we’ll start by creating an OSC In Channel Operator (CHOP). I’ve set my network port to 5001 (the same as in TouchOSC). Once your port numbers are set you should see thee bars in the CHOP reporting out the data from TouchOSC. We can see with a quick glance here that we have three channels of information: accxyz1, accxyz2, and accxyz3. Once we dive into our network we’ll come back to this in order to control our visualization.

The Big Picture

First let’s take  look at the full patch and talk through what our next steps are going to be. In order to have a fuller sense of what the accelerometer data is doing, we’re going to create a simple visualization. We’ll do this by creating a small sphere for each data stream, and map their change to the vertical axis, while mapping a constant change to the horizontal axis. We’ll then set up a simple feedback effect so we can see a rough graph of this data over time that will automatically be cleared each time our visualization wraps around the screen. This approach should allow us to see how the three data channels from the accelerometer correlate to one another, as well as giving us a chance to consider how we might work with this kind of data flow inside of a TouchDesigner network. Start to finish we’ll work with all of the different classes of operators (Components, Textures, Channels, Surface, Materials, and Data), as look at connecting operators and exporting paraemters.

Getting our OSC Data Ready

While the process of programming often bounces back and forth between modules, I’m going to start us out by looking at how I’ve parsed the OSC data so that it can be useful for us as we transition to other operators in our network. The flow of operators looks like this:

OSC In – Select – Lag – Math – Null

The same OSC in is passed to two copies of the “select – lag – math – null” string. While there are plenty of other ways to accomplish this same flow of data, this will allow us to make changes to the mapping of data coming out of individual channels (if we happen to need that option). We start with the OSC In, and then pass this value to a Select CHOP. The Select CHOP helps us to pull a single stream of data out of our OSC In Chop. In adding the Select CHOP make sure that in your parameters window (short-cut key “p”) you’ve selected the appropriate channel. You’ll notice that in the example below, I’ve selected “accxz1” as the first select operator. Next I’ve added a Lag CHOP. This operator will allow us to control the rate at which the data is being processed from our sensor. In essence, this allows us to smooth out the noise from the accelerometer, in effect making it a little less sensitive. Your mileage may vary here, but for my current configuration I’ve set the Lag values to 0.5 and 0.5. After you get your system up and running you may well want to return to these parameters, but for the time being this is a fine place to start. The Math CHOP allows us to remap the incoming values to a different range. This is another place that will require some individual adjustments once you get your whole network set up. To get started we can begin by setting the from range to -2, and 2. The To Range should be set to -1.5, and 1.5. While you’re not likely to see values of -2 or 2 coming from your accelerometer when you get started, this will make sure that there isn’t any data that’s outside of our measurement range. We’ll end our string with a Null. It’s considered good TouchDesigner programming practice to always end your strings with a Null. This ensures that if you want to change your string, add operators, or in any way alter your work, you don’t need to re-export any data. So long as you have ended your work with a Null, you can always make changes upstream of this object. While it feels cumbersome when you’re first starting in TouchDesigner, it’s hands down a practice that will save you time and headaches as you make increasingly complicated networks. Last, but not least, don’t forget to add a Text DAT to the front end of your string. This operator doesn’t do anything in terms of function, but it does allow you a space to write comments to yourself (or whoever is going to be working with your network). Making a few notes about how your network is working, and your underlying thoughts in setting up your chain of operators will help refresh your memory when you come back to a portion of your network after focusing on other areas.

Working with Surface Operators

Anytime we’re drawing some geometry in TouchDesigner we need to Surface Operators (SOPs). While we could do this same process with an image and by only using Texture Operators (TOPs) that means that we’re relying on something other than TouchDesigner to create our artwork. For this particular exercise, I think it’s worth thinking about how we might use TouchDesigner exclusively to create the visualization we’re going to see. Let’s take a look at the whole SOP network before we dive into how to put this all together.

As a general practice we can see out the gate that we’re going to use a single sphere and then create three copies of that single geometry that we’ll then change by exporting our TouchOSC CHOPs to change the behavior of our spheres. We’ll end the whole process by passing our SOPs into a Geometry component, and applying a Phong shader.

Let’s start by first creating a single Sphere SOP. Next we’ll connect that to a Transform SOP. Next we’ll pass our sphere into a Geo Component. To do this, we’ll need to make a few changes to the standard Geo Component. Start by creating a new Geo Component. Double click on the Geo. Once inside, delete the place holding Torus and add an In SOP. Make sure that you turn on both the blue display flag and the purple render flag on the the In SOP. By replacing the Torus with an In we’ll now be able to pass in the SOP operator strings outside of the Geo Component. Exit the Geo by hitting the “u” key (u for up), or by using the mouse wheel to zoom out of the component. You should now notice that there’s an inlet on the left side of our Geo Component. You can now connect the Transform SOP to the Geo, and you should see a picture of the sphere in the window.

Next create a Material called a Phong. Our Phong is going to allow us to add some color to our Geo. Drag the Phong on top of the Geo – you should see an arrow with a plus sign appear to tell you that you’re applying this Phong shader to this Geo Component. In the options for the Phong you can now select any color you’d like. With one whole string completed, you can select the string from Transform through Phong, copy and paste the string two more times for a total of three spheres. Make sure that you connect the Sphere SOP to the two additional strings, and you should be in business.

Next we’re going to export the CHOPs from our accelerometer channels to change the location of our spheres. To make this tutorial readable, I’m going to forgo detailing how to export CHOPs. If this isn’t something you’ve done before, you can look for more information about exporting CHOPs in TouchDesigner by first looking at Derivative’s official documentation here.

Export the Null CHOPs for accxyz1-3 to the Transform SOPs of the three respective Spheres. Make sure that you export the Nulls to the Y position field of the Transform SOPs. All three speheres should now be moving up and down based upon the change in accelerometer data  for each channel.

We’re also going to want to be able to change the size of our spheres based upon the over all look of our visualization. To do this we’re going to set up a simple string of CHOPs that we’ll then export to all three Transform SOPs. Start by creating a Constant CHOP and connect it to a Null CHOP. The Constant outputs a constant number in a channel and will allow us to change the dimensions of our spheres collectively. Next export the Null from this string to all three size attributes (x, y, z) of all three transform SOPs. In the parameters of the Constant change the value of the channel to 0.025.

Finally, don’t forget to add a Text DAT. Again, this is a great place to add some notes to yourself, or notes for your operator about what you’ve done and what’s important to consider when coming back to work with this string later.

Changes over Time

Moving up and down is only a part of this challenge. While what we’ve made so far is interesting, it still lacks a few key elements. For starters, we want to be able to see this change over time. In our case that means seeing the spheres move from left to right across the screen. Here’s a look at the string of operators that’s going to help us make that happen:

Wave – Math – Null 
and
Wave – Math – Trigger – Null

We’re going to use a single Wave CHOP to drive the lateral movement of our spheres, and to act as the trigger to erase our video feedback that we’ll make once we’ve set our geometry to render.

The first thing we need to do is to create a Wave CHOP. This operator will create waves a regular intervals which we can control with the attributes of this operator. First we want to change the type of wave to be ramp. A ramp will start at 0 and increase to a specified value, before returning to 0. Next we’ll set the the Period and the Amplitude to 30.

Next we need to look at the Channel Tab of this operator’s parameters. On the Channel tab we need to set the End value to 30 seconds. The values that we’re currently changing are time it takes for the wave to cycle. You may ultimately find that you’d like this to happen faster or slower, but we can use these values at least as a place to start. We also need to set the End and REnd values to be 1800 ( 60 frames per second * 30 seconds = 1800 frames). This will ensure that we have enough time for our animation to actually wrap from right to left.

The first Math CHOP that we’re going to change is on the top string. This operation is going to scale our wave values to make sure that we’re starting just off the screen on the left, and traveling to just off the screen on the right. In the From Range insert the values 0 and 30. In the To Range change the values to -2.1 and 2.1. Connect this to the Null. Next export the Null CHOP to the X position field of all three Transform SOPs.

One of the things we can do to plan ahead, is to build a trigger based on this same wave that will erase our video feedback that’s going to be generated in the rendering process. Create a new Math CHOP and connect it to the same Wave CHOP. In the From Range change the values to 0 and 30, and in the To Range change the values to 0 and 1.

Next we’re going to add a Trigger CHOP to our second string. A trigger can be used for any number of purposes, and in our case it’s going to help ensure that we have a clean slate for each time our spheres wrap from the right side of the screen back to the left. With the Math CHOP connected to the trigger, the only change we should need to make is to ensure that the Trigger Threshold is set to 0. Connect the trigger to a Null. We’ll come back to this CHOP once we’re building the rendering system for our visualization.

Rendering

Rendering a 3D object is fairly straight forward provided that you keep a few things in mind. The process of rendering an object requires at three components: at least one Geometry (the thing being rendered), a Camera (the perspective from which it’s being rendered), and a Light (how the object is being illuminated). We’ve already got three Geo components, now we just need to add a light and a camera. Next we’ll add a Render TOP, and we should see our three spheres being drawn in the Render TOP window. If you want to know a little more about how this all works, take a moment to read through the process of rendering in a little more detail. Let’s take a look at our network to get a sense of what’s going to happen in the rendering process.

There are a couple of things happening here, but the long and short of it is that we’re going to create a little bit of video feedback, blur our image, add black background, and create a composite out of all of it.

We’re going to start by making these two strings:

Render – Feedback – Level – Composite – Blur
and
Render – Level – Over – Over – Out

Here’s a closer look at the left side of the strings of operators that are rendering our network (as a note, our Render TOP is off the screen the left):

Once our string is set up, make sure that your Feedback is set to target Comp1. Essentailly what we’re creating here a layer of rendered video that is the persistent image of the spheres passing from left to right. Additionally, at this point we want to export the trigger CHOP to the feedback TOP to the bypass parameter. When the bypass value is greater than 0 it turns off the feedback effect. This means that when our trigger goes off the screen will clear, and then the bypass value will be set back to 0.

Now let’s look at the right side of our rendering network:

Here we can see our over TOPs acting as composite agents. The final Over TOP is combined with a Constant TOP acting as a black background. Finally all of this is passed to an Out TOP so it can be passed out of our container.

That’s it! Now we’ve built a simple (ha!) visualization for the three channels of data coming out of an iPod Touch accelerometer passed over a wireless network. The next steps here are to start to play. What relationships are interesting, or not interesting. How might this be used in a more compelling or interesting way? With a programming environment like TouchDesigner the sky is really the limit, it’s just a matter of stretching your wings.