Tag Archives: OSC

TouchOSC | Serious Show Control

TouchOSC BuffetI know. You love your iDevice / Android. You love your phone, your tablet, your phablet, your you name it. You love them. Better yet, you’ve discovered Hexler’s TouchOSC and the thought of controlling your show / set / performance set you on fire – literally. You were beside yourself with glee and quickly set yourself to the task of triggering things remotely. Let’s be honest, it’s awesome. It’s beyond awesome, in some respects there are few things cooler than being able to build a second interface for a programming environment that works on your favorite touch screen device. But before we get too proud of ourselves, let’s have a moment of honesty. True honesty. You’ve dabbled and scrambled, but have you ever really sat down to fully understand all of the different kinds of buttons and switches in TouchOSC? I mean really looked at them and thought about what they’re good for? I know, because I hadn’t either. But it’s time, and it’s time in a big way.

TouchOSC_Editor_-_Untitled_1_touchoscFirst things first, make sure you download a copy of the TouchOSC editor for your OS of choice. If you happen to be using windows, save yourself a huge hassle and make sure that you download the version of the editor (32 or 64 bit) that corresponds to the version of Java that you’re running – otherwise you’ll find yourself unable to open the editor and be grumpy. Great. Now in the editor create a new control setup. Make sure you’re configured to work on the device that you’re planning on playing with / use the most. In my case I’m working with an iPad layout. I prefer to use my iPad in a horizontal orientation most of the time for show control. I’m also only working with one page for now, and happy to leave it as named “1”. You’ll notice for now that the box next to auto for OSC is checked – we’ll leave that for now. Alright, now we’re going to do something that most of us have never done.

In your empty control panel, right click and add one of every different kind of object. Yep. Add one of each so we can look at how they all work. You might choose to skip the repetition of vertical and horizontal sliders – that’s a-okay, but make sure you pick one of them. By the end you should have something that looks like this:

TouchOSC_Editor_-_Untitled_1_touchosc

That’s certainly not sexy, but it’s going to teach you more than you can imagine. Next upload that interface to your device of choice. If you’ve never uploaded a new interface to TouchOSC you’ll need to know the IP address of your computer. If you’re using a windows machine you can use the command prompt and ipconfig to find this information, on a mac use the network pane in System Preferences. Next you’ll want to transfer this layout to your device, Helxer has a wonderful piece of documentation about do get this done, and you can read it here.

Next take a moment to read through what all of those lovely widgets do and how they talk to your programming environment. After that the real fun starts. Start listening to your network with your programming environment of choice, and look at what kinds of messages you get from all of these different kinds of buttons and sliders.

Isadora

If you’re working with Troikatronix Isadora you can start to see the signal flow coming from your layout by first going to the communications drop down menu, and then selecting “Stream Setup.”

Communications_and_Menubar

Next you’ll want to select “auto detect input.”

Menubar_and_Untitled

Now you can start moving sliders and toggling buttons so you can record their address. Make sure that you select “Renumber Ports” and click “OK” on the bottom of the page.

Menubar_and_Untitled

Now you’re ready to use the OSC listener actor to use those inputs to drive any number of elements inside of your scene. You’ll notice that the “channel number” on the listener actor corresponds to the port number on the Stream-Setup dialog.

Menubar_and_Untitled

 

Now you’re off to the races. Do something exciting, or not, but play with all of the buttons, sliders, and gizmos so you know how they all work. For extra credit, remember that you can also send messages back to your layout. To do this you need to use the OSC Transmit actor. You’ll also need to know the IP address of your device, and the port that you’re transmitting to – you can find all of this information on the same page in TouchOSC where you initially set your target IP address.

Quartz Composer

If you’re working with Apple’s Quartz Composer you have to do a little more work to get your OSC stream up and running. For starters, you’ll need to know a little bit more about your OSC messages. You’ll notice in the TouchOSC editor that each object has an address that’s associated with it. For example, my fader is ” /1/fader1 “. To Touch OSC this reads as Fader 1 on Page 1 of your layout. In order to read information from this slider in Quartz, we’ll need to know this address, along with the type of information that we’re sending. In the case of a slider we’re sending a float (a number with fractional parts). To get started let’s open up a new Quartz Composer patch, and add an “OSC Receiver” from the library.

Menubar_and_Untitled_-_Editor

Now let’s take a closer look at that node. If we look at the inspector, and at the settings page we can see lots of useful information.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

Here we can see what port we’re set to receive information on by default, as well as an example of how we need to format the key’s from our layout so Quartz can properly receive the signals coming over the network. Let’s set up our fader to be properly decoded by Quartz. First we’ll need to remove the test key. Next we’ll want to add a new key by typing in the dialog box /1/fader1 and designating this as a float. Finally, click the plus sign to add this key the receiver. I usually add something else to my Quartz patch to make sure that I’m passing values through my network, this is optional.

Menubar_and_OSC_Receiver_and_Untitled_-_Editor

There you go. Now, to add the other buttons and sliders from your layout, you’ll need to similarly add keys for each of the buttons, sliders, and gizmos. Remember that you can find this information in your TouchOSC editor for this layout:

TouchOSC_Editor_-_Untitled_1_touchosc

Now you’re cooking with gas. Experiment with your layout, and how you can use this buttons and sliders to drive your Quartz Composer creations. For extra credit remember that you can also transmit back to your device. In this case you’ll need to use the “OSC Sender” object in Quartz. You’ll need to know the IP address of your target device, as well as the port that you’re communicating on. Have fun, and making something interesting… or if nothing else, learn something along the way.

TouchDesigner

If you’re working with Derivative’s TouchDesigner you have two options for receiving OSC information – CHOPs or DATS. Both of these methods work well but might have different uses for different situations.

Screenshot_032514_083023_PMLet’s start by looking at the OSC in CHOP. First we need to specify what port we want to listen to. Take a look at the TouchOSC layout to make sure the ports and addresses match. Then start moving sliders and buttons to see them appear in TouchDesigner.

Screenshot_032514_083345_PM

To use these to drive various parts of your network you can either use a select CHOP or reference these channels with expressions.

Next let’s look at what we see coming from the OSC In DAT.

Screenshot_032514_083545_PMHere instead of a changing single we see table with a message header and then with our float values coming in as static messages. Depending on the circumstance one or the other of these methods is going to help you drive your network.

For extra credit use the OSC out Chop to push values back to your TouchOSC layout – remember that you’ll need the IP address of your device, the port you’re receiving messages on, and you’ll need to format the header of the message correctly so that it corresponds to the right slider or button. Have fun learning and playing with your layout of all of the different kinds of controls.

WonderDome | Workshop Weekend 1

WonderDome

In 2012 Dan Fine started talking to me about a project he was putting together for his MFA thesis. A fully immersive dome theatre environment for families and young audiences. The space would feature a dome for immersive projection, a sensor system for tracking performers and audience members, all built on a framework of affordable components. While some of the details of this project have changed, the ideas have stayed the same – an immersive environment that erases boundaries between the performer and the audience, in a space that can be fully activated with media – a space that is also watching those inside of it.

Fast forward a year, and in mid October of 2013 the team of designers and our performer had our first workshop weekend where we began to get some of our initial concepts up on their feet. Leading up to the workshop we assembled a 16 foot diameter test dome where we could try out some of our ideas. While the project itself has an architecture team that’s working on an portable structure, we wanted a space that roughly approximated the kind of environment we were going to be working in. This test dome will house our first iteration of projection, lighting, and sound builds, as well as the preliminary sensor system.

Both Dan and Adam have spent countless hours exploring various dome structures, their costs, and their ease of assembly. Their research ultimately landed the team on using a kit from ZipTie Domes for our test structure. ZipTie Domes has a wide variety of options for structures and kits. With a 16 foot diameter dome to build we opted to only purchase the hub pieces for this structure, and to cut and prep the struts ourselves – saving us the costs of ordering and shipping this material.

In a weekend and change we were able to prep all of the materials and assemble our structure. Once assembled we were faced with the challenge of how to skin it for our tests. In our discussion about how to cover the structure we eventually settled on using a parachute for our first tests. While this material is far from our ideal surface for our final iteration, we wanted something affordable and large enough to cover our whole dome. After a bit of searching around on the net, Dan was able to locate a local military base that had parachutes past their use period that we were able to have for free. Our only hiccup here was that the parachute was multi colored. After some paint testing we settled on treating the whole fabric with some light gray latex paint. With our dome assembled, skinned, and painted we were nearly ready for our workshop weekend.

Media

There’s healthy body of research and methodology for dome projection on the web, and while reading about a challenge prepped the team for what we were about to face it wasn’t until we go some projections up and running that we began to realize what we were really up against. Our test projectors are InFocus 3118 HD machines that are great. There are not, however, great when it comes to dome projection. One of our first realizations in getting some media up on the surface of the dome was the importance of short throw lensing. Our three HD projectors at a 16 foot distance produced a beautifully bright image, but covered less of our surface than we had hoped. That said, our three projectors gave us a perfect test environment to begin thinking about warping and edge blending in our media.

TouchDesigner

One of the discussions we’ve had in this process has been about what system is going to drive the media inside of the WonderDome. One of the most critical elements to the media team in this regard is the ability to drop in content that the system is then able to warp and edge blend dynamically. One of the challenges in the forefront of our discussions about live performance has been the importance of a flexible media system that simplifies as many challenges as possible for the designer. Traditional methods of warping and edge blending are well established practices, but their implementation often lives in the media artifact itself, meaning that the media must be rendered in a manner that is distorted in order to compensate for the surface that it will be projected onto. This method requires that the designer both build the content, and build the distortion / blending methods. One of the obstacles we’d like to overcome in this project is to build a drag and drop system that allows the designer to focus on crafting the content itself, knowing that the system will do some of the heavy lifting of distortion and blending. To solve that problem, one of the pieces of software that we were test driving as a development platform is Derivative’s TouchDesigner.

Out of the workshop weekend we were able to play both with rendering 3D models with virtual cameras as outputs, as well as with manually placing and adjusting a render on our surface. The flexibility and responsiveness of TouchDesigner as a development environment made this process relatively fast and easy. It also meant that we had a chance to see lots of different kinds of content styles (realistic images, animation, 3D rendered puppets, etc.) in the actual space. Hugely important was a discovery about the impact of movement (especially fast movement) coming from a screen that fills your entire field of view.

TouchOSC Remote

Another hugely important discovery was the implementation of a remote triggering mechanism. One of our other team members, Alex Oliszewski, and I spent a good chunk of our time talking about the implementation of a media system for the dome. As we talked through our goals for the weekend it quickly became apparent that we needed for him to have some remote control of the system from inside of the dome, while I was outside programming and making larger scale changes. The use of TouchOSC and Open Sound Control made a huge difference for us as we worked through various types of media in the system. Our quick implementation gave Alex the ability to move forward and backwards through a media stack, zoom, and translate content in the space. This allowed him the flexibility to sit away from a programming window to see his work. As a designer who rarely gets to see a production without a monitor in front of me, this was a huge step forward. The importance of having some freedom from the screen can’t be understated, and it was thrilling to have something so quickly accessible.

Lights

Adam Vachon, our lighting designer, also made some wonderful discoveries over the course of the weekend. Adam has a vested interest in interactive lighting, and to this end he’s also working in TouchDesigner to develop a cue based lighting console that can use dynamic input from sensors to drive his system. While this is a huge challenge, it’s also very exciting to see him tackling this. In many ways it really feels like he’s doing some exciting new work that addresses very real issues for theaters and performers who don’t have access to high end lighting systems. (You can see some of the progress Adam is making on his blog here)

Broad Strokes

While it’s still early in our process it’s exciting to see so many of the ideas that we’ve had take shape. It can be difficult to see a project for what it’s going to be while a team is mired in the work of grants, legal, and organization. Now that we’re starting to really get our hands dirty, the fun (and hard) work feels like it’s going to start to come fast and furiously.


Thoughts from the Participants:

From Adam Vachon

What challenges did you find that you expected?

The tracking; I knew it would hard, and it has proven to be even more so. While a simple proof-of-concept test was completed with a Kinect, a blob tracking camera may not be accurate enough to reliably track the same target continuously. More research is showing that Ultra Wide Band RFID Real Time Locations System may be the answer, but such systems are expensive. That said, I am now in communications with a rep/developer for TiMax Tracker (an UWB RFID RTLS) who might be able to help us out. Fingers crossed!

What challenges did you find that you didn’t expect?

The computers! Just getting the some of computers to work the way they were “supposed” to was a headache! That said, it is nothing more than what I should have expected in the first place. Note for the future: always test the computers before workshop weekend!

DMX addressing might also become a problem with TouchDesigner, though I need to do some more investigation on that.

How do you plan to overcome some of these challenges?

Bootcamping my macbook pro will help on the short term computer-wise, but it is definitely a final solution. I will hopefully be obtaining a “permanent” test light within the next two weeks as well, making it easier to do physical tests within the Dome.

As for TouchDesigner, more playing around, forum trolling, and attending Mary Franck’s workshop at the LDI institute in January.

What excites you the most about WonderDome?

I get a really exciting opportunity: working to develop a super flexible, super communicative lighting control system with interactivity in mind. What does that mean exactly? Live tracking of performers and audience members, and giving away some control to the audience. An idea that is becoming more an more to me as an artist is finding new ways for the audience to directly interact with a piece of art. On our current touch-all-the-screens-and-watch-magic-happen culture, interactive and immersive performance is one way for an audience to have a more meaningful experience at the theatre.

 

From Julie Rada

What challenges did you find that you expected?

From the performer’s perspective, I expected to wait around. One thing I have learned in working with media is to have patience. During the workshop, I knew things would be rough anyway and I was there primarily as a body in space – as proof of concept. I expected this and didn’t really find it to be a challenge but as I am trying to internally catalogue what resources or skills I am utilizing in this process, so far one of the major ones is patience. And I expect that to continue.

I expected there to be conflicts between media and lights (not the departments, the design elements themselves). There were challenge, of course, but they were significant enough to necessitate a fundamental change to the structure. That part was unexpected…

Lastly, directing audience attention in an immersive space I knew would be a challenge, mostly due to the fundamental shape of the space and audience relationship. Working with such limitations for media and lights is extremely difficult in regard to cutting the performer’s body out from the background imagery and the need to raise the performer up.

What challenges did you find that you didn’t expect?

Honestly, the issue of occlusion on all sides had not occurred to me. Of course it is obvious, but I have been thinking very abstractly about the dome (as opposed to pragmatically). I think that is my performer’s privilege: I don’t have to implement any of the technical aspects and therefore, I am a bit naive about the inherent obstacles therein.

I did not expect to feel so shy about speaking up about problem solving ideas. I was actually kind of nervous about suggesting my “rain fly” idea about the dome because I felt like 1) I had been out of the conversation for some time and I didn’t know what had already been covered and 2) every single person in the room at the time has more technical know-how than I do. I tend to be relatively savvy with how things function but I am way out of my league with this group. I was really conscious of not wanting to waste everyone’s time with my kindergarten talk if indeed that’s what it was (it wasn’t…phew!). I didn’t expect to feel insecure about this kind of communication.

How do you plan to overcome some of these challenges?

Um. Tenacity?

What excites you the most about WonderDome?

It was a bit of a revelation to think of WonderDome as a new performance platform and, indeed, it is. It is quite unique. I think working with it concretely made that more clear to me than ever before. It is exciting to be in dialogue on something that feels so original. I feel privileged to be able to contribute, and not just as a performer, but with my mind and ideas.

Notes about performer skills:

Soft skills: knowing that it isn’t about you, patience, sense of humor
Practical skills: puppeteering, possibly the ability to run some cues from a handheld device

Sending and Receiving OSC Values with TouchDesigner

The other day I posted look at how to send and receive TouchOSC data with Troikatronix’s Isadora. In an effort to diversify my programming background I’m often curious about how to translate a given task from one programing environment to another. To this end, I was curious about repeating this same action in Derivative’s TouchDesigner. If you’re one of a growing number of artists interested in visual programming, VJing, media design, interactive system design, media installation, or media for live performance than it’s well worth your time to look at both Isadora and TouchDesigner.

That said, I initially started thinking about control panel interfaces for TouchOSC when I saw Graham Thorne’s instructional videos about how to transmit data from Isadora to Touch OSC (If you have a chance take a look at his posts here: Part 1 & Part 2). Last year I used TouchOSC accelerometer data to drive an installation at Bragg’s Pie Factory in Downtown Phoenix, but I didn’t do much with actual control panels. Receiving data from TouchOSC in TouchDesigner is a piece of cake, but sending it back out again didn’t work the first time that I tried it. To be fair, I did only spend about 3 minutes trying to solve this particular question as it was a rather fleeting thought while working on another project.

Getting this to working Isadora, however, gave me a renewed sense of focus and determination, so with a little bit of experimenting I finally got this working the way that I wanted. While there are any number of things you might do with TouchDesigner and TouchOSC, I wanted to start with a simple deliverable: I wanted the position of Slider 1 to inversely change the position of Slider 2, and vise versa. In this way, moving Slider 1 up moves Slider 2 down, and moving Slider 2 up moves Slider 1 down. In performance setting it’s unlikely that I’d need something this simple, but for the sake of testing an idea this seemed like it would give me the kind of information that I might need.

For the most part this is pretty easy, but there are a few things to watch for in order to get the results that you want.

Here’s the network stream of Chanel Operators (CHOPs):

OSC In – Select – Math – Rename – OSC Out

OSC In

Derivative Documentation on the OSC In CHOP 

The OSC in CHOP is pretty excellent out the gate. We need to do a little configuring to get this up and running, but after that we’re truly off to the races. First off it’s important to make sure that your TouchOSC device is connected to your network and broadcasting to your TouchDesigner machine. If this process is new to you it’s worth taking a little time to read about out to set this all up – start on this page if this is new to you. Once your TouchOSC device is talking to your TouchDesigner network we’re ready to start making magic happen. One of the first things that I did was to activate (just adjust up or down) the two faders that I wanted to work with. This made sure that their IDs were sent to TouchDesigner, making it much easier for me to determine how I was going to rename in them in the future.

Screenshot_100913_094345_PMSelect

Derivative Documentation on the Select CHOP

When you’re working with the OSC In CHOP, all of your TouchOSC data comes through a single pipe. One of the first things to do here is to use the select CHOP to single out the signal that you’re looking for (as a note, you could also export the Channel in question to a Null, or handle this a variety of other ways – in my case a Select CHOP was just the first method I tried). This operator allows you to pull out a single Channel from your OSC In.

Screenshot_100913_094428_PM

Math

Derivative Documentation on the Math CHOP

Next I wanted to remap the values of my incoming signal to be inverted. A Math CHOP lets us quickly remap the range for our data by changing a few values in the Range Tab of that parameters window. You’ll notice that you first need to specify the from range (what’s the range of incoming values), and then specify the to range (what’s the range of outgoing values). In my case, the from values are 0 and 1, and the to values are 1 and 0.

Screenshot_100913_094528_PM

Rename

Derivative Documentation on the Rename CHOP

The Rename CHOP is one of the most important steps in your whole network. When communicating with TouchOSC you need to know exactly what object you’re trying to drive with your data. To accomplish this you need to know the UDP address of your device, the port number that you’re board casting to, and the name of the slider or button that you’re wanting to change. While we can specify the UPD address and port number with another CHOP, we need to use the Rename CHOP (a quick aside – there are, of course, other ways to do this – this just happens to be the method that I got working after a bit of trial and error) to specify the name of the object that we’re driving. In my case I wanted fader 1 to drive fader 2. You’ll notice that formatting of the fader is important. In this particular OSC layout there are multiple tabs. We can see this as evidenced by the “1/” preceding the name of our fader. In changing the name of this input we need to enter the precise name of the fader that we want to change – this is why we activated both faders initially, this gave us the names of both objects in the TouchOSC panel. You’ll notice that changed the name of my fader to “1/fader2”.

Screenshot_100913_094606_PM

OSC Out

Derivative Documentation on the OSC Out CHOP 

Now that we’ve processed and named our signal we’re ready to send it back out to TouchOSC. In this CHOP we need to input the UDP address of the TouchOSC device (you can find this in the info tab of the TouchOSC app) as well as the port that we’re sending to. We also want to make sure that we’re sending this as a sample, and finally we want to turn off “send events every cook.” This last step is important because it ensures that we’re only sending values when they change. If we send messages for every cook we won’t be able to see the inverse relationship between our two sliders. Correction, we’ll be able to see the relationship so long as we don’t set up slider 2 to drive slider 1. In order to create an inverse relationship with both sliders we only want to transmit data as it’s changed (otherwise we’ll find that we’re fighting with the constantly streamed position data from our slider that isn’t being activated).

Screenshot_100913_094705_PM

The last step in this process is to copy the whole string and set up fader 2 to transmit to fader 1, this last step allows both faders to drive one another in turn.

There you have it. You’ve now successfully configured your TouchDesigner network to receive and send data to a mobile device running TouchOSC.