Category Archives: Programming

TouchDesigner | The Big Bad Ass Lister | Part 1/3

The Big Bad Ass Lister

This one is a different in part because we’ll be looking at something from the TouchDesigner pallet in depth, and also because Ivan from Derivative has very graciously volunteered to help demystify one of the coolest components that you’re not using enough in your projects – Lister. 

There’s a few things to keep in mind as you read through this post – there are two authors: Matthew and Ivan. They talked through what pieces would be great to share about lister, and then split up the examples and writing – that means that you’ll notice a slightly different voice in the writing, and you’ll get to see two different perspectives on how lister works. It’s also worth noting that Ivan is the developer at Derivative who has spent the most time working on lister, and the person who knows the Dark Dank Secrets of this component. There are also a handful of examples that go with everything written here, and we’re hoping that this post helps you find new and exciting ways to use lister in your projects!

Lister Basics

What’s a lister anyways? ! Lister is a shared component that both lives in the palette browser as its own comp, and is deeply integrated into the latest widgets UI kit Derivative has been cooking up. You can find both of these in the Palette browser on the on left hand side of the TouchDesigner UI, or by pulling up the palette browser with ctrl + l (win) / cmd + l (mac). 

Lister from the UI Folder in the Palette
Lister from Basic Widgets

Lister is actually built on top of the List COMP. The List COMP is a wildly powerful component, and replaces some of the pieces that you used to do with the Table COMP. If you haven’t used the Table COMP before, it’s amazing… but gets a little unwieldy pretty quickly. The Table COMP can be used to make radio buttons or other list style interfaces (the op create dialogue is a good example of the Table COMP in action), but it’s all set up with Table DATs, and can be a little hard to read / understand.

The list COMP

The List COMP does many the same things, but unlike the Table COMP, the List COMP has a deep set of python integrations with an internal callback DAT that allows you to think about UI building and operation from a Pythonic perspective rather than with Table DATs. That’s very exciting, but to make a truly reusable and extensible UI component you have to write a lot of Python… and this is where Lister comes into play. Lister is built on top of the List COMP and solves a large number of challenges you might typically encounter. It still has all the raw power of the List COMP, but gives you a huge headstart over trying to create a list UI element from the ground up.

So what can you do with Lister? Anything you might use a list for!

With that said, let’s actually look at how we can get started with lister, and some of the ways we can start to style and configure a lister for use in your projects.

Configuration

When it comes to working with lister, the config COMP is where you’ll actually find all of the most important secrets. The config COMP is docked below lister, and has a few easy to access helpers. On lister you’ll find a handful of pulse open parameters that will make configuration easier to set-up. 

Before we can really dig-in let’s set-up our workspace so we can see some of the interesting pieces of lister in action. To begin we’ll start with a Table DAT that holds a set of rows and columns that we want to turn into a UI element. 

We can start by creating a Table DAT that has two columns and a few rows. I’m going to give mine 6 rows, and name the columns “col1” and “col2”.

A Simple Table DAT

Let’s also add a lister from the palette – in this case we want the lister that’s in the UI Folder.

lister from the palette

To see just how fast lister can be to use, let’s start by just connecting our table DAT to our lister COMP:

lister out of the box

Right away we’ve already got a table up and running. This is a nice start, but we can do a little more to better configure our lister UI element. Some of the most important configuration pieces can be found in the docked base called “listerConfig”:

listerConfig

The listerConfig Base holds the sweet secrets for our lister – let’s take a closer look at how we might set a few configuration elements. Before we dive inside of the config, let’s first set a few of the parameters on lister.

On lister let’s first turn off the Auto-define Columns parameter:

Change pars on Lister COMP

Let’s also change the parameter “Input Table Has Headers” to True – since our input table has headers:

Table has header pars

Like it’s parameter name sake, Auto-define columns will attempt to set-up your lister based on the input table. For this example, we’re going to turn off that parameter, and then click on the “Edit Column Definitions” pulse parameter. This will open a floating window of the colDefine table DAT inside of the listerConfig:

colDefine Table

This table contains many of the configuration elements for our lister’s look. We can see that the table already has some elements set up. We’re going to replace these with our own values so we can see how lister will change. 

First let’s set the column name to be displayed as our header. Our starting table contains fruits in col1 and their quantities in col2 – So I want to change the column Row to be “Fruit” and “Quantity” respectively. 

Our lister’s data structure supports an internal column name that’s different from the display label – in some advanced cases you may want to use this. For now, let’s just set the columnLabel row to have a value of “*” which we can read as – “use the value from the column row.” 

Next let’s change the sourceData row to be the name of the input table’s columns that we’d like to use. My input table had columns called “col1” and “col2” – what’s important to consider here is that our lister’s columns don’t have to be arranged in the same order as our input table. This can be very helpful for more complex list UI elements. 

Let’s set our sourceDataMode to be “string” for both of our columns.

For now let’s style the width and stretch parameters for the column to be the same. Each column can have a width value of 100, and a stretch value of 0. At this point you should have a config table that looks something like this:

finished colDefine Table

Your lister should look something like this:

lister with Updated colDefine Table

Before we experiment with styling our lister anymore, let’s quickly take a look at some of the handy features that already exist. For starters we can see that there’s a hover highlight from our mouse position, and when we click on a row it highlights another color. 

lister with Highlighted rows

Lister also has two DAT outputs. The first lister DAT output is a table DAT version of exactly what’s in lister. The second output is the header columns from our lister, along with the selected row or rows from our lister:

lister output DATs

Let’s next look at how we might style our lister’s output a little more. 

In addition to the column definitions table, there are a host of additional operators we can change inside of our listerConfig DAT that are worth playing with. Let’s head back to our lister’s parameters, and this time click on the pulse parameter called “Edit Config COMP”.

Edit Config COMP

This will open a floating network window that looks into the listerConfig base:

listerConfig COMP

In addition to the column definitions table that we’ve been working with, there’s also a whole cluster of other TOPs. The first two columns of TOPs hold text TOPs that are used to define how our lister is styled. 

By changing these TOPs we in turn change the style for the lister’s columns. Let’s take a quick look at what we might change and the impact that will have on our lister. 

Let’s start on the text TOP called “header” on the font page. We can see that a number of the parameters are set to read-only – but there are a few on this page of parameters that we can change. Let’s start by turning the font x size up to 12 and turning on the “Bold” parameter.

changing the lister Header font size and bold attributes

Next let’s move to the Color page, and change the background color for our header. Here I’m going to make the Font Color bright white, change teh background color to a plumb, and set the bottom border of my TOP to be Border A. You’ll want to make sure that you also set the Border A Alpha parameter to 1 – it’s set to 0 by default. The result should look something like this:

changing the lister Header color attirbutes

Next if we look at our lister we’ll see those changes in action:

results from changes in listerConfig

This same idea also works with our rows – we just have to edit the first column of TOPs in our liserConfig base. I’m going to change my TOPs so that my lister uses this plumb / purple color palette. The only changes I have to make will be to the first column of TOPs:

styling lister rows

The resulting look for the whole lister is something like this:

styled lister

If you wanted to also change the row heights, you can achieve that change by adjusting the header TOP’s vertical resolution, and the master TOP’s vertical resolution. I’m going to change the header TOP’s resolution to 36, and the master TOP’s to 30. I’m also going to turn on row Striping on the lister COMP. The resulting changes look like this:

lister with height adjustments

This is just a start of the kind of customization we can achieve with lister. Now that we have a handle on some of the essential elements for manipulating the look of our lister, let’s take a deeper dive into customization and other lister features.

Auto Config and Copy Auto-Cols

For a simple input table this is a pretty straightforward approach, but sometimes we end up with a table that’s slightly more complicated. A good example of this might be when we’re converting SOP data to a table – this can be helpful in all sorts of situations, but setting up our lister for this many columns could take a  minute. Let’s set up a quick table that holds data converted from a sphere SOP.

We can start with a sphere SOP and change its primitive type to Polygon (this will start us off with fewer rows). Let’s add a null SOP (in case we want to add any other transformations), and then use a SOP to DAT to convert this in a table format. 

SOP to DAT

Similar to Houdnini’s geometry spreadsheet, this table holds all sorts of useful information about our SOP. This often works great for internal project uses, but if we want to create a styled interface for this data we would have to think about how to display this information. 

Geometry Spreadsheet

A lister here can be a great help. Let’s add a new lister form the palette to get started. Next we can connect our DAT to lister.

lister from Geometry Spreadsheet

Lister starts with the “Auto Define Cols” parameter on, which helps us get a starting point from our data right away. If, however, we want to style our lister a little more we’d need to repeat the steps we practiced earlier – going though and creating a column in our colDefine table for each column that we want to display. That’s all well and good, but it’d be oh so nice if there was a way to copy the current auto-config format into our colDefine table to give us a headstart. This gives us an opportunity to look at one of the handiest parameters on lister – Copy Auto-Cols to Config. 

For a table with this many columns it might be a little cumbersome to go through and set all of the colDeinfe elements manually – which makes this parameter incredibly helpful. So let’s go through an setup this lister so we can see the “Copy auto Cols To Config” in action. First, I want to turn on the parameter “Input Table Has Headers.” This SOP to DAT has header information already, and I want to make sure that the first row is treated as a header.

Input Table has Headers

Now we can use the pulse parameter called “Copy Auto Cols to Config.” A good trick to keep in mind here is that pulsing the “Recreate Auto-columns” parameter will adjust column widths to their contents when you press it. This would be helpful when you’re looking to have your columns auto size themselves. 

For this example we’re going to make some changes to our columns after we’ve captured them in the colDefine table. Let’s pulse the “Copy Auto Cols to Config” parameter. After we pulse this parameter, let’s turn off the “Auto-define Columns” parameter so we can make some changes to our lister.

Setting our Auto-define pars

Now let’s use the pulse parameter on lister to open our Edit Column Definitions table. We should see that lister has done all of the hard work of setting up our column definitions table, and now we can focus on just modifying the contents of this table.

The initial Auto-Defined columns

Now that we have the initial work done let’s go through and make some edits to our definitions. For starters, we might consider changing the labels for our columns. The names that come from our SOP to DAT are great when working in TouchDesigner, but aren’t always as descriptive as we might like for a user interface. We can update what’s displayed in the headers by making some changes to the first row of our colDefine DAT. I’m going to change a few names:

  • Index -> Point Index
  • P(0) -> Point.x
  • P(1) -> Point.y
  • P(2 -> Point.z
  • N(0) -> Normal.x
  • N(1) -> Normal.y
  • N(2) -> Normal.z

I’m also going to remove the point weight column, and the groups column. I also want the columns to stretch to fill the lister’s width, so let’s change the stretch row to be 1 for each of the columns. I want the Point Index values to be center justified instead of left justified, so we can change the justify cell in column 1 to be CENTER. Finally, I want the text in the first column to be Bold, so let’s change the fontBold cell to be 1. The resulting table should look like this:

Our adjusted auto defined cols

While we’re here practice using the font TOPs inside of the listerConfig COMP to style your lister. I’m going to adjust my row heights, and add a little color. After a few adjustments, my listerConfig looks like this:

listerConfig updates

And the final lister looks like this:

Geometry spreadsheet lister output

In our next installment we’re going to look at additional mechanics for working with tables, creating buttons in lister, using simple callback functions, and deeper styling with TOP paths. 

You can download all of the examples from this post here.

Happy programming… and lister-ing.

TouchDesigner | The Object CHOP

The Object CHOP has long been one of the most challenging CHOPs for me to really wrap my head around. Following along with some conversations on the Facebook Help Group 1, it’s clear that I’m not the only one who has bumped their head against how to take advantage of this operator.

With that in mind, here are a few tricks and techniques that you might find helpful when working with the object CHOP.

Distance Between Many Objects Part 1

At first glance, it seems like the object CHOP can only perform calculations between single objects, but in fact you can use this operator to perform calculations between many objects provided that you format the input data correctly, and set up your object CHOP to account for multiple samples.

In a first example let’s say that we want to find the distance between several green spheres and a blue box:

First let’s collect our position information. I’ve used an object CHOP per sphere to find its distance, but you might also use a script CHOP, or a put positions in a table that you reference for the spheres, or drive them with custom parameters. How you position them doesn’t matter. What we need, however, is a single CHOP with three channels that hold the transformation information of those spheres. My trick in this network is to use object CHOPs to find their positions, then put them in sequence with a join CHOP:

Next we can use a single object CHOP that’s fed reference positions from this join CHOP, and a target Geometry COMP:

Other important pieces here are the start and end parameters on the channel page.

This is where we set how many samples the object CHOP will evaluate. This can be a bit confusing – here especially as the join CHOP has started at a sample index of 1 rather than 0. The devil is in the details, so it’s worth keeping a close eye for these kinds of oddities. Because of this we compensate in our start position by moving back one sample index.

Next make sure to set your object CHOP to output measurements, and distance. What you’ll then end up with is a single channel with a sample for each distance between your box and spheres. We can convert this to a table if we wanted to see the actual values:

Distance Between Many Objects Part 2

We may also want to measure distances between multiple blue boxes. Say, for example, that we had two different blue boxes and we wanted to know the distances of our spheres to both of those boxes?

Similar to our first exercise we’ll start by collecting all of the position information for our spheres. We also need position information for our boxes. In this case, however, we need to stretch a single sample to be 4 samples long – this is part of our data preparation step to ensure we correctly calculate distance.

Here a simple stretch CHOP has been used to make sure we have four samples of data for each box. Next we can join this data so all of our box position information is in a single set of CHOP channels:

Before moving on, we need to take a moment to adjust our sphere position data. In our first example we only collected the four positions… we need to set up the correct extend behavior for this series so that our CHOPs know what values to use when CHOPs of mismatched lengths are combined. We can use an extend CHOP set to cycle to do this trick:

Finally, we can then use an object CHOP to calculate the distance between our box and our spheres:

Distance Between Many Objects plus Bearing

If we also calculate the bearing between our boxes and spheres, we’ll end up with rotation information… what can we do with this? We could use this to calculate the correct rotation for a set of instances. For example:

Here each line is correctly rotated, scaled, and placed based on calculations from the object CHOP.

Bearing

You can also use the object CHOP to just calculate bearing – or rotation from one object to another. Here you can see how this might be used to rotate instances to sit flat on a sphere’s surface, or rotate an arrow to point towards an object:

Bearing and Distance

Or you might use the combination of bearing and distance to make some strange abstract art:

Collision

You can also use the object CHOP to simulate a kind of collision calculation where the distance you’re measuring can help you tell how close an object is to another and if they’re on top of one another:

GitHub

Clone the Repo to Follow Along

TouchDesigner | Packing up a Tox for Distribution

If you’ve been following along with the workshop materials from the TD Summit 2019, there is one more exciting little tid-bit to dig-into. Thinking about how to create a reusable template for creating toxes is no small feat – especially if you want to include external Python libraries. I’ve been thinking about how to approach this challenge and mapped out a rough framework for approaching this challenge. You can see the working repo for this up on github here. During the last part of the workshop on External Python Libraries we covered this a little sneak peak of this technique, and to complete the workshop videos we added a section covering this approach.

The big picture ideas here are to standardize our approach to handling external libraries and automate their installation. This leans on the same concepts we explored in the workshop, and includes using things like a requirements file, using some automated installation scripts, and making sure the path to our project directory is included in our sys.path. The last major change is to convert our general scripts we wrote previously into an extension.

These days I tends to lean towards Extensions over Modules, though you could approach this challenge with Modules very similarly. This set of tutorials will walk us through taking what we created in the previous workshop videos and creating a portable tox we should be able to drag-and-drop into any project. Hopefully, what you take away from this example are some ideas about how to apply this technique and approach to other toxes that you build and share.

Happy Programming!


TouchDesigner | TD Summit 2019 | External Python Libraries

Overview

Hot off the presses, the workshop video from the 2019 TouchDesigner Summit workshop on External Python Libraries is up. You can follow along with all of the materials and outline from the workshop here – External Python Libraries.

If you just want to jump straight to the videos, you can find a playlist below. Happy Programming!


Python has more and more reach these days – from web services to internet of things objects, scientific and statistical analysis of data, what you can do with Python is ever expanding. While it’s possible to do all of this work from the ground up, it’s often easier (and faster) to use libraries that other people have published. TouchDesigner already comes with a few extra libraries included like OpenCV and Numpy. Once you have a handle on working with Python the world feels like it’s your oyster… but how you work with a magical little external library in TouchDesigner can be very tricksy. Worse yet, if you happen to get it working on your machine, making work on another can be infuriating. Over the course of this workshop we’ll take a look at what you can do to make this process as smooth and painless as possible, as well as some considerations and practices that will help you stay sane when you’re trouble shooting this wild Python roller coaster.


TouchDesigner | TD Summit 2019 | Modular Architectures


Hot off the presses, the workshop video from the 2019 TouchDesigner Summit workshop on Modular Architectures is up. You can follow along with all of the materials and outline from the workshop here – Modular Architectures.

If you just want to jump straight to the videos, you can find a playlist below. Happy Programming!


Overview

How you design, plan, and extend a system is a both an engineering and creative endeavor. The choices you make will have lasting impact in the way you work, what your system can do, what it can’t do, and how easy it will be to maintain and adapt for future work. A little planning and thoughtful organization will help make sure that you’re able to focus on the work that’s the most exciting rather than always remaking all of the same pieces. Take it from someone who has built a lot of systems from the ground up – a good foundation keeps you excited and interested in your work, rather than always repeating the same mistakes.

Over the course of this workshop we’ll take a look a fundamental concepts for this kind of work – What is externalization, why do it in the first place, and how much is too much. We’ll also look at some perspectives about organization that come from building large projects, what it means to build templates / blueprints that you can reuse, how to take advantage of some simple automation, and where you can begin to take advantage of some more advanced ideas.

Major Goals

  • No delay scripts unless there is no other solution
  • Complete control over initialization and load order
  • Text-port confirmation of each step (ultimately you want this to be a log-file)

TouchDesigner | Python and the Subprocess Module

I can has Subprocess?

At the TouchDesigner Summit in Montreal we’ll be taking some time to talk about working with external Python Modules in TouchDesigner. While we’ll have time to cover lots of information about how to incorporate external modules in Touch, we won’t have a lot of time to talk through the wobbles that you might run into when working with operations that might be slow, or otherwise unwieldy to run in Touch.

“What do you mean Matt?”

The types of pieces that usually fall into this category are blocking operations. For example, let’s say that you want to upload an image to the web somewhere. Many of the libraries that you might find will have an approach that’s probably blocking – as in it will appear as if TouchDesigner has frozen while the whole operation completes. While this is fine outside of Touch, we don’t typically like it when our applications appear to freeze – especially in installations or live performances. You might be able to move that process to another thread, though that might be a little more hassle that you really want it to be in the long run.

Enter the Python Subprocess module.

Subprocess

The subprocess module allows you to run a python script as a parallel execution that doesn’t touch your TouchDesigner application. You could use this for all sorts of interesting an powerful applications – from starting media syncing between machines, running a process that talks to the internet, or any number of solutions that execute outside of TouchDesigner. In the past I’ve used this for things like sending emails, or uploading images to Instagram – there’s lots you can do with this approach, it’s just a matter of wrangling python and the subprocess module.

What exactly is happening when we use the subprocess module?! Well, we can think of this as a situation where we write a python script in a text file, and then ask your operating system to run that file. There are great ways to pass in arguments into those situations, and if you really need data there are ways to get a response before the process quits. This can be a very flexible solution for a number of situations, and worth looking into if you want something that’s non-blocking and can be run outside of TouchDesigner.

Subprocess calls can be infuriating if you’re not familiar with them, so let’s look at some simple anatomy of making this work from Touch.


Scenario 1 – Execute this Script

Let’s start with the most basic of scenarios. Here we have some Python script that normally takes a long time to run, that we just want to kick off from TouchDesigner. For this example let’s just look at something that will print to a shell – nothing fancy, just a place to get our bearings. Our python script might look something like this:

Pure Python

import time
import sys

# a variable a divider that we're going to use
divider = '- ' * 10

# a for loop to print all of the paths in our sys.path
for each in sys.path:
    print(each)

# our divider
print(divider)

# a for loop that prints numbers 
for each in range(10):
    print(each)

# a call to time.sleep to keep our terminal open so we can see what's happening
time.sleep(120)

This works just the way that we might expect if we run it in our OS. But how can we run this script from TouchDesigner?

In TouchDesigner

In TouchDesigner we’d add a DAT that has the following contents. Here we assume that the script above has been saved in a folder called scripts that’s in the same folder as our project file, and the name of the script is cmd_line_python.py.

import subprocess

# point to our script that we're going to execute
cmd_python_script = '{}/scripts/cmd_line_python.py'.format(project.folder)

# quick debug print
print(cmd_python_script)

# call our script with subprocess
subprocess.Popen(['python', cmd_python_script], shell=False)

This is great, but this will actually execute with the version of Python that’s packaged with TouchDesigner. In some cases that’s exactly what we want… in other’s we might want to use a different version of python that’s installed on our OS. How can we do that?


Scenario 2 – Execute this Script with a Specific Python

This is very similar to our first situation, but here we want to run the python that’s installed on our OS, not the python that’s packaged with Touch. In this case we can use the exact same python script we saw above.

Pure Python

import time
import sys

# a variable a divider that we're going to use
divider = '- ' * 10

# a for loop to print all of the paths in our sys.path
for each in sys.path:
    print(each)

# our divider
print(divider)

# a for loop that prints numbers 
for each in range(10):
    print(each)

# a call to time.sleep to keep our terminal open so we can see what's happening
time.sleep(120)

Our real changes come when we’re issuing the subprocess call in TouchDesigner.

In TouchDesigner

Again, we add a DAT that has the following contents. Here we assume that the script above has been saved in a folder called scripts that’s in the same folder as our project file, and the name of the script is cmd_line_python.py. The additional wrinkle this time, is that we also need to specify which python.exe we want to use for this process.

import subprocess

# point to our script that we're going to execute
cmd_python_script = '{}/scripts/cmd_line_python.py'.format(project.folder)

# point to the specific version of python that we want to use
python_exe = 'C:/Program Files/Python35/python.exe'

# quick debug print
print(cmd_python_script)

# call our script with subprocess
subprocess.Popen([python_exe, cmd_python_script], shell=False)

If we look closely at the example above, we can see that we’ve been very specific about where our python_exe lives. This approach runs the same script, only this time with the python.exe that we’ve specifically pointed to.


Scenario 3 – Passing Over Args

There are several ways to approach this challenge. One that will line up with the format of many pure pythonic approaches here would be to use the argparse library. We find this in lots of stand-alone scripts, and it allows us the flexibility of setting default arguments, and creating some relatively clean inputs when calling a script form the command line. In this approach we set up a function that we’ll call, and pass arguments into – in the example below that’s our My_python_method(). By using ArgumentParser we can pass in command line arguments that we can in turn pass through to our function. Notice the syntax at the bottom to see how this works. ArgumentParser returns keyword arguments, which is why we use kwargs as the mechanism for sending args into our simple for loop.

Pure Python

import time
import sys
import time
from argparse import ArgumentParser

def My_python_method(kwargs):

    disp_str = 'key: {} | value: {} | type: {}'
    for each_key, each_value in kwargs.items():
        formatted_str = disp_str.format(each_key, each_value, type(each_value))
        print(formatted_str)

    # keep the shell open so we can debug
    time.sleep(int(kwargs.get('delay')))

# execution order matters -this puppy has to be at the bottom as our functions are defined above
if __name__ == '__main__':
    parser = ArgumentParser(description='A simple argument input example')
    parser.add_argument("-i", "--input", dest="in", help="an input string", required=True)
    parser.add_argument("-i2", "--input2", dest="in2", help="another input", required=True)    
    parser.add_argument("-d", "--delay", dest="delay", help="how long our terminal stays up", required=False, default=10)
    
    args = parser.parse_args()
    My_python_method(vars(args))
    pass

# example
# python .\cmd_line_python_args.py -i="a string" -i2="another string" -d=15

Where this becomes more interesting is when we look at what’s happening on the TouchDesigner side of this equation.

In TouchDesigner

A DAT in TouchDesigner needs to follow the same rules we established so far – we need to know what executable we’re using, which file we’re running, and finally we now need to send along some arguments that will be passed to that file. In Touch, our script this time should look something like this:

import subprocess

# point to our script that we're going to execute
cmd_python_script = '{}/scripts/cmd_line_python_args.py'.format(project.folder)

# construct a list of arguments for out external script
script_args = ['-i', 'Hello', '-i2', 'TouchDesigner']

# join our python instructions with our scirpt args
command_list = ['python', cmd_python_script] + script_args

# call our script with subprocess
subprocess.Popen(command_list, shell=False)

Scenario 4 – I Want a Message Back

Like all things Touch, and all things Python there are LOTS of ways to accomplish this task. One way we might want to consider, however, is using UDP messages. Touch happens to have a handy UDPIn DAT that’s ready to accept messages, and the other benefit here is that we could potentially target another machine on our network as the target for these messages. For this first exploration let’s imagine that we’re only sending a message locally, and that all of our variables are defined in the python script we’re running. We’ll need to use the socket library to help with the communication elements, and you’ll notice that we import that at the top of our script. This silly example just creates a UDP connection, and sends messages at a regular interval.

Pure Python

import time
import sys
import socket

upd_ip = "127.0.0.1"
udp_port = 7000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

num_iters = 11
sleep_interval = 2

def msg_to_bytes(msg):
    return msg.encode('utf-8')

starting_msg = "Sending messages at an interval of {} seconds".format(sleep_interval)
sock.sendto(msg_to_bytes(starting_msg), (upd_ip, udp_port))

for each in range(num_iters):
    msg = "{} of {}".format(each, num_iters-1)
    sock.sendto(msg_to_bytes(msg), (upd_ip, udp_port))
    time.sleep(sleep_interval)

ending_msg = "All messages sent"
sock.sendto(msg_to_bytes(ending_msg), (upd_ip, udp_port))

In TouchDesigner

For this simple execution in TouchDesigner we only need to worry about kicking off the script. That looks almost exactly like the other pieces we’ve set up so far.

import subprocess

# point to our script that we're going to execute
cmd_python_script = '{}/scripts/cmd_line_python_udp_msg.py'.format(project.folder)

# print our script path - quick debug
print(cmd_python_script)

# clear the last entries from the UDPin DAT
op('udpin1').par.clear.pulse()

# call our script with subprocess
subprocess.Popen(['python', cmd_python_script], shell=True)

Our catch this time is that we need to use a UDPIn DAT to receive those messages. Let’s also make sure the Row/Callback Format parameter is set to One Per Message. With this all set up we should see something like this when we kick off the script.


Scenario 5 – Messages, I can has args?!

That all seems mighty fine… but, what happens when I want to combine what we’ve done with passing along arguments, and messages? I’m so glad you asked. With a little extra work we can make exactly that happen. We do need to do a little more heavy lifting on the python front, but that work gives us some extra flexibility. You’ll notice below that we’re now passing along which port we want to use, how many iterations of our for loop, and the interval between repetitions.

Pure Python

import time
import sys
import socket
from argparse import ArgumentParser

def msg_to_bytes(msg):
    return msg.encode('utf-8')

def msg_loop(port, interval, loop):
    # localhost
    upd_ip = "127.0.0.1"

    # set udp port with input val and initialize socket connection
    udp_port = int(port)
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    # set additional variables with input vals
    num_iters = int(loop)
    sleep_interval = int(interval)

    # send message that we're starting
    starting_msg = "Sending messages at an interval of {} seconds".format(sleep_interval)
    sock.sendto(msg_to_bytes(starting_msg), (upd_ip, udp_port))

    # run message loop
    for each in range(num_iters):
        msg = "{} of {}".format(each, num_iters-1)
        sock.sendto(msg_to_bytes(msg), (upd_ip, udp_port))
        time.sleep(sleep_interval)

    # send message that we're ending
    ending_msg = "All messages sent"
    sock.sendto(msg_to_bytes(ending_msg), (upd_ip, udp_port))

# execution order matters - this puppy has to be at the bottom as our functions are defined above
if __name__ == '__main__':
    parser = ArgumentParser(description='A simple UDP example')
    parser.add_argument("-p", "--port", dest="port", help="UDP port", required=True, default=1234)
    parser.add_argument("-i", "--interval", dest="interval", help="loop interval", required=True, default=5)
    parser.add_argument("-l", "--loop", dest="loop", help="number of repetitions", required=True, default=10)
    args = parser.parse_args()

    msg_loop(args.port, args.interval, args.loop)
    pass

# example
# python .\cmd_line_python_udp_msg_args.py -p=5000 -i=2 -l=10

In TouchDesigner

Over in TouchDesigner, our subprocess script looks very similar to what we’ve done so far with just a few modifications.

import subprocess

# set up our variables for our subprocess call
port = str(op('udpin2').par.port.val)
interval = '1'
loop = '15'

# point to our script that we're going to execute
cmd_python_script = '{}/scripts/cmd_line_python_udp_msg_args.py'.format(project.folder)

# construct a list of our python args - which python and which script
python_args = ['python', cmd_python_script]

# construct a list of arguments for out external script
script_args = ['-p', port, '-i', interval, '-l', loop]

# join our two lists - python args and scirpt args
cmd_args = python_args + script_args

# quick debug print
print(cmd_args)

# clear the last entries from the UDPin DAT
op('udpin2').par.clear.pulse()

# call our script with subprocess
subprocess.Popen(cmd_args, shell=True)

Here the resulting messages look very similar, only we’ve not gotten to specify all the qualities about the for loop from our script.


What does this Matter?

Well, there are lots of things you might do with this, but especially interesting might be considering how you can use otherwise very costly and slow operations in Python with this approach. For example, a recent set of Style Transfer experiments I was working on used this style of approach to essentially create a TouchDesigner front end / UI for a pytorch style transfer backend. This let me pass along arguments from the UI over to a pure python execution that didn’t block or freeze touch while it was running. There are lots of ways you might get into mischief with this kind of work, and it’s worth pointing out that while all this is focused on python, there’s no reason you couldn’t instead think of other applications you want to run with a particular file and a set of command line arguments.

Happy Programming!


What to follow along? Download the sample code from github

TouchDesigner | Virtual MIDI Devices

In the past week I’ve found myself in multiple conversations about working with MIDI in TouchDesigner. Specifically, how you might pass a MIDI signal from one application to another on the same computer. So far, all of the conversations I’ve been involved in have been related to working with Windows – which means that this particular post is focused on that operating system, though the same idea should transfer to macOS if that’s your current platform.

So what’s the problem exactly?

All of my conversations about this topic have been centered around sending messages from TouchDesigner to WATCHOUT. For those unfamiliar, WATCHOUT is a media server application that’s used in Theatrical and live events contexts for controlling or sequencing media playback. It’s in the same family of tools as Disguise, GreenHippo, and Pandora’s Box. You may well be asking – “why not just use OSC?” That’s a great question, some versions of WATCHOUT don’t support OSC, though they do support MIDI. While this is the specific challenge, the more general idea that we can push against here is:

How can I locally (on the same machine) test a MIDI signal flow without adding extra hardware?

To get there we need to add in a virtual device to allow us to loopback. At it’s simplest, a loopback is just a way of capturing a signal that you’d other wise be sending somewhere else. Imagine using a cable from the headphone port on your laptop, and connecting it to the microphone in on your laptop (don’t actually do that, just imagine it). Why? Well, using actual interfaces often have different requirements, or set-up than just faking it – you’ll also likely find that operating systems don’t have an implicit knowledge of a particular signal or transfer format. Just because your computer is making noises doesn’t automatically mean that it can capture that noise locally.

How do we make this work then?

Enter the Virtual Device. A Virtual Device provides drivers that act as though it’s sending your signal to a dedicated piece of hardware, but instead allows you to rout it back to your machine. There are lots of examples of this in the audio world, and it just so happens that there’s a Virtual MIDI device we can install to pass around MIDI signals as well.

Doing a bit of googling I found LoobBe1. It has a free evaluation license, which is what we’ll use below. If you decide that this needs to be part of your commercial workflow, I’d encourage you purchase a license. It’s less than $15 US, and supporting folks that make useful tools is something we should all do more. As a disclaimer, I don’t have any affiliation or connection to LoopBe1 – I just found them after doing some internet searching.

We’re going to set-up a simple loopback test for a MIDI signal from TouchDesigner, and see if we can read that signal back. I don’t have a copy of WATCHOUT to work with, but if we can trouble shoot within TouchDesigner, we should at least know that it’s working.

Getting a MIDI Loopback Working

Ingredients

Windows 10 v1809
TouchDesigner 099 2019.15230
LoobBe1

To get started you’ll need to download and install LoobBe1. I like to limit what starts up automatically on my computer, so I’d recommend also turning off the automatic start-up of this application once you have it installed.

From there we can start up TouchDesigner and begin our configuration process. We’re going to start by opening the MIDI Mapper dialogue. You can open that with alt + d or you can find it from the Dialogues drop down:

Open the MIDI Mapper dialogue from the Dialogs drop-down

You should now see the MIDI Mapper:

The MIDI Mapper Dialog window

We’re going to test our output right in TouchDesigner. To do that let’s click on the Create New Mapping button in the middle left of the dialogue window. That will add a row to our Device Mapping where we can specify that both our In and Out Device will be LoopBe Internal MIDI.

The MIDI Mapper with an added Device

Before we can move forward here it’s important to recognize that the naming of MIDI channels matters. MIDI (Music Instrument Digital Interface) has a detailed specification that ensures continuity across devices and manufactures, so it shouldn’t be a surprise that there’s lots of detail here. You can read the whole spec by here, or get a summary from Wikipedia. What we really care about today is that we need to format our TouchDesigner channel names in a way that’s compatible with Derivative’s implementation. To figure all of that out, we need to head over to the wiki for a quick read. Let’s take a look at the MIDI Out CHOP page. Here we see that:

Naming the CHOP channels: Channels are mapped to events by their name. Events like notes, controllers and velocities must be followed by the note/controller number (n65, c7). If the number is left off a note event, the note number is the value of the channel. Other events, which are sent to the entire channel, do not need a trailing number (pc, pw). The channel prefix can be used to identify the MIDI channel the event should be sent on (i.e. “ch1n45” assigns that TouchDesigner channel to note 45 messages on MIDI channel 1). Channels can always be renamed with a Rename CHOP before entering the MIDI Out CHOP.

The MIDI Out CHOP sends MIDI velocity as well. The values of the channels entering the MIDI Out CHOPs are sent as the velocity of the note. If Normalize is “None”, the channel needs to be 0 to 127. If Normalize is “0 to 1”, channel values between 0 and 1 are scaled to be MIDI 0 to 127.

The “Cook Every Frame” option cooks the CHOP every frame, even if the CHOP isn’t being displayed. All Volume Off and All Volume On flags are new and emit events for Controller 7 of all 16 channels. MIDI output go in a separate thread to allow output that slows TouchDesigner less. It now works in Time Slice mode for note events and controller events. (Not for Program Change or Sysex messages yet) Note channels only trigger anew Note On when the input channel goes from 0 or less to a value greater than zero. Similar for Note Off events.The channel name determines how it is interpreted.

Derivative’s MIDI Out CHOP Wiki Page

What does that even mean?! Well, let’s set up a few pieces in a network to see if we can make sense of this.

I’m going to add a Noise CHOP to my network, and turn on the time-slice parameter so we have some constantly changing values. I’m going to change the Channel name parameter to be ch1c1. This would be Channel 1, Control 1.

By default our Noise CHOP will have values between -1 and 1. Based on what’s in the wiki, we’ll either need to normalize our values (change them to be in a range of 0-1) or we’ll need to change our values to be in a range of 0-127. We can do either of these with a Math CHOP so let’s add one of those in line and change our range parameters:

Now we can finish this off by adding a MIDI Out CHOP to our network:

So how do we make sure this is working?

Well, let’s first check with Touch. Since we added our LoopBe Internal MIDI as our input device, we should now be able to drop in a MIDI In CHOP to see our midi values:

Great – so we can see our values… only they’re expressed as whole numbers rather than as a floating point value. Why? MIDI works with 127 steps, so our MIDI Out CHOP is converting our 0-1 set of values into a range of 0-127. What this means is that you, dear developer-artist, have to make a choice. This approach works perfectly fine, but it may mean that at some point you’ll be doing some mental gymnastics in understanding how values in Touch correspond to values in your other application. If that’s too much to think about all the time, then you may want to change the math CHOP to be in a range of 0-127, with the values rounded either up or down (depending on your preference). This kind of change would let you see exactly what value you’re expected to be transmitted from Touch. If you’re working on a high profile AV installation, this kind of specificity might be worth having – if only so you can say with complete confidence that you’re transmitting a value of x on channelxyz.

Okay, so it works in Touch… but can we be sure? Of course. I’m going to use Isadora by Troikatronix to help us validate that we’re sending this correctly. I’m not going to go into too many of the details for how we set-this up in Izzy, just enough so we can validate what we’re seeing here.

To start I’m going to open up Isadora, go to the Communications drop down, and select MIDI Setup:

Like Touch, I’m going to set the Input Port to be LoopBe Internal MIDI:

To see if we’re actually transmitting values, let’s open up the Izzy Status monitor:

Sure enough, we should see a blinking Green light on Channel 1 and the value associated with it:

TADA! You’re passing data around locally from one application to another using MIDI.

Takeaways

Okay, so what exactly did we do? Well, we added a virtual MIDI interface that let us send MIDI encoded data between applications on a single computer. We then verified that this works by first looking at the data transmission in Touch, then further validated that this works working by looking outside of Touch with another application.

Why on earth do any of this? Well, that’s really a question for you. If MIDI is a core part of your work flow, and passing data between applications on the same computer has an important role for you, then the answers might seem obvious. For those still scratching their heads, imagine that you have raw sensor data that’s not MIDI friendly, but you’d like to get it into an application that does speak MIDI. This kind of workflow is great for that. Working with a Kinect but want to turn that data into MIDI, easy. What about a Leap – sure. Can I do my mouse – you betcha.

It might not be what you need today, but maybe it’ll inspire you to think about some mischief you could get yourself into.

Happy Programming.

Guest Post with Elburz Sorkhabi | Two TouchDesigner Beginner Tricks

If you’re reading this, you’re probably a fan of Elburz Sorkhabi and his work. That makes two of us! Elburz and I are always trying to find ways to collaborate and get into trouble, but we’re often on other sides of the world from one another. We thought it would be fun to do some guest blog posts for each other’s readers. So today you’re in for a treat with a guest post from Elburz himself! If you like a little variety in life, give Elburz blog a read over at Elburz.io.


TouchDesigner is a feature-rich software. It can be daunting for new users what they should be learning and what they’re missing in their toolbox. I thought it would be nice to share two TouchDesigner tricks that are easy to learn and will provide you a lot of value over your career. Some of these things can even evade experienced users as they come quickly in new updates and over time it can be hard to keep track of them all. With that said, let’s dive in!

Custom parameters

Custom parameters are one of the best features to come to TouchDesigner in the last few years. I use them all the time. Sometimes I use them to wrap complex functionality inside of a component while providing easy to use controls. Otherwise I use them for more architectural elements of a project, such as creating internal APIs. The great thing is that any kind of parameter type already available in TouchDesigner can be used for your own custom components. One thing to note is that you can only add Custom Parameters to COMP operators. The first step is to make a COMP, which usually will be a Container COMP or Base COMP, and then right click on it and select Customize Component…

This will open the Component Editor. In this window you can do things like make new extensions, and more importantly for us, this is a visual way to create Custom Parameters. The next step is type a name in to the top-left string field that will be used to name the new parameter page. Then go ahead and click Add Page. I use names like Settings or Controls for my parameter pages. You can confirm everything worked by checking the parameters of your COMP and looking for your new parameter page. It’ll be blank for now.

The next step is to start adding parameters to our new parameter page. Click on the parameter page you just created on the left side of the Component Editor. Now you can enter a name for the parameter you’re about to create in the second string field. Then we’ll go ahead and use the drop down menu to choose the type of parameter you’d like to create. Like I mentioned earlier, you can create any of the existing types of parameters including pulse buttons, toggles, colour pickers, file/folder selectors, and more. The drop down menu to the right of the parameter type has numbers from 1 to 4. These numbers represent the amount of value parts for parameters such as floats and integers. Parts can be thought of as the “amount” of values, for example if you make an integer with 3 selected, you’ll get an integer parameter with 3 separate integers, similar to an RGB parameter field. For this example, let’s select Toggle, name it Toggle Button, and click Add Par. You’ll immediately see the parameter appear in your parameter window as well as the component editor.

That’s it! That’s all there is to making custom parameters. The process is the same for any type of parameter, you just have to choose what you want from the drop down menu. How to use the values is our final step here, but it’s also quite easy.
The quickest way to access the parameter values of a COMP are to use the Parameter CHOP. When you drop one inside of the COMP, by default it’ll already be setup to show you only the values of your custom parameters. For most parameter types, it isn’t any more complicated than using these CHOP channels as you would any other channels for referencing.

For the parameters that hold non-numeric data, such as the file/folder selectors or string fields, you’ll need to access the data through some simple Python scripting. In this case, we can place a Parameter DAT right next to our Parameter CHOP. In the OP parameter enter .. which will select the parent container, for the Parameters enter *, and finally turn off the Built-In button. The Parameter DAT has callbacks like many of the other Execute type of DATs in TouchDesigner. This means that it has different functions where you add your code based on triggers. To access our same toggle button via the Parameter DAT we could add this under the onValueChange() callback:

if par.name == "Togglebutton":
    print(par.val)

This would parse the different customer parameters by name (par.name), find the toggle parameter by it’s scripting name (Togglename – the name you see when you click on a parameter and see it’s second name in the expanded area), and then prints it’s value (val).

Operator snippets

I’m still surprised to this day by how Operator Snippets isn’t talked about every day by new users. It’s the most helpful resource added that can help you learn how to use just about any operator. Operator Snippets is a project file that is built into TouchDesigner that contains tons and tons of examples of how to use operators. It’s similar to Max MSP’s examples per node.
There are two ways to access them. The first is to click Help in the top menu and then select Operator Snippets. This will open up a new window. The left side of the screen looks similar to the OP Create Dialogue (the menu when you double click on the network background) and allows you to choose which operator you want to find examples for. Underneath the operator selection area are a handful of big buttons with different names. Click on one of these changes between the different examples for the operator you’ve selected. On the right hand side of the window is the network area with the example in action.
The great thing about Operator Snippets is that the example is a live network running in real-time! You can copy the example, paste it into your project, and then tweak it to your needs. How useful it that?

Wrap up

These two tricks may sound simple but they are game changing. Being able to make your own custom parameters quickly and easily is game changing. You can make complex components that can be easily used by anyone or implemented into your own projects and controlled easily. Operator Snippets give you the greatest documentation you could ever ask for: live networks running in real-time. I bet if you spent a little bit of time just browsing the snippets, you’d find awesome little examples you’ll be eager to copy and paste into your next project. With that said, enjoy these two beginner tricks and happy programming! And if you’re interested in more content like this, check out elburz.io

TouchDesigner | New Features | Bindings

Spring time is lots of things – flowers, holidays, vigorous allergies, and the TouchDesigner Spring Update. For the second year running this is the time of year that features graduate from just being in experimental to being full fledged stable release features. Wowza.

This spring we’re seeing a feature that’s flat out amazing, and likely a bit of a sleeper. Bindings. Elburz has a great shout out to bindings on a recent blog, and I wanted to take some time to dig in and step through an example of both why they’re important, the paradigm they’re built on, and why they matter.

So what are parameter bindings anyway? The Derivative wiki has a great segment describing bindings:

Bound parameters keep their values in sync and will respond to changes from either parameter. For each parameter that is a bind master, it will have one or more bind reference parameters.
The bind master holds the current value. It can have exports and expressions and generally works like any other parameter, but its value can be also changed indirectly by the bind references.
A bind reference holds the location of the bind master. A bind reference is in a fourth “Bind” Parameter Mode, and will show up as purple text in parameter dialogs. It can only be changed via its its bind master, its UI, and its val property.

Derivative Wiki article on Binding

Model View Controller – MVC

Errm. Okay, so what does that mean exactly? Bindings are based on an interface architecture paradigm called Model – View – Controller, or MVC. Wikipedia has a nice starter debrief on the idea for us, but it’s easy to understand if you’ve spent much time working in Touch either for live set, or for a client application. Let’s consider a live set to help us get our footing here. Suppose you have parameter that can be updated by multiple touch points both in your own UI and in the TD UI. We all know this game, as soon as you export a CHOP to a parameter, you can no longer change that parameter except through the exported CHOP.

Fine.

So maybe instead of exporting from a single CHOP you instead write a bunch of scripts to handle this operation – only now you’re in a real pickle. Why? Well, because your script changes the parameter, but doesn’t update all of the UI elements that reflect the state of that parameter. So you write another script to update the UI. But now you’ve managed to save your project in a state where the UI and the parameters are not aligned. As soon as you change the UI everything is in sync – so you save over a few things, commit your changes, and now everything should be great. Until you need to load a saved preset state from disk. Now you’ve gotta write another set of scripts to do all of that updating, or hunker down for a more generalized solution – which probably means a code refactor. Who wanted to make some more sweet visuals anyway? There goes your night off. There goes your margin. Sigh.

The real world example of this is light switches. If you’ve ever lived in an apartment where multiple light switches control the same light / outlet, you understand this issue intimately. How do you know if the light is on or off? Only by looking at the light, because once the states of the light switches are out of phase they perform the opposite action.

The Model – View – Controller paradigm is a design architecture that’s intended to help resolve this issue. The MVC approach decouples the control of a UI element from the data it is manipulating. You could do this in touch before, you were just on the hook for doing all the set-up. This probably meant you had a master table or storage dictionary somewhere that was updated whenever a parameter was changed, that in turn would update all the other touch points. That’s a huge hassle, but it was the only way to solve this problem. It’s also the kind of silly thing you could really have a strangely strong opinion about – and consequently be convinced that your collaborators were doing it all wrong.

Enter Bindings

Okay, so as a refresher, Bindings are a new parameter mode – that’s the little multi-colored set of dots next to any parameter. This new mode is purple – one of the many colors of awesome. At the end of all of this, we’ll take a peek at the new widgets – and the UI redesign they offer, but to get started let’s build a use case for bindings so we can get a sense of what they’re good for.

Slider

We’re going to start with a good old fashioned slider. Why a slider?! Well, this is the kind of parameter we end up using all the time, and the fundamental nature of this UI piece should be foundational enough that if we can get a handle on this one, the jump to more abstract ideas should be a little easier.

Let’s get started by first adding a slider from the Op Create dialogue. We’re just going to add a run of the mill slider for now.

adding a slider

From here we’re going to customize our slider – let’s add a page called “Settings” and then a float custom parameter that we call “Slider” for now.

customizing our slider

So far so good. The next step is where it’s gonna get a little weird, and where it’ll get different than before. From here, let’s dive into our slider. We’re going to delete our Panel CHOP, and add our own Constant CHOP.

changing slider internals

Okay. Now here’s the wild part. On our new Constant CHOP we’ll use the new bindings parameter mode to write parent().par.Slider – that’s the reference to our newly created custom parameter.

write our binding expressing directly

If you’re not into that whole writing expressions exercise, you can also do this with a little drag-and-drop action:

drag and drop binding

Okay… so why is this interesting. Well, let’s see what happens when we move our new custom parameter, or move our Constant CHOP:

binding in action

Slick. Okay. That’s fly. So if we update the our parameter or our CHOP both changes are reflected in the other operator. Now let’s make a few final changes so that when we interact with the sliders panel we update both of these values. We can do this with a Panel Execute DAT. Let’s add our DAT and modify the contents with the following:

def onValueChange(panelValue):
     parent().par.Slider = panelValue
     return
adding our panel execute DAT

We should now see that if we move the slider that our Slider parameter updates, and our Constant CHOP updates.

panel and parameter binding

We’re very close now. All we need to do is to update the knob component in our slider. We need to change the expression there to be:

parent().par.Slider*parent().width-me.par.panelw/2
updated panel script

There we have it. Now we can change our custom parameter, our Constant CHOP, or the slider and all three stay in sync. MAGIC.

binding all around

Widgets?

Early on I mentioned that we find this same behavior in Widgets. What exactly are widgets you ask – currently Widgets are rolling out as a huge overhaul to the TUIK interface building kit that was relatively ubiquitous in TouchDesigner networks. Widgets are more modern take on UI building, and offer significant advances to UI building approaches for Touch. There’s too much to cover about widgets here, but it’s worth pointing out that the same binding approach you see above is a fundamental element of the widget system. It allows for bidirectional control of both user interaction elements and parameters – the core principle we just explored. You can dig in and learn a little more about widgets by reading through the article on the Derivative Wiki.

Why Bind…

You might be looking at this and feeling like it’s outside of your wheelhouse, or your workflow.

A reasonable reflection.

Regardless, I’d encourage you to think about the times when you’ve wanted to both control from more than one location – the parameter itself, as well as some other control interface. If nothing else, give them a try to see where they might fit – if they’re no good for you, don’t use them… though I suspect you’ll find they have all sorts of exciting uses.

Happy Programming!

TouchDesigner | Case Study | Custom Parameters and Cues

I recently had the good fortune of being able to collaborate with my partner, Zoe Sandoval, on their MFA thesis project at UC Santa Cruz – { 𝚛𝚎𝚖𝚗𝚊𝚗𝚝𝚜 } 𝚘𝚏 𝚊 { 𝚛𝚒𝚝𝚞𝚊𝚕 } Thesis work is strange, and even the best of us who have managed countless projects will struggle to find balance when our own work is on the line – there is always the temptation to add more, do more, extend the piece a little further, or add another facet for the curious investigator. Zoe had an enormous lift in front of them, and I wanted to help streamline some of the pieces that already had functioning approaches, but would have benefited from some additional attention and optimization. Specifically, how cues and states operated was an especially important area of focus. I worked closely with the artist to capture their needs around building cues / states and translate that into a straightforward approach that had room to grow as we made discoveries, and needed to iterate during the last weeks leading up to opening.

The Big Picture

{ remnants } of a { ritual } is an immersive installation comprised of projection, lighting, sound, and tangible media. Built largely with TouchDesigner, the installation required a coordinated approach for holistically transforming the space with discrete looks. The projection system included four channels of video (two walls, and a blended floor image); lighting involved one overhead DMX controlled instrument (driven by an ENTEC USB Pro), and four IoT Phillips Hue lights (driven by network commands – you can find a reusable approach on github); sound was comprised of two channels driven by another machine running QLab, which required network commands sent as OSC. The states of each of these end points, the duration of the transition, and the duration of the cue were all elements that needed to both be recorded, and recalled to create a seamless environmental experience.

Below we’re going to step though some of the larger considerations that led to the solution that was finally used for this installation, before we get there though it’s helpful to have a larger picture of what we actually needed to make. Here’s a quick run-down of some of the big ideas:

  • a way to convert a set of parameters to python dictionary – using custom parameters rather than building a custom UI is a fast way to create a standardized set of controls without the complexity of lots of UI building in Touch.
  • a reusable way to use storage in TouchDesigner to have a local copy of the parameters for fast reference – typically in these situations we want fast access to our stored data, and that largely looks like python storage; more than just dumping pieces into storage, we want to make sure that we’re thoughtfully managing a data structure that has a considered and generalized approach.
  • a way to write those collections of parameters to file – JSON in this case. This ensures that our preset data doesn’t live in our toe file and is more easily transportable or editable without having TouchDesigner open. Saving cues to file means that we don’t have to save the project when we make changes, and it also means that we have a portable version of our states / cues. This has lots of valuable applications, and is generally something you end up wanting in lots of situations.
  • a way to read those JSON files and put their values back into storage – it’s one thing to write these values to file, but it’s equally important to have a way to get the contents of our file back into storage.
  • a way to set the parameters on a COMP with the data structure we’ve been moving around – it’s great that we’ve captured all of these parameters, but what do we do with this data once we’ve captured it? The need here is thinking through what to do with that data once you have it captured.

Cuing needs

One of the most challenging, and most important steps in the process of considering a cuing system is to identify the granularity and scope of your intended control. To this end, I worked closely with the artist to both understand their design intentions, as well their needed degrees of control. For example, the composition of the projection meant that the blended floor projection was treated as a single input image source; similarly, the walls were a single image that spanned multiple projectors. In these cases, rather than thinking of managing all four projectors it was sufficient to only think in terms of the whole compositions that were being pulled. In addition to the images, it was important to the artist to be able to control the opacity of the images (in the case of creating a fade-in / out) as well as some image adjustments (black level, brightness, contrast, HSV Offset). Lighting, and sound had their own sets of controls – and we also needed to capture a name for the cue that was easily identifiable.

As lovely as it would be to suggest that we knew all of these control handles ahead of time, the truth is that we discovered which controls were necessary through a series of iterative passes – each time adding or removing controls that were either necessary or too granular. Another temptation in these instances is to imagine that you’ll be able to figure out your cuing / control needs on your feet – while that may be the case in some situations, it’s tremendously valuable to instead do a bit of planning about what you’ll want to control or adjust. You certainly can’t predict everything, and it’s a fool’s errand to imagine that you’re going to use a waterfall model for these kinds of projects. A more reasonable approach is to make a plan, test your ideas, make adjustments, test, rinse, repeat. An agile approach emphasizes smaller incremental changes that accumulate over time – this requires a little more patience, and a willingness to refactor more frequently, but has huge advantages when wrestling with complex ideas.

Custom Pars

In the past I would have set myself to the task of handling all of these controls in custom built UI elements – if was was creating an interface for a client and had sufficient time to address all of the UI / UX elements I might have taken that approach here, but since there was considerable time pressure it was instead easier (and faster) to think about working with custom parameters. Built in operators have their own set of parameters, and Touch now allows users to customize Component operators with all of the same parameters you find on other ops. This customization technique can be used to build control handles that might otherwise not need complete UI elements, and can be further extended by using the Parameter COMP – effectively creating a UI element out of the work you’ve already done while specifying the custom parameters. The other worthwhile consideration to call out here is your ability to essentially copy parameters from other objects. Consider the black level, contrast, and brightness pars included above. One approach would be to create each par individually, and set their min, max, and default values. It would, however, be much faster if we could just copy the attributes from the existing level TOP. Luckily we can do just that with a small trick.

We start by creating a base Comp (or any Component object), right clicking on the operator, and selecting customize component.

This opens the customize component dialogue where we can make alterations to our COMP. Start by adding a new page to your COMP and notice how this now shows up on the components parameters pages:

For now let’s also add a level TOP so we can see how this works. From your level TOP click and drag a parameter over to the customize component dialogue – dragging specifically to the Parameter column on the customize component dialogue:

This process faithfully captures the source parameter’s attributes – type, min, max, and default vals without you needing to input them manually. In this style of workflow the approach is to first start by building your operator networks so you know what ops you will want to control. Rather than designing the UI and later adding operator chain, you instead start with the operator chain, and only expose the parameters you’ll need / want to control. In this process you may find that you need more or fewer control handles, and this style of working easily accommodates that kind of workflow.

Capturing Pars

Creating a component with all of your parameters is part of the battle, how to meaningfully extract those values is another kettle of fish. When possible it’s always handy to take advantage of the features of a programming language or environment. In this case, I wanted to do two things – first I wanted to be able to stash cues locally in the project for fast retrieval, second I wanted to have a way to write those cues to disk so they were’t embedded in a toe or tox file. I like JSON as a data format for these kinds of pieces, and the Python equivalent to JSON is dictionaries. Fast in TD access means storage. So here we have an outline for our big picture ideas – capture the custom parameters, stash them locally in storage, and write them to disk.

One approach here would be to artisanaly capture each parameter in my hipster data structure – and while we could do that, time fighting with these types of ideas has taught me that a more generalized approach will likely be more useful, even if it takes a little longer to get it right. So what does that look like exactly?

To get started, let’s create a simple set of custom parameters on a base COMP. I’m going to use the trick we learned above to move over a handful of parameters form a level TOP: Black Level, Contrast, and Opacity:

To create a dictionary out of these pars I could write something very specific to my approach that might look something like this snippet:

def my_pars_to_dict(p_name):
# create empty par_dict with input name as the preset_name value
par_dict = {
"preset_name" : p_name,
"preset_vals" : {
"Blacklevel" : op('base1').par.Blacklevel.val,
"Contrast" : op('base1').par.Contrast.val,
"Opacity" : op('base1').par.Opacity.val
}
}
return par_dict
# example use
p_name = "Preset001"
presets = my_pars_to_dict(p_name)
print(presets)

At first glance that may seem like an excellent solution, but as time goes on this approach will let us down in a number of ways. I wont’ bother to detail all of them, but it is worth capturing a few of the biggest offenders here. This approach is not easily expanded – if I want to add more pars, I have to add them directly to the method itself. For a handful, this might be fine, but over ten and it will start to get very messy. This approach also requires duplicate work – the key name for values means that I need to manually verify if the key name and parameter name match (we don’t have to do this, but we’ll see later how this saves us a good chunk of work), if I misspell a word here I’ll be very sorry for it later. The scope of this approach is very narrow – very. In this case the target operator variable is set inside of the method, meaning that this approach will only ever work for this operator, at this level in the network. All of that and more largely mean that while I can use this method very narrowly, I can use this approach, but I’m going to be sorry for it in the long run.

Instead of the rather arduous process above, we might consider a more generalized approach to solving this problem. Lucky for us, we can use the pars() method to solve this problem. The pars() method is very handy for this kind of work, the major catch being that pars() will return all of the parameters on a given object. That’s all well and good, but what I really wanted here was to capture only custom parameters on a specific page, and to be able to ignore some parameters (I didn’t, for example, need / want to capture the “save cue” parameter). What might his kind of approach look like, let’s take a look at the snippet below.

Abstract Reusable code segment

def page_to_dict(target_op, target_page, p_name, ignore_list):
'''
A reusable method for capturing parameters on a single page of a COMP
Args
—————
target_op (TouchDesigner COMP):
> a ToughDesigner COMP that has custom parameters you would like to convert
> into a python dictionary.
target_page (str):
> the string name of the page whose parameters you would like to
> convert into a python dictionary.
p_name (str):
> a name for the preset / cue.
ignore_list (list):
> a list of parameters you do not want to include.
Returns
—————
par_dict (dict)
> a dictionary containing a preset name and dictionary of parameters.
'''
# create empty par_dict with input name as the preset_name value
par_dict = {
"preset_name" : p_name,
"preset_vals" : {}
}
# loop through each parameter in the target_op and capture its name and
# value only if its custom page matches the input string for target_page,
# and the pars are not on the ignore_list
for each_par in target_op.pars():
if each_par.isCustom and each_par.page == target_page and each_par.name not in ignore_list:
par_dict["preset_vals"][each_par.name] = each_par.val
return par_dict
# example use
target_op = op('base_preset_builder')
page = "Remnants Presets"
p_name = target_op.par.Presetname.val
ignore_list = ["Presetname", "Recordcue"]
presets = page_to_dict(target_op, page, p_name, ignore_list)
view raw page_to_dict.py hosted with ❤ by GitHub

What exactly is happening here? First off, this one is full of documentation so our future selves will know what’s happening – in fact this is probably more docstring than code. The big picture idea is rather than thinking about this problem one parameter at a time, we instead what to think of entire custom pages of parameters. Chances are we want to re-use this, so it’s been made to be fairly general – we pass in an operator, the name of the page we want to convert to a python dictionary, the name of our newly made preset, and a list of any parameters we might want to skip over. Once we pass all of those pieces into our function, what we get back is a dictionary full of those parameters.

Capture to Storage

Simply converting the page of parameters to a dictionary doesn’t do much for us – while it is a very neat trick, it’s really about what we do with these values once we have them in a data structure. In our case, I want to put them into storage. Why storage? We certainly could put these values into a table – though there are some gotchas there. Table values in TouchDesigner are always stored as strings – we might think of this as text. That matters because computers are notoriously picky about data, and find the challenge of differentiating between words, whole numbers, and numbers with decimal values very difficult. Programmers refer to words as strings, whole numbers as integers or ints, and numbers with decimal values as floats. Keeping all of our parameter values in a table DAT means they’re all converted to strings. Touch mostly does an excellent job of making this invisible to you, but when it goes wrong it tends to go wrong in ways that can be difficult to debug. Using storage places our values in a python dictionary where our data types are preserved – not converted to strings. If you’re only working with a handful of cues and a handful of parameters this probably doesn’t matter – but if you’re thinking about 20 or more parameters it doesn’t take many cues before working in native data types will make a big difference. For reference, an early iteration of the cuing system for this would have needed the equivalent of a table DAT with over 1000 rows to accommodate the stored parameters. These things add up quickly, more quickly than you first imagine that they might.

Okay, so what’s an example of a simple and reusable function we might use to get a dictionary into storage:

def dict_to_storage(target_op, storage_dict_key, preset_name, dict_to_store):
'''
A reusable method for capturing parameters on a single page of a COMP
Args
—————
target_op (TouchDesigner COMP):
> a ToughDesigner COMP that has custom parameters you would like to convert
> into a python dictionary.
storage_dict_key (str):
> the string name of the storage dictionary you'd like to add
> your preset / cue to.
preset_name (str):
> a name for the preset / cue.
dict_to_store (dict):
> a python dictionary to put into storage.
Returns
—————
None
'''
# grab the dictionary from storage
all_presets = target_op.fetch(storage_dict_key)
# create a new entry
all_presets[preset_name] = dict_to_store
# put dictionary back into storage
target_op.store(storage_dict_key, all_presets)
# example use
target_op = op('base_example')
storage_dict_key = 'presets'
preset_name = 'look001'
dict_to_store = {"preset_values": {
'Blacklevel' : 0.0,
'Contrast' : 1.0,
'Opacity' : 1.0
}
}
dict_to_storage(target_op, storage_dict_key, preset_name, dict_to_store)
view raw dict_to_storage.py hosted with ❤ by GitHub

Write to file

Similar to the above, we likely want a simple way to write our stored cues to disk in the same format we’re using internally. Python dictionaries and JSON are nearly interchangeable data structures and for our needs we can think of them as being the same thing. We do need to import the JSON module to get this to work correctly, but otherwise this is a straightforward function to write.

What you end up with will look like this:

import json
def load_store_json(target_file, storage_op, target_key, storage_name):
'''
A Helper function that reads JSON from disk
Args
—————
target_file (file path):
> a path to a .json file on disk. This is where the file will
> be read from.
storage_op (TouchDesigner operator):
> the target operator where we will store the dictionary.
target_key (str):
> the string key we want to pull from our JSON file.
storage_name (str):
> the string name we want to use for storage.
Returns
—————
None
'''
# open the json file
json_file = open(target_file, 'r')
# create a dictionary out of our json file
json_dict = json.load(json_file).get(target_key)
# store our dictionary in the target op
storage_op.store(storage_name, json_dict)
# close the file
json_file.close()
# example use
target_file = '{}/cues.json'.format(project.folder)
storage_op = op('base_example')
target_key = 'presets'
storage_name = 'presets'
load_store_json(target_file, storage_op, target_key, storage_name)
view raw load_store_json.py hosted with ❤ by GitHub

Reading from file

We’re close now to having a complete approach for working with cues / states. Our next puzzle piece here would be a way to read our JSON from disk, and replace what we have in storage with the file’s contents. This means that whatever is in the file can be used to replace what we have in storage.

What you end up with here might look like this:

import json
def write_dict_to_json(target_file, dict_to_save):
'''
A Helper function that writes JSON file to disk
Args
—————
target_file (file path):
> a path to a .json file on disk. This is where the file will
> be written.
dict_to_save (dict):
> the dictionary to save as json.
Returns
—————
None
'''
# open the json file
json_file = open(target_file, 'w')
# ensure the format for the json is human readable
pretty_json = json.dumps(dict_to_save, indent=4)
# write the json to file
json_file.write(pretty_json)
# close the file
json_file.close()
# example use
target_file = '{}/cues.json'.format(project.folder)
target_dict = {"presets": op('base_example').fetch('presets')}
write_dict_to_json(target_file, target_dict)

Loading pars – does it work

This part is the most tricky. Here the the big idea is to create a duplicate operator that has all of the same custom parameters in our preset maker. Why? Well, that would mean that all of the parameter names match – so which would make loading parameters significantly easier and more straightforward. The other trick here is to remove any of the ignored pars from our ignore list – thinking back this is to ensure that we don’t use any of the parameters that we don’t want / need outside of recording them. We can start this process by making a copy of our operator that’s being used to capture our parameters and then deleting the pars we don’t need. Next we need to write a little script to handle moving around all of the values. That should look something like this:

def load_preset(preset_name, storage_op, target_op):
'''
A Helper function that writes JSON file to disk
Args
—————
preset_name (str):
> the string name of the preset we want to load
storage_op (TouchDesigner operator):
> the target operator storing the presets.
target_op (TouchDesigner operator):
> the target operator whose values we want to set.
Returns
—————
None
'''
# safety to ensure we have a preset to use
try:
preset_vals = storage_op.fetch("presets")[preset_name]
except:
print("This preset does not exist")
# loop through all pars and set them based on the vals in storage
for each_par, each_val in preset_vals.items():
target_op.pars(each_par)[0].val = each_val
target_op.par.Presetname = preset_name
# example use
target_op = op('base_target')
storage_op = op('base_example')
preset_name = 'look002'
load_preset(preset_name, storage_op, target_op)
view raw load_preset.py hosted with ❤ by GitHub

Making a Module

All of this is a good bit of work, and if you’ve been following along, you probably now have a handful of text DATs doing all of this work. For the sake of keeping tidy, we can instead put all of this into a single DAT that we can use as a python module. Wrapping all of these pieces together will give us something like this:

import json
def page_to_dict(target_op, target_page, p_name, ignore_list):
'''
A reusable method for capturing parameters on a single page of a COMP
Args
—————
target_op (TouchDesigner COMP):
> a ToughDesigner COMP that has custom parameters you would like to convert
> into a python dictionary.
target_page (str):
> the string name of the page whose parameters you would like to
> convert into a python dictionary.
p_name (str):
> a name for the preset / cue.
ignore_list (list):
> a list of parameters you do not want to include.
Returns
—————
par_dict (dict)
> a dictionary containing a preset name and dictionary of parameters.
'''
# create empty par_dict with input name as the preset_name value
par_dict = {
"preset_name" : p_name,
"preset_vals" : {}
}
# loop through each parameter in the target_op and capture its name and
# value only if its custom page matches the input string for target_page,
# and the pars are not on the ignore_list
for each_par in target_op.pars():
if each_par.isCustom and each_par.page == target_page and each_par.name not in ignore_list:
par_dict["preset_vals"][each_par.name] = each_par.val
return par_dict
def dict_to_storage(target_op, storage_dict_key, preset_name, dict_to_store):
'''
A reusable method for capturing parameters on a single page of a COMP
Args
—————
target_op (TouchDesigner COMP):
> a ToughDesigner COMP that has custom parameters you would like to convert
> into a python dictionary.
storage_dict_key (str):
> the string name of the storage dictionary you'd like to add
> your preset / cue to.
preset_name (str):
> a name for the preset / cue.
dict_to_store (dict):
> a python dictionary to put into storage.
Returns
—————
None
'''
# grab the dictionary from storage
all_presets = target_op.fetch(storage_dict_key)
# create a new entry
all_presets[preset_name] = dict_to_store
# put dictionary back into storage
target_op.store(storage_dict_key, all_presets)
def load_store_json(target_file, storage_op, target_key, storage_name):
'''
A Helper function that reads JSON from disk
Args
—————
target_file (file path):
> a path to a .json file on disk. This is where the file will
> be read from.
storage_op (TouchDesigner operator):
> the target operator where we will store the dictionary.
target_key (str):
> the string key we want to pull from our JSON file.
storage_name (str):
> the string name we want to use for storage.
Returns
—————
None
'''
# open the json file
json_file = open(target_file, 'r')
# create a dictionary out of our json file
json_dict = json.load(json_file).get(target_key)
# store our dictionary in the target op
storage_op.store(storage_name, json_dict)
# close the file
json_file.close()
def write_dict_to_json(target_file, dict_to_save):
'''
A Helper function that writes JSON file to disk
Args
—————
target_file (file path):
> a path to a .json file on disk. This is where the file will
> be written.
dict_to_save (dict):
> the dictionary to save as json.
Returns
—————
None
'''
# open the json file
json_file = open(target_file, 'w')
# ensure the format for the json is human readable
pretty_json = json.dumps(dict_to_save, indent=4)
# write the json to file
json_file.write(pretty_json)
# close the file
json_file.close()
def load_preset(preset_name, storage_op, target_op):
# safety to ensure we have a preset to use
try:
preset_vals = storage_op.fetch("presets")[preset_name]
except:
print("This preset does not exist")
# loop through all pars and set them based on the vals in storage
for each_par, each_val in preset_vals.items():
target_op.pars(each_par)[0].val = each_val
target_op.par.Presetname = preset_name
view raw presetsMOD.py hosted with ❤ by GitHub

If you want to see how this works and pull it apart from there you can pull an example TOE file form this repo.

TD JSON – another alternative

There’s another alternative to this approach – which is the new TD JSON elements that are in the TouchDesigner. You can read about them on Derivative’s wiki here. These tools are a promising alternative, and you can do very similar pieces here. In particular we might use something like pageToJSONDict()to do what we’re after. That might look something like this:

import json
# import TouchDesigner JSON modules
TDJ = op.TDModules.mod.TDJSON
# list comprehension to narrow our scope to only the custom page we want
target_page = [page for page in op('base_example').customPages if page == 'Test']
# convert pars into json
json_page_pars = TDJ.pageToJSONDict(target_page[0])
# pretty print our json
pretty_pars = json.dumps(json_page_pars, indent=4)

That’s slick, but what we get back is almost 75 lines worth of JSON. This feels a little overkill to me for what we’re after here – there’s lots of potential here, but it might be a little more than we need for this actual implementation. Maybe not though, depending on how you want to change this process, it might be just perfect.

Safety Rails

There are some pieces missing in the approach above that I ended up including in the actual implementation for the artist – I’m not going dig into some of these pieces, but it is worth calling attention to some of the other elements that were included. The biggest pieces that needed to be addressed were how we handle failure, duplicates, provided confirmation on an operation, or warned the user about possibly unintended operations.

The artist, for example, wanted to both have the UI flash and to get a message printed to the text port when a preset was successfully saved. The artist also wanted to make sure that a preset wasn’t automatically overwritten – instead they wanted to see a pop up message warning that a preset was going to be overwritten, allowing the user confirm or cancel that operation.

That may seem unnecessary for a tool you build for yourself… until it’s 2am and you haven’t slept, or you’re working fast, or there’s a crit in 10 minutes and you want to make one more adjustment, and and and, or or or. Handling these edge cases can not only add piece of mind, but also ensure you keep your project on the straight and narrow.

Additionally, how you handle failure in these situations is also important to plan – we never want these pieces to fail, but having a gracefully solution for how to handle these moments are tremendously important to both work through and plan. If nothing else, it’s elegantly handling the failure and printing a message – better still is if give yourself a clue about what went wrong. A few breadcrumbs can go a long way towards helping you find the trail you lost. In my opinion, failing code gets a bad wrap – it’s something we grumble over, not something we celebrate. The truth of the matter, however, is that failures are how we make projects better. Being able to identify where things went wrong is how you can improve on your process. It’s a little thing, but if you can shift (even if only slightly) how you feel about a failing process, it will allow you some room to embrace iterative process more easily.

Conclusions

Managing states / cues is tricky. It’s easy to consider this a rather trivial problem, and it isn’t until you really take time to think carefully about what you’re trying to achieve that you uncover the degree of complexity in the questions around how you manage the flow of information in your network. You wont get it right the first time, but chances are you didn’t ride a bike without a few falls, and you probably didn’t learn to play that instrument without getting a few scales wrong. It’s okay to get it wrong, it’s okay to refactor your code, it’s okay to go back to the drawing board as you search to find what’s right – that’s part of the process, it’s part of what will ultimately make for a better implementation.

No matter what, hang in there… keep practicing, leave yourself breadcrumbs – you’ll get there, even if it takes you longer than you want.

Happy programming.


Zoe Sandoval’s { remnants } of a { ritual }

You can see { remnants } of a { ritual } and the work of the DANM MFA Cohort through May 12th at UC Santa Cruz.