# TouchDesigner | The Object CHOP

The Object CHOP has long been one of the most challenging CHOPs for me to really wrap my head around. Following along with some conversations on the Facebook Help Group 1, it’s clear that I’m not the only one who has bumped their head against how to take advantage of this operator.

With that in mind, here are a few tricks and techniques that you might find helpful when working with the object CHOP.

## Distance Between Many Objects Part 1

At first glance, it seems like the object CHOP can only perform calculations between single objects, but in fact you can use this operator to perform calculations between many objects provided that you format the input data correctly, and set up your object CHOP to account for multiple samples.

In a first example let’s say that we want to find the distance between several green spheres and a blue box:

First let’s collect our position information. I’ve used an object CHOP per sphere to find its distance, but you might also use a script CHOP, or a put positions in a table that you reference for the spheres, or drive them with custom parameters. How you position them doesn’t matter. What we need, however, is a single CHOP with three channels that hold the transformation information of those spheres. My trick in this network is to use object CHOPs to find their positions, then put them in sequence with a join CHOP:

Next we can use a single object CHOP that’s fed reference positions from this join CHOP, and a target Geometry COMP:

Other important pieces here are the start and end parameters on the channel page.

This is where we set how many samples the object CHOP will evaluate. This can be a bit confusing – here especially as the join CHOP has started at a sample index of 1 rather than 0. The devil is in the details, so it’s worth keeping a close eye for these kinds of oddities. Because of this we compensate in our start position by moving back one sample index.

Next make sure to set your object CHOP to output measurements, and distance. What you’ll then end up with is a single channel with a sample for each distance between your box and spheres. We can convert this to a table if we wanted to see the actual values:

## Distance Between Many Objects Part 2

We may also want to measure distances between multiple blue boxes. Say, for example, that we had two different blue boxes and we wanted to know the distances of our spheres to both of those boxes?

Similar to our first exercise we’ll start by collecting all of the position information for our spheres. We also need position information for our boxes. In this case, however, we need to stretch a single sample to be 4 samples long – this is part of our data preparation step to ensure we correctly calculate distance.

Here a simple stretch CHOP has been used to make sure we have four samples of data for each box. Next we can join this data so all of our box position information is in a single set of CHOP channels:

Before moving on, we need to take a moment to adjust our sphere position data. In our first example we only collected the four positions… we need to set up the correct extend behavior for this series so that our CHOPs know what values to use when CHOPs of mismatched lengths are combined. We can use an extend CHOP set to cycle to do this trick:

Finally, we can then use an object CHOP to calculate the distance between our box and our spheres:

## Distance Between Many Objects plus Bearing

If we also calculate the bearing between our boxes and spheres, we’ll end up with rotation information… what can we do with this? We could use this to calculate the correct rotation for a set of instances. For example:

Here each line is correctly rotated, scaled, and placed based on calculations from the object CHOP.

## Bearing

You can also use the object CHOP to just calculate bearing – or rotation from one object to another. Here you can see how this might be used to rotate instances to sit flat on a sphere’s surface, or rotate an arrow to point towards an object:

## Bearing and Distance

Or you might use the combination of bearing and distance to make some strange abstract art:

## Collision

You can also use the object CHOP to simulate a kind of collision calculation where the distance you’re measuring can help you tell how close an object is to another and if they’re on top of one another:

#### GitHub

Clone the Repo to Follow Along

# TouchDesigner | Reflection and Refraction

## I can haz Reflections?! Refractions?

Zoe loves all things reflective and refractive and it was almost a year ago that they started looking into how to achieve compelling illusions of reflection and refraction. Then I went to Macau, then Chicago, then Zoe dove headlong into their thesis project… fast forward to 2019, and it was time for me to finally follow through on a long overdue promise to create some examples of reflection and refraction in TouchDesigner. It didn’t hurt that Zoe gently reminded me that it was time for more refractive rendering in life. Good places to start for these kinds of questions are to look at existing references in the world.

## Reflection

Reflections are hard. In part because they often mean that we need to see the whole world – even the parts that our virtual camera can’t. We might know this intuitively, but the reach of this is easy to forget. When we point the camera in our smartphone at a mirror we see ourselves, the world behind us, above us, and and and. If we point a virtual camera at a virtual mirror we need the same things. That can be a wobbly bit to wrap your head around, and develop a better sense of this challenge I look a look at a reference book I picked up earlier this year – OpenGL 4 Shading Language Cookbook – Second Edition. This has a great chapter on reflection techniques, specifically generating them by using cube-maps. Cubemaps look like an unfolded box, and have a long history of use in computer graphics.

One of the primary challenges of using cubemaps is that you need to also know the perspective of the object that’s reflective. In other words, cube maps can be very convincing as long as you move the camera, but not the reflective object. But what if we want the option to both move the camera, and the object? In this quick tutorial, we look at how we can use a cube map to create convincing reflections, as well as what steps we need to consider if want not only the camera to move, but the object itself.

## Refraction

The one and only Carlos Garcia (L05) has a great example posted on the TouchDesigner forum. This great example helps illustrate the part of what we’re after with this kind of work is the sleight of hand that hints at refraction, but isn’t necessarily true to the physics of light. Almost all realtime rendering tricks are somewhere between the Truth (with a capital T) of the world, and the truth (sneaky lower case t) of perception. We’ve all fallen for the perceptual tricks of optical illusions, and many times the real work of the digital alchemist is to fool observers into believing a half truth. Carlos’ example proves just that point, and helps us see that with a little tricksy use of the displacement TOP we can achieve a healthy bit of trickery.

That’s an excellent start to our adventure, but we can dig-in a little more if we keep searching. Another post on the forum links over to an article on medium that showcases an approach for webGL that leverages the use of a UV map to “pre-compute” the direction of displacement of the light that passes through a transparent object. This is an interesting approach, and in fact there’s a middle ground between the webGL example and Carlos’ TOX that gives us some interesting results.

In the following tutorial we can see how we remix these two ideas. The big picture perspective here is that we can leverage TouchDesigner’s real-time rendering engine to provide the same “pre-computed” asset that’s utilized in the webGL approach, and then use Carlos’ displacement TOP technique. We can also short-cut Carlos’ use of a rendering a second version of the object as a mask, and instead use a threshold TOP looking at our alpha channel to achieve the same effect. This isn’t a huge change, but it’s a bit faster in some implementations and saves us the wobbles that sometimes come with multiple render passes. Finally, a little post processing can help us achieve some more convincing effects that help sell our illusion as a true to the eye.

# TouchDesigner | Packing up a Tox for Distribution

If you’ve been following along with the workshop materials from the TD Summit 2019, there is one more exciting little tid-bit to dig-into. Thinking about how to create a reusable template for creating toxes is no small feat – especially if you want to include external Python libraries. I’ve been thinking about how to approach this challenge and mapped out a rough framework for approaching this challenge. You can see the working repo for this up on github here. During the last part of the workshop on External Python Libraries we covered this a little sneak peak of this technique, and to complete the workshop videos we added a section covering this approach.

The big picture ideas here are to standardize our approach to handling external libraries and automate their installation. This leans on the same concepts we explored in the workshop, and includes using things like a requirements file, using some automated installation scripts, and making sure the path to our project directory is included in our sys.path. The last major change is to convert our general scripts we wrote previously into an extension.

These days I tends to lean towards Extensions over Modules, though you could approach this challenge with Modules very similarly. This set of tutorials will walk us through taking what we created in the previous workshop videos and creating a portable tox we should be able to drag-and-drop into any project. Hopefully, what you take away from this example are some ideas about how to apply this technique and approach to other toxes that you build and share.

Happy Programming!

# TouchDesigner | GitHub and External Toxes

git push git paid

Working with GitHub and TouchDesigner can be a real adventure – and what I hear most often is why and how. A bit ago I did a little write-up about working with git and Touch, but realize that’s still a bit intimidating for folks. This year at the TouchDesigner summit Zoe Sandoval and I taught a workshop about Modular Architectures and working with TouchDesigner. One of the pieces we didn’t have time to really pull apart was working with Git, so before we shared the videos from the workshop we thought it might be helpful to give some perspective about working with Git and where it might save you on a project.

# TouchDesigner | Virtual MIDI Devices

In the past week I’ve found myself in multiple conversations about working with MIDI in TouchDesigner. Specifically, how you might pass a MIDI signal from one application to another on the same computer. So far, all of the conversations I’ve been involved in have been related to working with Windows – which means that this particular post is focused on that operating system, though the same idea should transfer to macOS if that’s your current platform.

## So what’s the problem exactly?

All of my conversations about this topic have been centered around sending messages from TouchDesigner to WATCHOUT. For those unfamiliar, WATCHOUT is a media server application that’s used in Theatrical and live events contexts for controlling or sequencing media playback. It’s in the same family of tools as Disguise, GreenHippo, and Pandora’s Box. You may well be asking – “why not just use OSC?” That’s a great question, some versions of WATCHOUT don’t support OSC, though they do support MIDI. While this is the specific challenge, the more general idea that we can push against here is:

How can I locally (on the same machine) test a MIDI signal flow without adding extra hardware?

To get there we need to add in a virtual device to allow us to loopback. At it’s simplest, a loopback is just a way of capturing a signal that you’d other wise be sending somewhere else. Imagine using a cable from the headphone port on your laptop, and connecting it to the microphone in on your laptop (don’t actually do that, just imagine it). Why? Well, using actual interfaces often have different requirements, or set-up than just faking it – you’ll also likely find that operating systems don’t have an implicit knowledge of a particular signal or transfer format. Just because your computer is making noises doesn’t automatically mean that it can capture that noise locally.

How do we make this work then?

Enter the Virtual Device. A Virtual Device provides drivers that act as though it’s sending your signal to a dedicated piece of hardware, but instead allows you to rout it back to your machine. There are lots of examples of this in the audio world, and it just so happens that there’s a Virtual MIDI device we can install to pass around MIDI signals as well.

Doing a bit of googling I found LoobBe1. It has a free evaluation license, which is what we’ll use below. If you decide that this needs to be part of your commercial workflow, I’d encourage you purchase a license. It’s less than \$15 US, and supporting folks that make useful tools is something we should all do more. As a disclaimer, I don’t have any affiliation or connection to LoopBe1 – I just found them after doing some internet searching.

We’re going to set-up a simple loopback test for a MIDI signal from TouchDesigner, and see if we can read that signal back. I don’t have a copy of WATCHOUT to work with, but if we can trouble shoot within TouchDesigner, we should at least know that it’s working.

## Getting a MIDI Loopback Working

### Ingredients

Windows 10 v1809
TouchDesigner 099 2019.15230
LoobBe1

To get started you’ll need to download and install LoobBe1. I like to limit what starts up automatically on my computer, so I’d recommend also turning off the automatic start-up of this application once you have it installed.

From there we can start up TouchDesigner and begin our configuration process. We’re going to start by opening the MIDI Mapper dialogue. You can open that with `alt + d` or you can find it from the Dialogues drop down:

You should now see the MIDI Mapper:

We’re going to test our output right in TouchDesigner. To do that let’s click on the `Create New Mapping` button in the middle left of the dialogue window. That will add a row to our Device Mapping where we can specify that both our In and Out Device will be `LoopBe Internal MIDI`.

Before we can move forward here it’s important to recognize that the naming of MIDI channels matters. MIDI (Music Instrument Digital Interface) has a detailed specification that ensures continuity across devices and manufactures, so it shouldn’t be a surprise that there’s lots of detail here. You can read the whole spec by here, or get a summary from Wikipedia. What we really care about today is that we need to format our TouchDesigner channel names in a way that’s compatible with Derivative’s implementation. To figure all of that out, we need to head over to the wiki for a quick read. Let’s take a look at the MIDI Out CHOP page. Here we see that:

Naming the CHOP channels: Channels are mapped to events by their name. Events like notes, controllers and velocities must be followed by the note/controller number (n65, c7). If the number is left off a note event, the note number is the value of the channel. Other events, which are sent to the entire channel, do not need a trailing number (pc, pw). The channel prefix can be used to identify the MIDI channel the event should be sent on (i.e. “ch1n45” assigns that TouchDesigner channel to note 45 messages on MIDI channel 1). Channels can always be renamed with a Rename CHOP before entering the MIDI Out CHOP.

The MIDI Out CHOP sends MIDI velocity as well. The values of the channels entering the MIDI Out CHOPs are sent as the velocity of the note. If Normalize is “None”, the channel needs to be 0 to 127. If Normalize is “0 to 1”, channel values between 0 and 1 are scaled to be MIDI 0 to 127.

The “Cook Every Frame” option cooks the CHOP every frame, even if the CHOP isn’t being displayed. All Volume Off and All Volume On flags are new and emit events for Controller 7 of all 16 channels. MIDI output go in a separate thread to allow output that slows TouchDesigner less. It now works in Time Slice mode for note events and controller events. (Not for Program Change or Sysex messages yet) Note channels only trigger anew Note On when the input channel goes from 0 or less to a value greater than zero. Similar for Note Off events.The channel name determines how it is interpreted.

Derivative’s MIDI Out CHOP Wiki Page

What does that even mean?! Well, let’s set up a few pieces in a network to see if we can make sense of this.

I’m going to add a `Noise CHOP` to my network, and turn on the time-slice parameter so we have some constantly changing values. I’m going to change the Channel name parameter to be `ch1c1`. This would be Channel 1, Control 1.

By default our `Noise CHOP` will have values between -1 and 1. Based on what’s in the wiki, we’ll either need to normalize our values (change them to be in a range of 0-1) or we’ll need to change our values to be in a range of 0-127. We can do either of these with a `Math CHOP` so let’s add one of those in line and change our range parameters:

Now we can finish this off by adding a `MIDI Out CHOP` to our network:

So how do we make sure this is working?

Well, let’s first check with Touch. Since we added our LoopBe Internal MIDI as our input device, we should now be able to drop in a `MIDI In CHOP` to see our midi values:

Great – so we can see our values… only they’re expressed as whole numbers rather than as a floating point value. Why? MIDI works with 127 steps, so our MIDI Out CHOP is converting our 0-1 set of values into a range of 0-127. What this means is that you, dear developer-artist, have to make a choice. This approach works perfectly fine, but it may mean that at some point you’ll be doing some mental gymnastics in understanding how values in Touch correspond to values in your other application. If that’s too much to think about all the time, then you may want to change the math CHOP to be in a range of 0-127, with the values rounded either up or down (depending on your preference). This kind of change would let you see exactly what value you’re expected to be transmitted from Touch. If you’re working on a high profile AV installation, this kind of specificity might be worth having – if only so you can say with complete confidence that you’re transmitting a value of x on channelxyz.

Okay, so it works in Touch… but can we be sure? Of course. I’m going to use Isadora by Troikatronix to help us validate that we’re sending this correctly. I’m not going to go into too many of the details for how we set-this up in Izzy, just enough so we can validate what we’re seeing here.

To start I’m going to open up Isadora, go to the Communications drop down, and select MIDI Setup:

Like Touch, I’m going to set the Input Port to be LoopBe Internal MIDI:

To see if we’re actually transmitting values, let’s open up the Izzy Status monitor:

Sure enough, we should see a blinking Green light on Channel 1 and the value associated with it:

TADA! You’re passing data around locally from one application to another using MIDI.

## Takeaways

Okay, so what exactly did we do? Well, we added a virtual MIDI interface that let us send MIDI encoded data between applications on a single computer. We then verified that this works by first looking at the data transmission in Touch, then further validated that this works working by looking outside of Touch with another application.

Why on earth do any of this? Well, that’s really a question for you. If MIDI is a core part of your work flow, and passing data between applications on the same computer has an important role for you, then the answers might seem obvious. For those still scratching their heads, imagine that you have raw sensor data that’s not MIDI friendly, but you’d like to get it into an application that does speak MIDI. This kind of workflow is great for that. Working with a Kinect but want to turn that data into MIDI, easy. What about a Leap – sure. Can I do my mouse – you betcha.

It might not be what you need today, but maybe it’ll inspire you to think about some mischief you could get yourself into.

Happy Programming.

# TouchDesigner | New Features | Bindings

Spring time is lots of things – flowers, holidays, vigorous allergies, and the TouchDesigner Spring Update. For the second year running this is the time of year that features graduate from just being in experimental to being full fledged stable release features. Wowza.

This spring we’re seeing a feature that’s flat out amazing, and likely a bit of a sleeper. Bindings. Elburz has a great shout out to bindings on a recent blog, and I wanted to take some time to dig in and step through an example of both why they’re important, the paradigm they’re built on, and why they matter.

So what are parameter bindings anyway? The Derivative wiki has a great segment describing bindings:

Bound parameters keep their values in sync and will respond to changes from either parameter. For each parameter that is a bind master, it will have one or more bind reference parameters.
The bind master holds the current value. It can have exports and expressions and generally works like any other parameter, but its value can be also changed indirectly by the bind references.
A bind reference holds the location of the bind master. A bind reference is in a fourth “Bind” Parameter Mode, and will show up as purple text in parameter dialogs. It can only be changed via its its bind master, its UI, and its `val` property.

Derivative Wiki article on Binding

## Model View Controller – MVC

Errm. Okay, so what does that mean exactly? Bindings are based on an interface architecture paradigm called Model – View – Controller, or MVC. Wikipedia has a nice starter debrief on the idea for us, but it’s easy to understand if you’ve spent much time working in Touch either for live set, or for a client application. Let’s consider a live set to help us get our footing here. Suppose you have parameter that can be updated by multiple touch points both in your own UI and in the TD UI. We all know this game, as soon as you export a CHOP to a parameter, you can no longer change that parameter except through the exported CHOP.

Fine.

So maybe instead of exporting from a single CHOP you instead write a bunch of scripts to handle this operation – only now you’re in a real pickle. Why? Well, because your script changes the parameter, but doesn’t update all of the UI elements that reflect the state of that parameter. So you write another script to update the UI. But now you’ve managed to save your project in a state where the UI and the parameters are not aligned. As soon as you change the UI everything is in sync – so you save over a few things, commit your changes, and now everything should be great. Until you need to load a saved preset state from disk. Now you’ve gotta write another set of scripts to do all of that updating, or hunker down for a more generalized solution – which probably means a code refactor. Who wanted to make some more sweet visuals anyway? There goes your night off. There goes your margin. Sigh.

The real world example of this is light switches. If you’ve ever lived in an apartment where multiple light switches control the same light / outlet, you understand this issue intimately. How do you know if the light is on or off? Only by looking at the light, because once the states of the light switches are out of phase they perform the opposite action.

The Model – View – Controller paradigm is a design architecture that’s intended to help resolve this issue. The MVC approach decouples the control of a UI element from the data it is manipulating. You could do this in touch before, you were just on the hook for doing all the set-up. This probably meant you had a master table or storage dictionary somewhere that was updated whenever a parameter was changed, that in turn would update all the other touch points. That’s a huge hassle, but it was the only way to solve this problem. It’s also the kind of silly thing you could really have a strangely strong opinion about – and consequently be convinced that your collaborators were doing it all wrong.

## Enter Bindings

Okay, so as a refresher, Bindings are a new parameter mode – that’s the little multi-colored set of dots next to any parameter. This new mode is purple – one of the many colors of awesome. At the end of all of this, we’ll take a peek at the new widgets – and the UI redesign they offer, but to get started let’s build a use case for bindings so we can get a sense of what they’re good for.

## Slider

We’re going to start with a good old fashioned slider. Why a slider?! Well, this is the kind of parameter we end up using all the time, and the fundamental nature of this UI piece should be foundational enough that if we can get a handle on this one, the jump to more abstract ideas should be a little easier.

Let’s get started by first adding a slider from the Op Create dialogue. We’re just going to add a run of the mill slider for now.

From here we’re going to customize our slider – let’s add a page called “Settings” and then a float custom parameter that we call “Slider” for now.

So far so good. The next step is where it’s gonna get a little weird, and where it’ll get different than before. From here, let’s dive into our slider. We’re going to delete our `Panel CHOP`, and add our own `Constant CHOP`.

Okay. Now here’s the wild part. On our new `Constant CHOP` we’ll use the new bindings parameter mode to write `parent().par.Slider` – that’s the reference to our newly created custom parameter.

If you’re not into that whole writing expressions exercise, you can also do this with a little drag-and-drop action:

Okay… so why is this interesting. Well, let’s see what happens when we move our new custom parameter, or move our `Constant CHOP`:

Slick. Okay. That’s fly. So if we update the our parameter or our CHOP both changes are reflected in the other operator. Now let’s make a few final changes so that when we interact with the sliders panel we update both of these values. We can do this with a `Panel Execute DAT`. Let’s add our DAT and modify the contents with the following:

```def onValueChange(panelValue):
parent().par.Slider = panelValue
return```

We should now see that if we move the slider that our Slider parameter updates, and our `Constant CHOP` updates.

We’re very close now. All we need to do is to update the `knob` component in our slider. We need to change the expression there to be:

`parent().par.Slider*parent().width-me.par.panelw/2`

There we have it. Now we can change our custom parameter, our `Constant CHOP`, or the slider and all three stay in sync. MAGIC.

## Widgets?

Early on I mentioned that we find this same behavior in Widgets. What exactly are widgets you ask – currently Widgets are rolling out as a huge overhaul to the TUIK interface building kit that was relatively ubiquitous in TouchDesigner networks. Widgets are more modern take on UI building, and offer significant advances to UI building approaches for Touch. There’s too much to cover about widgets here, but it’s worth pointing out that the same binding approach you see above is a fundamental element of the widget system. It allows for bidirectional control of both user interaction elements and parameters – the core principle we just explored. You can dig in and learn a little more about widgets by reading through the article on the Derivative Wiki.

## Why Bind…

You might be looking at this and feeling like it’s outside of your wheelhouse, or your workflow.

A reasonable reflection.

Regardless, I’d encourage you to think about the times when you’ve wanted to both control from more than one location – the parameter itself, as well as some other control interface. If nothing else, give them a try to see where they might fit – if they’re no good for you, don’t use them… though I suspect you’ll find they have all sorts of exciting uses.

Happy Programming!

# TouchDesigner | Case Study | Custom Parameters and Cues

I recently had the good fortune of being able to collaborate with my partner, Zoe Sandoval, on their MFA thesis project at UC Santa Cruz – { 𝚛𝚎𝚖𝚗𝚊𝚗𝚝𝚜 } 𝚘𝚏 𝚊 { 𝚛𝚒𝚝𝚞𝚊𝚕 } Thesis work is strange, and even the best of us who have managed countless projects will struggle to find balance when our own work is on the line – there is always the temptation to add more, do more, extend the piece a little further, or add another facet for the curious investigator. Zoe had an enormous lift in front of them, and I wanted to help streamline some of the pieces that already had functioning approaches, but would have benefited from some additional attention and optimization. Specifically, how cues and states operated was an especially important area of focus. I worked closely with the artist to capture their needs around building cues / states and translate that into a straightforward approach that had room to grow as we made discoveries, and needed to iterate during the last weeks leading up to opening.

## The Big Picture

{ remnants } of a { ritual } is an immersive installation comprised of projection, lighting, sound, and tangible media. Built largely with TouchDesigner, the installation required a coordinated approach for holistically transforming the space with discrete looks. The projection system included four channels of video (two walls, and a blended floor image); lighting involved one overhead DMX controlled instrument (driven by an ENTEC USB Pro), and four IoT Phillips Hue lights (driven by network commands – you can find a reusable approach on github); sound was comprised of two channels driven by another machine running QLab, which required network commands sent as OSC. The states of each of these end points, the duration of the transition, and the duration of the cue were all elements that needed to both be recorded, and recalled to create a seamless environmental experience.

Below we’re going to step though some of the larger considerations that led to the solution that was finally used for this installation, before we get there though it’s helpful to have a larger picture of what we actually needed to make. Here’s a quick run-down of some of the big ideas:

• a way to convert a set of parameters to python dictionary – using custom parameters rather than building a custom UI is a fast way to create a standardized set of controls without the complexity of lots of UI building in Touch.
• a reusable way to use storage in TouchDesigner to have a local copy of the parameters for fast reference – typically in these situations we want fast access to our stored data, and that largely looks like python storage; more than just dumping pieces into storage, we want to make sure that we’re thoughtfully managing a data structure that has a considered and generalized approach.
• a way to write those collections of parameters to file – JSON in this case. This ensures that our preset data doesn’t live in our toe file and is more easily transportable or editable without having TouchDesigner open. Saving cues to file means that we don’t have to save the project when we make changes, and it also means that we have a portable version of our states / cues. This has lots of valuable applications, and is generally something you end up wanting in lots of situations.
• a way to read those JSON files and put their values back into storage – it’s one thing to write these values to file, but it’s equally important to have a way to get the contents of our file back into storage.
• a way to set the parameters on a COMP with the data structure we’ve been moving around – it’s great that we’ve captured all of these parameters, but what do we do with this data once we’ve captured it? The need here is thinking through what to do with that data once you have it captured.

## Cuing needs

One of the most challenging, and most important steps in the process of considering a cuing system is to identify the granularity and scope of your intended control. To this end, I worked closely with the artist to both understand their design intentions, as well their needed degrees of control. For example, the composition of the projection meant that the blended floor projection was treated as a single input image source; similarly, the walls were a single image that spanned multiple projectors. In these cases, rather than thinking of managing all four projectors it was sufficient to only think in terms of the whole compositions that were being pulled. In addition to the images, it was important to the artist to be able to control the opacity of the images (in the case of creating a fade-in / out) as well as some image adjustments (black level, brightness, contrast, HSV Offset). Lighting, and sound had their own sets of controls – and we also needed to capture a name for the cue that was easily identifiable.

As lovely as it would be to suggest that we knew all of these control handles ahead of time, the truth is that we discovered which controls were necessary through a series of iterative passes – each time adding or removing controls that were either necessary or too granular. Another temptation in these instances is to imagine that you’ll be able to figure out your cuing / control needs on your feet – while that may be the case in some situations, it’s tremendously valuable to instead do a bit of planning about what you’ll want to control or adjust. You certainly can’t predict everything, and it’s a fool’s errand to imagine that you’re going to use a waterfall model for these kinds of projects. A more reasonable approach is to make a plan, test your ideas, make adjustments, test, rinse, repeat. An agile approach emphasizes smaller incremental changes that accumulate over time – this requires a little more patience, and a willingness to refactor more frequently, but has huge advantages when wrestling with complex ideas.

## Custom Pars

In the past I would have set myself to the task of handling all of these controls in custom built UI elements – if was was creating an interface for a client and had sufficient time to address all of the UI / UX elements I might have taken that approach here, but since there was considerable time pressure it was instead easier (and faster) to think about working with custom parameters. Built in operators have their own set of parameters, and Touch now allows users to customize Component operators with all of the same parameters you find on other ops. This customization technique can be used to build control handles that might otherwise not need complete UI elements, and can be further extended by using the Parameter COMP – effectively creating a UI element out of the work you’ve already done while specifying the custom parameters. The other worthwhile consideration to call out here is your ability to essentially copy parameters from other objects. Consider the black level, contrast, and brightness pars included above. One approach would be to create each par individually, and set their min, max, and default values. It would, however, be much faster if we could just copy the attributes from the existing level TOP. Luckily we can do just that with a small trick.

We start by creating a base Comp (or any Component object), right clicking on the operator, and selecting customize component.

This opens the customize component dialogue where we can make alterations to our COMP. Start by adding a new page to your COMP and notice how this now shows up on the components parameters pages:

For now let’s also add a level TOP so we can see how this works. From your level TOP click and drag a parameter over to the customize component dialogue – dragging specifically to the Parameter column on the customize component dialogue:

This process faithfully captures the source parameter’s attributes – type, min, max, and default vals without you needing to input them manually. In this style of workflow the approach is to first start by building your operator networks so you know what ops you will want to control. Rather than designing the UI and later adding operator chain, you instead start with the operator chain, and only expose the parameters you’ll need / want to control. In this process you may find that you need more or fewer control handles, and this style of working easily accommodates that kind of workflow.

## Capturing Pars

Creating a component with all of your parameters is part of the battle, how to meaningfully extract those values is another kettle of fish. When possible it’s always handy to take advantage of the features of a programming language or environment. In this case, I wanted to do two things – first I wanted to be able to stash cues locally in the project for fast retrieval, second I wanted to have a way to write those cues to disk so they were’t embedded in a toe or tox file. I like JSON as a data format for these kinds of pieces, and the Python equivalent to JSON is dictionaries. Fast in TD access means storage. So here we have an outline for our big picture ideas – capture the custom parameters, stash them locally in storage, and write them to disk.

One approach here would be to artisanaly capture each parameter in my hipster data structure – and while we could do that, time fighting with these types of ideas has taught me that a more generalized approach will likely be more useful, even if it takes a little longer to get it right. So what does that look like exactly?

To get started, let’s create a simple set of custom parameters on a base COMP. I’m going to use the trick we learned above to move over a handful of parameters form a level TOP: Black Level, Contrast, and Opacity:

To create a dictionary out of these pars I could write something very specific to my approach that might look something like this snippet:

At first glance that may seem like an excellent solution, but as time goes on this approach will let us down in a number of ways. I wont’ bother to detail all of them, but it is worth capturing a few of the biggest offenders here. This approach is not easily expanded – if I want to add more pars, I have to add them directly to the method itself. For a handful, this might be fine, but over ten and it will start to get very messy. This approach also requires duplicate work – the key name for values means that I need to manually verify if the key name and parameter name match (we don’t have to do this, but we’ll see later how this saves us a good chunk of work), if I misspell a word here I’ll be very sorry for it later. The scope of this approach is very narrow – very. In this case the target operator variable is set inside of the method, meaning that this approach will only ever work for this operator, at this level in the network. All of that and more largely mean that while I can use this method very narrowly, I can use this approach, but I’m going to be sorry for it in the long run.

Instead of the rather arduous process above, we might consider a more generalized approach to solving this problem. Lucky for us, we can use the `pars()` method to solve this problem. The `pars()` method is very handy for this kind of work, the major catch being that `pars()` will return all of the parameters on a given object. That’s all well and good, but what I really wanted here was to capture only custom parameters on a specific page, and to be able to ignore some parameters (I didn’t, for example, need / want to capture the “save cue” parameter). What might his kind of approach look like, let’s take a look at the snippet below.

## Abstract Reusable code segment

What exactly is happening here? First off, this one is full of documentation so our future selves will know what’s happening – in fact this is probably more docstring than code. The big picture idea is rather than thinking about this problem one parameter at a time, we instead what to think of entire custom pages of parameters. Chances are we want to re-use this, so it’s been made to be fairly general – we pass in an operator, the name of the page we want to convert to a python dictionary, the name of our newly made preset, and a list of any parameters we might want to skip over. Once we pass all of those pieces into our function, what we get back is a dictionary full of those parameters.

## Capture to Storage

Simply converting the page of parameters to a dictionary doesn’t do much for us – while it is a very neat trick, it’s really about what we do with these values once we have them in a data structure. In our case, I want to put them into storage. Why storage? We certainly could put these values into a table – though there are some gotchas there. Table values in TouchDesigner are always stored as strings – we might think of this as text. That matters because computers are notoriously picky about data, and find the challenge of differentiating between words, whole numbers, and numbers with decimal values very difficult. Programmers refer to words as strings, whole numbers as integers or ints, and numbers with decimal values as floats. Keeping all of our parameter values in a table DAT means they’re all converted to strings. Touch mostly does an excellent job of making this invisible to you, but when it goes wrong it tends to go wrong in ways that can be difficult to debug. Using storage places our values in a python dictionary where our data types are preserved – not converted to strings. If you’re only working with a handful of cues and a handful of parameters this probably doesn’t matter – but if you’re thinking about 20 or more parameters it doesn’t take many cues before working in native data types will make a big difference. For reference, an early iteration of the cuing system for this would have needed the equivalent of a table DAT with over 1000 rows to accommodate the stored parameters. These things add up quickly, more quickly than you first imagine that they might.

Okay, so what’s an example of a simple and reusable function we might use to get a dictionary into storage:

## Write to file

Similar to the above, we likely want a simple way to write our stored cues to disk in the same format we’re using internally. Python dictionaries and JSON are nearly interchangeable data structures and for our needs we can think of them as being the same thing. We do need to import the JSON module to get this to work correctly, but otherwise this is a straightforward function to write.

What you end up with will look like this:

We’re close now to having a complete approach for working with cues / states. Our next puzzle piece here would be a way to read our JSON from disk, and replace what we have in storage with the file’s contents. This means that whatever is in the file can be used to replace what we have in storage.

What you end up with here might look like this:

This part is the most tricky. Here the the big idea is to create a duplicate operator that has all of the same custom parameters in our preset maker. Why? Well, that would mean that all of the parameter names match – so which would make loading parameters significantly easier and more straightforward. The other trick here is to remove any of the ignored pars from our ignore list – thinking back this is to ensure that we don’t use any of the parameters that we don’t want / need outside of recording them. We can start this process by making a copy of our operator that’s being used to capture our parameters and then deleting the pars we don’t need. Next we need to write a little script to handle moving around all of the values. That should look something like this:

## Making a Module

All of this is a good bit of work, and if you’ve been following along, you probably now have a handful of text DATs doing all of this work. For the sake of keeping tidy, we can instead put all of this into a single DAT that we can use as a python module. Wrapping all of these pieces together will give us something like this:

If you want to see how this works and pull it apart from there you can pull an example TOE file form this repo.

## TD JSON – another alternative

There’s another alternative to this approach – which is the new TD JSON elements that are in the TouchDesigner. You can read about them on Derivative’s wiki here. These tools are a promising alternative, and you can do very similar pieces here. In particular we might use something like `pageToJSONDict()`to do what we’re after. That might look something like this:

That’s slick, but what we get back is almost 75 lines worth of JSON. This feels a little overkill to me for what we’re after here – there’s lots of potential here, but it might be a little more than we need for this actual implementation. Maybe not though, depending on how you want to change this process, it might be just perfect.

## Safety Rails

There are some pieces missing in the approach above that I ended up including in the actual implementation for the artist – I’m not going dig into some of these pieces, but it is worth calling attention to some of the other elements that were included. The biggest pieces that needed to be addressed were how we handle failure, duplicates, provided confirmation on an operation, or warned the user about possibly unintended operations.

The artist, for example, wanted to both have the UI flash and to get a message printed to the text port when a preset was successfully saved. The artist also wanted to make sure that a preset wasn’t automatically overwritten – instead they wanted to see a pop up message warning that a preset was going to be overwritten, allowing the user confirm or cancel that operation.

That may seem unnecessary for a tool you build for yourself… until it’s 2am and you haven’t slept, or you’re working fast, or there’s a crit in 10 minutes and you want to make one more adjustment, and and and, or or or. Handling these edge cases can not only add piece of mind, but also ensure you keep your project on the straight and narrow.

Additionally, how you handle failure in these situations is also important to plan – we never want these pieces to fail, but having a gracefully solution for how to handle these moments are tremendously important to both work through and plan. If nothing else, it’s elegantly handling the failure and printing a message – better still is if give yourself a clue about what went wrong. A few breadcrumbs can go a long way towards helping you find the trail you lost. In my opinion, failing code gets a bad wrap – it’s something we grumble over, not something we celebrate. The truth of the matter, however, is that failures are how we make projects better. Being able to identify where things went wrong is how you can improve on your process. It’s a little thing, but if you can shift (even if only slightly) how you feel about a failing process, it will allow you some room to embrace iterative process more easily.

## Conclusions

Managing states / cues is tricky. It’s easy to consider this a rather trivial problem, and it isn’t until you really take time to think carefully about what you’re trying to achieve that you uncover the degree of complexity in the questions around how you manage the flow of information in your network. You wont get it right the first time, but chances are you didn’t ride a bike without a few falls, and you probably didn’t learn to play that instrument without getting a few scales wrong. It’s okay to get it wrong, it’s okay to refactor your code, it’s okay to go back to the drawing board as you search to find what’s right – that’s part of the process, it’s part of what will ultimately make for a better implementation.

No matter what, hang in there… keep practicing, leave yourself breadcrumbs – you’ll get there, even if it takes you longer than you want.

Happy programming.

## Zoe Sandoval’s { remnants } of a { ritual }

You can see { remnants } of a { ritual } and the work of the DANM MFA Cohort through May 12th at UC Santa Cruz.

# TouchDesigner | Switch Statements in Python

Hang onto your socks programmers, we’re about to dive deep. What are we up to here today? Well, we’re going to look into switch statement alternatives in Python (if you don’t know what a switch statement is don’t worry we’ll cover that bit), how you might use that in a practical real-world situation, and why that’s even an idea worth considering. With that in mind let’s dig-in and start to pull apart what Switch Statements are, and why you should care.

From 20,000 feet, switch-case statements are an approach to handling different situations by way of a look-up table rather than with a series of if-else statements. If you’re furrowing your brow consider situations when you may have encountered complex if-else statements where once change breaks everything… for so so much longer than you might want. Also consider what happens if you want to extend that if-else ladder into something more complicated… maybe you want to call different functions or methods based on input conditions, maybe you need to control a remote machine and suddenly you’re scratching your head as you ponder how on earth you’re going to handle complex logic statements across a network. Maybe you’re just after a better code-segmentation solution. Or maybe you’ve run into a function so long you’re starting to loose cycles to long execution times. These are just a few of the situations you might find yourself in and a switch statement might just be the right tool to help – except that there are no switch-case statements in Python.

What gives?!

While there aren’t any switch-case statements, we can use dictionary mappings to get to a similar result… a result so powerful we’re really in for a treat. Before we get there though, we need to look at the situation we’re trying to avoid.

So what exactly is that situation? Let’s consider a problem where we want to only call one function and then let that code block handle all of the various permutations of our actions. That might look like our worst case solution below.

### Worst

 def switcher(func_name, val1, val2): if func_name == "Add": result = val1 + val2 elif func_name == "Subtract": result = val1 – val2 elif func_name == "Multiply": result = val1 * val2 elif func_name == "Divide": result = val1 / val2 return result print( switcher("Subtract", 1, 2) )

To get started, what do we have above? We have a single function called switcher() that takes three arguments – the name of the function we want to call, and two values. In this example we have four different math operations, and we want to be able to access any of the four as well as pass in two values and get a result just by calling a single function. That doesn’t seem so hideous on the face of it, so why is this the worst approach?

This example probably isn’t so terrible, but what it does do is bury all the functional mathematical portions of our code inside of a single function. It means we can’t add and test a new element without possibly breaking our whole functional code block, we can only access these operations from within switcher(), and if we decide to add additional operations in the future our code block will just continue to accrue lines of code. It’s a naive approach (naive in the programming sense – as in the first brute force solution you might think of), but it doesn’t give us much room for modularity or growth that doesn’t also come with some unfortunate side effects.

Okay… fine… so what’s a good solution then?

### Good

 def switcher(func_name, val1, val2): if func_name == "Add": result = Add(val1, val2) elif func_name == "Subtract": result = Subtract(val1, val2) elif func_name == "Multiply": result = Multiply(val1, val2) elif func_name == "Divide": result = Divide(val1, val2) return result def Add(val1, val2): sum_val = val1 + val2 return sum_val def Subtract(val1, val2): sum_val = val1 – val2 return sum_val def Multiply(val1, val2): sum_val = val1 * val2 return sum_val def Divide(val1, val2): sum_val = val1 / val2 return sum_val print( switcher("Subtract", 1, 2) )

A good solution segments our functions into their own blocks. This allows us to develop functions outside of our switcher() function, call them independently, and have a little more flexible modularity. You might well be thinking that this seems like a LOT more lines… can we really say this is better?! Sure. The additional lines are worth it if we also get some more handles on what we’re doing. It also means we probably save some serious debugging time by being able to isolate where a problem is happening. In our worst case approach we’re stuck with a single function that if it breaks, none of our functions work… and if our logic got sufficiently complex we might be sifting through a whole heap of code before we can really track down what’s happening. Here at least there’s a better chance that a problem is going to be isolated to a single function block – that alone is a HUGE help.

All that said, we’re still not really getting to switch-case statements… we’re still stuck in if-else hell where we’ll have to evaluate our incoming string against potentially all of the possible options before we actually execute our actual code block. At four functions this isn’t so bad, but if we had hundreds we might really be kicking ourselves.

So how can we do better?

### Better

 def switcher(func_name, val1, val2): functions = { "Add" : Add, "Subtract" : Subtract, "Multiply" : Multiply, "Divide" : Divide } active_function = functions.get(func_name) result = active_function(val1, val2) return result def Add(val1, val2): sum_val = val1 + val2 return sum_val def Subtract(val1, val2): sum_val = val1 – val2 return sum_val def Multiply(val1, val2): sum_val = val1 * val2 return sum_val def Divide(val1, val2): sum_val = val1 / val2 return sum_val print( switcher("Subtract", 1, 2) )

Better is to remember that the contents of a python dictionary can be any data type – in fact they can even be function names, or Python objects. How does that help use? Well, it means we can look up what function we want to call on the fly, call it, and even pass in variables. In the example above our switcher() function holds a dictionary of all the possible functions at our disposal – when we call our switcher we pass in the name of the function with the variables that will in turn get passed to the function. Above our active_function variable becomes the variable that’s fetched from our dictionary, which we in turn pass our incoming variables along to.

That’s great in a lot of ways, but especially in that it gets us away from long complicated if-else trees. We can also use this as a mechanism for handling short-hand names for our methods, or multiple assignments – we might want two different keys to access the same function (maybe “mult” and “Multiple” both call the same function for example).

So far this is far away a better approach, so how might we make this better still?

### Best

 def switcher(func_name, vals): functions = { "Add" : Add, "Subtract" : Subtract, "Multiply" : Multiply, "Divide" : Divide } active_function = functions.get(func_name) result = active_function(vals) return result def Add(vals): sum_val = 0 for item in vals: sum_val += item return sum_val def Subtract(vals): sum_val = vals[0] for item in vals[1:]: sum_val -= item return sum_val def Multiply(vals): if len(vals) > 2: sum_val = 'Invalid Call – "Multiply" only takes two vals' else: sum_val = vals[0] * vals[1] return sum_val def Divide(vals): if len(vals) > 2: sum_val = 'Invalid Call – "Divide" only takes two vals' else: sum_val = vals[0] / vals[1] return sum_val vals = [3, 4] print( switcher("Multiply", vals) )

We might take this one step further and start to consider how we might address accepting an arbitrary number of vals. Above we have a simple way to tackle this – probably not what you’d end up with in production, but something that should hopefully get you thinking. Here the variable vals becomes a list that can be any number of values. In the case of both our Add() and Subtract() functions we loop through all of the values – adding each val, or subtracting each val respectively. In the case of our Multiply() and Divide() functions we limit these operations to only two values for the sake of our example. What’s interesting here is that we can return can think about error handling based on the array of values that’s coming into our function.

The above is great, of course, but it’s really just the beginning of the puzzle. Where this really starts to become interesting is how you might think of integrating this approach in your python extensions.

Or if vals is a a dictionary in it’s own right rather than a simple list.

Or if you can send a command like this over the network.

Or if you can start to think about how to build out blocks of code that are specific to a single job, and universal blocks that apply to all of your projects.

Next we’ll start to pull apart some of those very ideas and see where this concept really gets exciting and creates spaces for building tools that persists right alongside the tools that you have to build for a single job.

In the meantime, experiment with some Python style switch statements to see if you can get a handle on what’s happening here, and how you might take better advantage of this method.

Happy programming!

#### References

Looking for another perspective on this approach form a more pure Python perspective? Check out this post on Jaxenter.com.

# TouchDesigner | Delay Scripts

It’s hard to appreciate some of the stranger complexities of working in a programming environment until you stumble on something good and strange. Strange how Matt? What a lovely question, and I’m so glad that you asked!

Time is a strange animal – our relationship to it is often changed by how we perceive the future or the past, and our experience of the now is often clouded by what we’re expecting to need to do soon or reflections of what we did some time ago. Those same ideas find their way into how we program machines, or expect operations to happen – I need some-something to happen at some time in the future. Well, that’s simple enough on the face of it, but how do we think about that when we’re programming?

Typically we start to consider this through operations that involve some form of delay. I might issue the command for an operation now, but I want the environment to wait some fixed period of time before executing those instructions. In Python we have a lovely option for using the time module to perform an operation called sleep – this seems like a lovely choice, but in fact you’ll be oh so sorry if you try this approach:

 import time # it may be tempting to do this, but # this is not the correct way to delay # an operation in Python inside of # TouchDesigner time.sleep(1) print("oh, hello there")

But whyyyyyyyy?!

Well, Python is blocking inside of TouchDesigner. This means that all of the Python code needs to execute before you can proceed to the next frame. So what does that mean? Well, copy and paste the code above into a text DAT and run this script.

If you keep an eye on the timeline at the bottom of the screen, you should see it pause for 1 second while the time.sleep() operation happens, then we print “oh, hello there” to the text port and we start back up again. In practice this will seem like Touch has frozen, and you’ll soon be cursing yourself for thinking that such a thing would be so easy.

So, if that doesn’t work… what does? Is there any way to delay operations in Python? What do we do?!

Well, as luck would have it there’s a lovely method called run() in the td module. That’s lovely and all, but it’s a little strange to understand how to use this method. There’s lots of interesting nuance to this method, but for now let’s just get a handle on how to use it – both from a simple standpoint, and with more complex configurations.

To get started let’s examine the same idea that we saw above. Instead of using time.sleep() we can instead use run() with an argument called delayFrames. The same operation that we looked at above, but run in a non-blocking way would look like this:

 # delay scripts example # first we write out script as a string delay_print = "print('hello world')" # next we run this string, and specify # how many frames we want to wait before # we run this operation. run(delay_print, delayFrames = 60 )

view raw
delay-script-ex1.py
hosted with ❤ by GitHub

If you try copying and pasting the code above into a text DAT you should have much better results – or at least results where TouchDesigner doesn’t stop running while it waits for the Python bits to finish.

Okay… so that sure is swell and all, so what’s so complicated? Well, let’s suppose you want to pass some arguments into that script – in fact we’ll see in a moment that we sometimes have to pass arguments into that script. First things first – how does that work?

 # delay scripts example # first we write out script as a string delay_print = '''print( '{noun} sure does {verb} {adj}'.format( noun=args[0], verb=args[1], adj=args[2]))''' noun = "Matthew" verb = "love" adj = "TouchDesigner" # next we run this string. # this time we'll pass in arguments for our script. # args are accessed as a list called args. Arguments # go in order in the run command, and you can see in # how the script is formatted how we access them. run(delay_print, noun, verb, adj, delayFrames = 60 )

Notice how when we wrote our string we used args[some_index_value] to indicate how to use an argument. That’s great, right? I know… but why do we need that exactly? Well, as it turns out there are some interesting things to consider about running scripts. Let’s think about a situation where we have a constant CHOP whose parameter value0 we want to change each time in a for loop. How do we do that? We need to pass a new value into our script each time it runs. Let’s try something like:

 # delay scripts example # first we write out script as a string delay_script = "constant_op.par.value0 = args[0]" constant_op = op('constant1') # next we run our script for each_time in range(10): run(delay_script, (each_time+1), delayFrames = 60 * (each_time + 1))

view raw
args-and-delay.py
hosted with ❤ by GitHub

What you should see is that your constant CHOP increments every second:

But that’s just the tip of the iceberg. We can run strings, whole DATs, even the contents of a table cell.

This approach isn’t great for everything… in fact, I’m always hesitant to use delay scripts too heavily – but sometimes they’re just what you need, and for that very reason they’re worth understanding.

If you’ve gotten this far and are wondering why on earth this is worth writing about – check out this post on the forum: Replicator set custom parms error. It’s a pretty solid example of how and why it’s worth having a better understanding of how delay scripts work, and how you can make them better work for you.

Happy Programming.

# TouchDesigner | Finding Dominant Color

Programming is a strange practice. It’s not uncommon that in order to make what’s really interesting, or what you promised the client, or what’s driving a part of your project you have to build another tool.

You want to detect motion, so you need to build out a means of comparing frames, and then determining where the most change has occurred. You want to make visuals that react to audio, but first you need to build out the process for finding meaningful patterns in the audio. And on and on and on.

And so it goes that I’ve been thinking about finding dominant color in an image. There are lots of ways to do this, and one approach is to use a technique called KMeans clustering. This approach isn’t without its faults, but it is interesting and relatively straightforward to implement. The catch is that it’s not fast enough for a realtime application – at least not if you’re using Python. So what can we do? Well, we can still use KMeans clustering, but we need to understand how to use multi-threading in python so we don’t block our main thread in TouchDesigner.

The project / tool / example below does just that – it’s a mechanism for finding dominant color in an image with an approach that uses a background thread for processing that request.

# TouchDesigner Dominant Color

An approach for finding dominant color in an image using KMeans clustering with scikit learn and openCV. The approach here is built for realtime applications using TouchDesigner and python multi-threading.

099
Build 2018.22800

• numpy
• scipy
• sklearn
• cv2

## Overview

A tool for finding Dominant Color with openCV.

Here we find an attempt at locating dominant colors from a source image with openCV and KMeans clustering. The large idea is to sample colors from a source image build averages from clustered samples and return a best estimation of dominant color. While this works well, it’s not perfect, and in this class you’ll find a number of helper methods to resolve some of the shortcomings of this process.

Procedurally, you’ll find that that the process starts by saving out a small resolution version of the sampled file. This is then hadned over to openCV for some preliminary analysis before being again handed over to sklearn (sci-kit learn) for the KMeans portion of the process. While there is a built-in function for KMeans sorting in openCV the sklearn method is a little less cumbersome and has better reference documentation for building functionality. After the clustering process each resulting sample is processed to find its luminance. Luminance values outside of the set bounds are discarded before assembling a final array of pixel values to be used.

It’s worth noting that this method relies on a number of additional python libraries. These can all be pip installed, and the recommended build approach here would be to use Python35. In the developer’s experience this produces the least number of errors and issues – and boy did the developer stumble along the way here.

Other considerations you’ll find below are that this extension supports a multi-threaded approach to finding results.

## Parameters

Dominant Color

• Image Process Status – (string) The thread process status.
• Temp Image Cache – (folder) A directory location for a temp image file.
• Source Image – (TouchDesigner TOP) A TOP (still) used for color analysis.
• Clusters – (int) The number of requested clusters.
• Luminance Bounds – (int, tuple) Luminance bounds, mine and max expressed as value between 0 and 1.
• Clusters within Bounds – (int) The number of clusters within the Luminance Bounds.
• Smooth Ramp – (toggle) Texture interpolation on output image.
• Ramp Width – (int) Number of pixels in the output Ramp.
• Output Image – (menu) A drop-down menu for selecting a ramp or only the returned clusters.
• Find Colors – (pulse) Issues the command to find dominant colors.

Python

• Python Externals – (path) A path to the directory with python external libraries.
• Check Imports – (pulse) A pulse button to check if sklearn was correctly imported.

## Using this Module

To use this module there are a few essential elements to keep in mind.

### Getting Python in Order

If you haven’t worked with external Python Libraries inside of Touch yet, please take a moment to familiarize yourself with the process. You can read more about it on the Derivative Wiki – Importing Modules

Before you can run this module you’ll need to ensure that your Python environment is correctly set-up. I’d recommend that you install Python 3.5+ as that matches the Python installation in Touch. In building out this tool I ran into some wobbly pieces that largely centered around installing sklearn using Python 3.6 – so take it from someone whose already ran into some issues, you’ll encounter the fewest challenges / configuration issues if you start there. Sklearn (the primary external library used by this module) requires both scipy and numpy – if you have pip installed the process is straightforward. From a command prompt you can run each of these commands consecutively:

`pip install numpy`
`pip install scipy`
`pip install sklearn`

Once you’ve installed the libraries above, you can confirm that they’re available in python by invoking python in your command prompt, and then importing the libraries one by one. Testing to make sure you’ve correctly installed your libraries in a Python only environment first, will help ensure that any debugging you need to do in TouchDesigner is more straightforward.

### Working with TouchDesigner

#### Python | Importing Modules

If you haven’t imported external libraries in TouchDesigner before there’s an additional step you’ll need to take care of – adding your external site-packages path to TouchDesigner. You can do this with a regular text DAT and by modifying the example below:

```import sys
mypath = "C:/Python35/Lib/site-packages/mymodule"
if mypath not in sys.path:
sys.path.append(mypath)```

Copy and paste the above into your text DAT, and modify `mypath` to be a string that points do your Python externals site-packages directory.

If that sounds a little out of your depth, you can use a helper feature on the Dominant Color module. On the `Python` page, navigate to your Python Externals directory. It should likely be a path like: `C:\Program Files\Python35\Lib\site-packages`

Your path may be different, especially if when you installed Python you didn’t use the checkbox to install for all users. After navigating to your externals directory, pulse the `Check imports` parameter. If you don’t see a pop-up window then `sklearn`was successfully imported. If you do see a pop-up window then something is not quite right, and you’ll need to do a bit of leg-work to get your Python pieces in order before you can use the module.

#### Using the Dominant Color

With all of your Python elements in order, you’re ready to start using this module.

The process for finding dominant color uses a KMeans clustering algorithm for grouping similar values. Luckily we don’t need to know all of the statistics that goes into that mechanism in order to take full advantage of the approach, but it is important to know that we need to be mindful a few elements. For this to work efficiently, we’ll need to save our image out to an external file. For this to work you need to make sure that this module has a cache for saving temporary images. The process will verify that the directory you’ve pointed it to exists before saving out a file, and will create a directory if one doesn’t yet exist. That’s mostly sanity checking to ensure that you don’t have to loose time trying to figure out why your file isn’t saving.

Give that this process happens in another thread, it’s also important to consider that this functions based on a still image, not on a moving one. While it would be slick to have a fast operation for finding KMeans clusters in video, that’s not what this tool does. Instead the assumption here is that you’re using a single frame of reference content, not video. You point this module to a target source, by dropping a TOP onto the Source Image parameter.

Next you’ll need to define the number of clusters you want to look for. Here the term clusters is akin to what’s the target number of dominant colors you’re looking to find – the top 3, the top 10, the top 20? It’s up to you, but keep in mind that more clusters takes longer to produce a result. You’re also likely to want to bound your results with some luminance measure – for example, you probably don’t want colors that are too dark, or too light. The luminance bounds parameters are for luminance measures that are normalized as 0 to 1. Clusters within bounds, then, tells you how many clusters were returned from the process that fell within your specified regions. This is, essentially, a way to know how many swatches work within the brightness ranges you’ve set.

The output ramp from this process can be interpolated and smooth, or Nearest Pixel swatches. You can also choose to output a ramp that’s any length. You might, for example, want a gradient that’s spread over 100 or 1000 pixels rather than just the discrete samples. You can set the number of output pixels with the ramp width parameter.

On the otherside of that equation, you might just want only the samples that came out of the process. In the Output Image parameter, if you choose `clusters` from the drop down menu you’ll get only the valid samples that fell within your specified luminance bounds.

Finally, to run the operation pulse `Find Colors`. As an operational note, this process would normally block / lock-up TouchDesigner. To avoid that unsavory circumstance, this module runs the KMeans clustering process in another thread. It’s slightly slower than if it ran in the main thread, but the benefit is that Touch will continue running. You’ll notice that `Image Process Status` parameter displays `Processing` while the separate thread is running. Once the result has been returned you’ll `Ready` displayed in the parameter.

`conda install scikit-learn`