Category Archives: Programming

TouchDesigner | Email | Volumetric Lights

dest.PNG

Hello Matthew,
Ive been following your videos about TouchDesigner, great work! Really appreciate the content and learning resources you’ve made.

We are experimenting with TouchDesigner, so that we could perhaps use it in our dynamic / interactive installations.

Recently I have been researching usage of TD with dmx/artnet lighting and have come to a certain problem, which I just cannot solve easily. Ive been looking all over the internet, read two books about TD and still cannot figure this out. So I wanted to get in touch withou you and perhaps aks for advice (if you have time of course 🙂 )

Let me introduce the problem:

Imagine you have a 3D lighting sculpture, where there are DMX light sources all over a certain 3D space.

The lights are not uniformly spaced relatively to each other (no cubic shape or something light that), they are randomly placed in the space.

Now, I want to take a certain shape (for example a sphere) and map it as an lighting effect to the lighting fixtures. The sphere would for example had its radius increased / decreased in time, and the application should “map” which light source should light up when the sphere “crosses” it in space.

I would then somehow sample the color of the points and use that information and feed it to a DMX chop after some other operations…

It’s kinda difficult to explain, but hopefully I got it right.. 🙂

Do you know of any tricks or components I could use, so that I could “blend” 3d geometry with points in space in order to control lighting?

Im certainly able to work out how DMX works and all the other stuff, I just dont know how to achieve the effect in 3D.

(In 2D, it would be really simple. For example for a LED screen, its pretty straightforward, I would just draw a circle or whatever on a TOP and then sample it..)

Thanks a lot,
I would appreciate any tips or advice, really.. 🙂

Best regards,


Great question!

A sphere is a pretty easy place to start, and there are a few ways we can tackle this.

The big picture idea is to sort out how you can compute the distance from your grid points to the center of your sphere. By combining this with the diameter of your sphere we can then determine if a point is inside or outside of that object.

We could do this in SOP space and use a group SOP – this is the most straightforward to visualize, but also the least efficient – the grouping and transformation operations on SOPs are pretty expensive, and while this is a cool technique, you bottle-neck pretty quickly with this approach.

To do this in CHOPs what we need is to first compute the difference between our grid points and our sphere – we can do this with a math CHOP set to subtract. From there we’ll use another math CHOP to compute the length of our vector. In essence, this tells us how far away any given point is from the center of our sphere. From here we have few options – we might use a delete CHOP to remove samples that our outside of our diameter, or we might use a logic CHOP to tell us if we’re inside or outside of our sphere.

From there we should be able to pretty quickly see results.

2D-concept.gif

 

3D-concept.gif

Attached set of examples made in 099.

  • base_SOPs – this illustrates how this works in SOP space using groups
  • base_concept – here you can see how the idea works out with just a flat regular distribution of points. It’s easier to really pull apart the mechanics of this idea starting with a regular distribution first as it’s much easier to debug.
  • base_volume – the same ideas but applied to a 3D volume.
  • base_random – here you can see this process applied to a sudo random distribution of points. This is almost the same network as we looked at in base_concept, with a few adjustments to compensate for the different point density.

base_dist_test

 

presets and cue building – a beyond basics checklist | TouchDesigner 099

from the facebook help group

Looking for generic advice on how to make a tox loader with cues + transitions, something that is likely a common need for most TD users dealing with a playback situation. I’ve done it for live settings before, but there are a few new pre-requisites this time: a looping playlist, A-B fade-in transitions and cueing. Matthew Ragan‘s state machine article (https://matthewragan.com/…/presets-and-cue-building-touchd…/) is useful, but since things get heavy very quickly, what is the best strategy for pre-loading TOXs while dealing with the processing load of an A to B deck situation?

https://www.facebook.com/groups/touchdesignerhelp/permalink/835733779925957/

I’ve been thinking about this question for a day now, and it’s a hard one. Mostly this is a difficult question as there are lots of moving parts and nuanced pieces that are largely invisible when considering this challenge from the outside. It’s also difficult as general advice is about meta-concepts that are often murkier than they may initially appear. So with that in mind, a few caveats:

  • Some of suggestions below come from experience building and working on distributed systems, some from single server systems. Sometimes those ideas play well together, and sometimes they don’t. Your mileage may vary here, so like any general advice please think through the implications of your decisions before committing to an idea to implement.
  • The ideas are free, but the problems they make won’t be. Any suggestion / solution here is going to come with trade-offs. There are no silver bullets when it comes to solving these challenges – one solution might work for the user with high end hardware but not for cheaper components; another solution may work well across all component types, but have an implementation limit. 
  • I’ll be wrong about some things. The scope of anyone’s knowledge is limited, and the longer I work in ToiuchDesigner (and as a programmer in general) the more I find holes and gaps in my conceptual and computational frames of reference. You might well find that in your hardware configuration my suggestions don’t work, or something I suggest won’t work does. As with all advice, it’s okay to be suspicious.

A General Checklist

Plan… no really, make a Plan and Write it Down

The most crucial part of this process is the planning stage. What you make, and how you think about making it, largely depends on what you want to do and the requirements / expectations that come along with what that looks like. This often means asking a lot of seemingly stupid questions – do I need to support gifs for this tool? what happens if I need to pulse reload a file? what’s the best data structure for this? is it worth building an undo feature? and on and on and on. Write down what you’re up to – make a checklist, or a scribble on a post-it, or create a repo with a readme… doesn’t matter where you do it, just give yourself an outline to follow – otherwise you’ll get lost along or forget the features that were deal breakers.

Data Structures

These aren’t always sexy, but they’re more important than we think at first glance. How you store and recall information in your project – especially when it comes to complex cues  – is going to play a central role in how your solve problems for your endeavor. Consider the following questions:

  • What existing tools do you like – what’s their data structure / solution?
  • How is your data organized – arrays, dictionaries, etc.
  • Do you have a readme to refer back to when you extend your project in the future?
  • Do you have a way to add entries?
  • Do you have a way to recall entries?
  • Do you have a way to update entries?
  • Do you have a way to copy entries?
  • Do you have a validation process in-line to ensure your entries are valid?
  • Do you have a means of externalizing your cues and other config data

Time

Take time to think about… time. Silly as it may seem, how you think about time is especially important when it comes to these kinds of systems. Many of the projects I work on assume that time is streamed to target machines. In this kind of configuration a controller streams time (either as a float or as timecode) to nodes on the network. This ensures that all machines share a clock – a reference to how time is moving. This isn’t perfect and streaming time often relies on physical network connections (save yourself the heartache that comes with wifi here). You can also end up with frame discrepancies of 1-3 frames depending on the network you’re using, and the traffic on it at any given point. That said, time is an essential ingredient I always think about when building playback projects. It’s also worth thinking about how your toxes or sub-components use time.

When possible, I prefer expecting time as an input to my toxes rather than setting up complex time networks inside of them. The considerations here are largely about sync and controlling cooking. CHOPs that do any interpolating almost always cook, which means that downstream ops depending on that CHOP also cook. This makes TOX optimization hard if you’re always including CHOPs with constantly cooking foot-prints. Providing time to a TOX as an expected input makes handling the logic around stopping unnecessary cooking a little easier to navigate. Providing time to your TOX elements also ensures that you’re driving your component in relationship to time provided by your controller.

How you work with time in your TOXes, and in your project in general can’t be understated as something to think carefully about. Whatever you decide in regards to time, just make sure it’s a purposeful decision, not one that catches you off guard.

Identify Your Needs

What are the essential components that you need in modular system. Are you working mostly with loading different geometry types? Different scenes? Different post process effects? There are several different approach you might use depending on what you’re really after here, so it’s good start to really dig into what you’re expecting your project to accomplish. If you’re just after an optimized render system for multiple scenes, you might check out this example.

Understand / Control Component Cooking

When building fx presets I mostly aim to have all of my elements loaded at start so I’m only selecting them during performance. This means that geometry and universal textures are loaded into memory, so changing scenes is really only about scripts that change internal paths. This also means that my expectation of any given TOX that I work on is that its children will have a CPU cook time of less than 0.05ms and preferably 0.0ms when not selected. Getting a firm handle on how cooking propagates in your networks is as close to mandatory as it gets when you want to build high performing module based systems.

Some considerations here are to make sure that you know how the selective cook type on null CHOPs works – there are up and downsides to using this method so make sure you read the wiki carefully.

Exports vs. Expressions is another important consideration here as they can often have an impact on cook time in your networks.

Careful use of Python also falls into this category. Do you have a hip tox that uses a frame start script to run 1000 lines of python? That might kill your performance – so you might need to think through another approach to achieve that effect.

Do you use script CHOPs or SOPs? Make sure that you’re being carefully with how you’re driving their parameters. Python offers an amazing extensible scripting language for Touch, but it’s worth being careful here before you rely too much on these op types cooking every frame.

Even if you’re confident that you understand how cooking works in TouchDesigner, don’t be afraid to question your assumptions here. I often find that how I thought some op behaved is in fact not how it behaves.

Plan for Scale

What’s your scale? Do you need to support an ever expanding number of external effects? Is there a limit in place? How many machines does this need to run on today? What about in 4 months? Obscura is often pushing against boundaries of scale, so when we talk about projects I almost always add a zero after any number of displays or machines that are going to be involved in a project… that way what I’m working on has a chance of being reusable in the future. If you only plan to solve today’s problem, you’ll probably run up against the limits of your solution before very long.

Shared Assets

In some cases developing a place in your project for shared assets will reap huge rewards. What do I mean? You need look no further than TouchDesigner itself to see some of this in practice. In ui/icons you’ll find a large array of moviefile in TOPs that are loaded at start and provide many of the elements that we see when developing in Touch:

icon_library.PNG

icon_library_example.PNG

Rather than loading these files on demand, they’re instead stored in this bin and can be selected into their appropriate / needed locations. Similarly, if your tox files are going to rely on a set of assets that can be centralized, consider what you might do to make that easier on yourself. Loading all of these assets on project start is going to help ensure that you minimize frame drops.

While this example is all textures, they don’t have to be. Do you have a set of model assets or SOPs that you like to use? Load them at start and then select them. Selects exist across all Op types, don’t be afraid to use them. Using shared assets can be a bit of a trudge to set up and think through, but there are often large performance gains to be found here.

Dependencies

Sometimes you have to make something that is dependent on something else. Shared assets are a kind of single example of dependencies – where a given visuals TOX wouldn’t operate correctly in a network that didn’t have our assets TOX as well. Dependencies can be frustrating to use in your project, but they can also impose structure and uniformity around what you build. Chances are the data structure for your cues will also become dependent on external files – that’s all okay. The important consideration here is to think through how these will impact your work and the organization of your project.

Use Extensions

If you haven’t started writing extensions, now is the time to start. Cue building and recalling are well suited for this kind of task, as are any number of challenges that you’re going to find. In the past I’ve used custom extensions for every external TOX. Each module has a Play(state) method where state indicates if it’s on or off. When the module is turned on it sets of a set of scripts to ensure that things are correctly set up, and when it’s turned off it cleans itself up and resets for the next Play() call. This kind of approach may or may not be right for you, but if you find yourself with a module that has all sorts of ops that need to be bypassed or reset when being activated / deactivated this might be the right kind of solution.

Develop a Standard

In that vein, cultivate a standard. Decide that every TOX is going to get 3 toggles and 6 floats as custom pars. Give every op access to your shared assets tox, or to your streamed time… whatever it is, make some rules that your modules need to adhere to across your development pipeline. This lets you standardize how you treat them and will make you all the happier in the future.

That’s all well and good Matt, but I don’t get it – why should my TOXes all have a fixed number of custom pars? Let’s consider building a data structure for cues let’s say that all of our toxes have a different number of custom pars, and they all have different names. Our data structure needs to support all of our possible externals, so we might end up with something like:

{
      "cues": {
           "cue1": {
                "Tox": "Swank",
                "Level_1": 0,
                "Noise": 1,
                "Level3": 4,
                "Blacklvl": 0.75
           },
           "cue2": {
               "Tox": "Curl",
               "Bouncy": 0.775,
               "Curve": 100.0,
               "Augment": 13,
               "Blklvl": 0.75
           },
           "cue3": {
               "Tox": "Boop",
               "Boopness": 0.775
           }
      }
}

That’s a bummer. Looking at this we can tell right away that there might be problems brewing at the circle k – what happens if we mess up our tox loading / targeting and our custom pars can’t get assigned? In this set-up we’ll just fail during execution and get an error… and our TOX won’t load with the correct pars. We could swap this around and include every possible custom par type in our dictionary, only applying the value if it matches a par name, but that means some tricksy python to handle our messy implementation.

What if, instead, all of our custom TOXes had the same number of custom pars, and they shared a name space to the parent? We can rename them to whatever makes sense inside, but in the loading mechanism we’d likely reduce the number of errors we need to consider. That would change the dictionary above into something more like:

{
      "cues": {
           "cue1": {
                "Tox": "Swank",
                "Par1": 0,
                "Par2": 1,
                "Par3": 4,
                "Par4": 0.75
           },
           "cue2": {
               "Tox": "Curl",
               "Par1": 0.775,
               "Par2": 100.0,
               "Par3": 13,
               "Par4": 0.75
           },
           "cue3": {
               "Tox": "Boop",
               "Par1": 0.875,
               "Par2": None,
               "Par3": None,
               "Par4": None
           }
      }
}

Okay, so that’s prettier… So what? If we look back at our lesson on dictionary for loops we’ll remember that the pars() call can significantly reduce the complexity of pushing dictionary items to target pars. Essentially we’re able to store the par name as the key, and the target value as the value in our dictionary we’re just happier all around. That makes our UI a little harder to wrangle, but with some careful planning we can certainly think through how to handle that challenge. Take it or leave it, but a good formal structure around how you handle and think about these things will go a long way.

Cultivate Realistic Expectations

I don’t know that I’ve ever met a community of people with such high standards of performance as TouchDesigner developers. In general we’re a group that wants 60 fps FOREVER (really we want 90, but for now we’ll settle), and when things slow down or we see frame drops be prepared for someone to tell you that you’re doing it all wrong – or that your project is trash.

Waoh is that a high bar.

Lots of things can cause frame drops, and rather than expecting that you’ll never drop below 60, it’s better to think about what your tolerance for drops or stutters is going to be. Loading TOXes on the fly, disabling / enabling containers or bases, loading video without pre-loading, loading complex models, lots of SOP operations, and so on will all cause frame drops – sometimes big, sometimes small. Establishing  your tolerance threshold for these things will help you prioritize your work and architecture. You can also think about where you might hide these behaviors. Maybe you only load a subset of your TOXes for a set – between sets you always fade to black when your new modules get loaded. That way no one can see any frame drops.

The idea here is to incorporate this into your planning process – having a realistic expectation will prevent you from getting frustrated as well, or point out where you need to invest more time and energy in developing your own programming skills.

Separation is a good thing… mostly

Richard’s killer post about optimization in touch has an excellent recommendation – keep your UI separate. This suggestion is HUGE, and it does far more good than you might intentionally imagine.

I’d always suggest keeping the UI on another machine or in a seperate instance. It’s handier and much more scaleable if you need to fork out to other machines. It forces you to be a bit more disciplined and helps you when you need to start putting previz tools etc in. I’ve been very careful to take care of the little details in the ui too such as making sure TOPs scale with the UI (but not using expressions) and making sure that CHOPs are kept to a minimum. Only one type of UI element really needs a CHOP and that’s a slider, sometimes even they don’t need them.

I’m with Richard 100% here on all fronts. That said, be mindful of why and when you’re splitting up your processes. It might be temping to do all of your video handling in one process, that gets passed to process only for rendering 3d, before going to a process that’s for routing and mapping.

Settle down there cattle rustler.

Remember that for all the separating you’re doing, you need strict methodology for how these interchanges work, how you send messages between them, how you debug this kind of distribution, and on and on and on.

There’s a lot of good to be found how you break up parts of your project into other processes, but tread lightly and be thoughtful. Before I do this, I try to ask myself:

  • “What problem am I solving by adding this level of additional complexity?”
  • “Is there another way to solve this problem without an additional process?”
  • “What are the possible problems / issues this might cause?”
  • “Can I test this in a small way before re-factoring the whole project?”

Don’t Forget a Start up Procedures

How your project starts up matters. Regardless of your asset management process it’s important to know what you’re loading at start, and what’s only getting loaded once you need it in touch. Starting in perform mode, there are a number of bits that aren’t going to get loaded until you need them. To that end, if you have a set of shared assets you might consider writing a function to force cook them so they’re ready to be called without any frame drops. Or you might think about a way to automate your start up so you can test to make sure you have all your assets (especially if your dev computer isn’t the same as your performance / installation machine).

Logging and Errors

It’s not much fun to write a logger, but they sure are useful. When you start to chase this kind of project it’ll be important to see where things went wrong. Sometimes the default logging methods aren’t enough, or they happen to fast. A good logging methodology and format can help with that. You’re welcome to make your own, you’re also welcome to use and modify the one I made.

Unit Tests

Measure twice, cut once. When it comes to coding, unit tests are where it’s at. Simple proof of concept complete tests that aren’t baked into your project or code can help you sort out the limitations or capabilities of an idea before you really dig into the process of integrating it into your project. These aren’t always fun to make, but they let you strip down your idea to the bare bones and sort out simple mechanics first.

Build the simplest implementation of the idea. What’s working? What isn’t? What’s highly performant? What’s not? Can you make any educated guesses or speculation about what will cause problems? Give yourself some benchmarks that your test has to prove itself against before you move ahead with integrating it into your project as a solution.

Document

Even though it’s hard – DOCUMENT YOUR CODE. I know that it’s hard, even I have a hard time doing it – but it’s so so so very important to have a documentation strategy for a project like this. Once you start building pieces that depend on a particular message format, or sequence of events, any kind of breadcrumbs you can leave for yourself to find your way back to your original thoughts will be helpful.

GLSL TD Tutorials in TouchDesigner

screen-shot.png

As a follow up to the Book of Shaders port last week, I wanted to add another resource that I read through several times when first getting my bearings with GL. GLSL 2D Tutorials – an example that’s currently up on Shader Toy https://www.shadertoy.com/view/Md23DV.

From Uğur:

by Uğur Güney. March 8, 2014.

Hi! I started learning GLSL a month ago. The speedup gained by using GPU to draw real-time graphics amazed me. If you want to learn how to write shaders, this tutorial written by a beginner can be a starting place for you.

Please fix my coding errors and grammar errors. :-)

Getting your bearings with GLSL can be a bit of a rodeo when you’re first getting started. Uğur’s 2D tuts were a huge help to me when I was first getting started, and they often show examples that are a little more granular than The Book of Shaders.

Hopefully this set of examples will help you get started and get your gl bearings here in Touch.

When possible, I’ve copied the examples as faithfully as possible. What that means is that there may be better ways to approach some challenges – but what you’ll find here is as close to the original tutorial as I can manage.

GLSL 2D Tutorials in TouchDesigner
https://github.com/raganmd/glsl2dTuts-in-TouchDesigner

The Book of Shaders in TouchDesigner

BOS-screen-shot

I can’t say enough good things about The Book of Shaders by Patricio Gonzalez Vivo & Jen Lowe. If you’re looking to get a handle on how to write shaders, or find some inspiration this is an incredible resource.

For TouchDesigner programmers who are accustomed to the nodal environment of TD, working with straight code might feel a bit daunting – and making the transition from Patrico’s incredible resource to Touch might feel hard – it certainly was for me at first. This repo is really about helping folks make that jump.

Here you’ll find the incredible examples made by Patricio and Jen ported to the TouchDesigner environment. There are some differences here, and I’ll do my best to help provide some clarity about where those come from.

This particular set of examples is made in TouchDesigner 099. In the UI you’ll find a list of examples below the rendered shader in the left pane, on the right you’ll find the shader code and the contents of an info DAT. You can live edit the contents of the shader code, you just have to click off of the pane for the code to be updated under the hood. If you hit the escape key you can dig into the network to see how everything is organized.

Each ported shader exists as a stand alone file – making it easy to drop the pixel shader into another network. When possible I’ve tried to precisely preserve the shader from the original, though there are some cases where small alterations have been made. In the case of touch specific uniforms I’ve tried to make sure there are notes for the programmer to see what’s happening.

Download the repo and start getting your GL on.

https://github.com/raganmd/BOS-in-TouchDesigner

presets and cue building | TouchDesigner 099

I’ve been thinking about state machines and cuing systems lately. Specifically, that there aren’t many good resources that I’ve found so far that talk new artist programmers through how to think about or wrestle with these ideas. Like many Touch programmers I’ve tried lots of different ways of thinking about this problem, and just today I saw someone post on the Facebook help group.

from the facebook help group:

Hi, i’m working arround Matthews AME394 Simple VJ-Setup Tutorial. No Questions, but how can i do nearly the same with different blending times between the moduls. I tried a lot with getting different values out of a table DAT into the length parameter of a timerCHOP. But cannot figur out the right steps to get my goal. Any helps? this i need in a theater situation with different scenes to blend one after another with scenebuttons or only one button and a countCHOP or something else.

This challenge is so very familiar, and while there are lots of ways to solve this problem sometimes the hardest part is having an idea of where to start.  Today what I want to look at is just that – where do we start? This isn’t the best solution, or the only solution – it’s just a starting point solution. It’s a pass at the most basic parts of this equation to help us get started in thinking about what the real problems are, how we want to tackle them, and how we can go about exposes the real issues we need to solve for.

So where do we start? In this simple little state machine we’re going to start with a table full of states. For the sake of simplicity I’m going to keep this as simple as possible… though it might well uncover the more interesting and more challenging pieces that lie ahead.

I’m going to start with the idea that I’ve got a piece of content (an image or a movie) that I want to play. I want to apply some post process effects (in this case just black level and image inversion changes), and I want to have different transition times between these fixed states. Here the only transition I’m worrying about is one that goes from one chain of operations to another. I’m also going to stick with images for now.

So what do we need in our network to get started?!

We’re going to borrow from an idea that often gets used in these kinds of challenges, and we’re going to think of this as operating with two decks – an A deck, and a B deck. Our deck is essentially a chain of operators that allow for all of the possibilities that we might want to explore in our application. In this case I’m only working with a level TOP, but you can imagine that we might use all sorts of operations to make for interesting composition choices.

Alright, so we’re going to lay out a quick easy deck:

moviefilein > level > fit 

adeck.PNG

Next we’re going to repeat this whole chain, then connect both of our fit TOPs to a cross TOP:

ab_deck.PNG

If you’re scratching your head at this fit TOP in line, that’s okay. For us, the fit TOP is going to act as our safety. This makes sure that no matter what the resolution of the incoming file might be, that we always make sure that both decks have matching proportions. We’d probably want a little more thought in how this would work for an event or a show, but for now this is enough to help ensure that don’t experience any unexpected resolution shifts during our transitions.

Next we’re going to add a simple tweening system to our network to control how we blend between states. In this case I’m going to use a constant, a speed, and a null. I need to make sure that my speed is set to clamp, and that my min and max values are 0 and 1 respectively. Right now I only have two different decks, so I don’t want to go any higher that 1 or any lower than 0.

Now we’re cooking with propane! So where do we go next?

some simple cues

movie_file trans_time blk_lvl invert
Banana.tif 1 0 0
Butterfly1.tif 2 0.12 1
Butterfly5.tif 5 0.2 0
Mettler.2.jpg 10 0.05 0
OilDrums.jpg 0.5 0.25 1
Starfish.tif 1 0 1

In this simple examination of this challenge I’m going to use a table to store our cues. In a larger system I’d probably use python storage (which is really a dictionary), but for the sake of keeping it simple let’s start with just a table. Our simple cues are organized above, and we can put all of those values into a table DAT. You’ll notice that for now I’m only worrying about file name and not path – all of these files come from the same directory so we can treat them mostly the same way. We’ll also notice that I’m thinking of my transition times in terms of seconds. All of this can, of course, be much more complicated. The trick is to sort out a very simple example first to identify pressure points and challenges before you dig yourself into a hole.

Okay, let’s add a table DAT to our network and copy all over our cues over.

table_dat.PNG

Now that we have all of our pieces organized it is time to think through the logic of how we make this all work. For now let’s use a button, a count CHOP, and a CHOP Execute DAT. We need to make sure our button is set to be momentary, and we also need to make sure our count CHOP is set to loop – starting at 1 and ending at 6. That matches our row indices from our table DAT.

move-through-cues.PNG

This is great Matt, but why python?

Well, we could do a lot of this with a complex set of CHOPs and selects but these kinds of states tend to be better handled, logically at least, through written code. Python will let us explicitly describe exactly what happens, and in what order those things happen. That’s no small thing, and while it might be a little rocky to wrap your head around using Python in Touch at first, it’s well worth it in the end. So what do we write in our CHOP Execute?

a little bit of logic | python

# me - this DAT
# 
# channel - the Channel object which has changed
# sampleIndex - the index of the changed sample
# val - the numeric value of the changed sample
# prev - the previous sample value
# 
# Make sure the corresponding toggle is enabled in the CHOP Execute DAT.

attr       = op( 'constant_attr' )
deck       = op( 'null_deck' )[ 'trans' ]

deckA      = op( 'moviefilein_a' )
levelA     = op( 'level_a' )
deckB      = op( 'moviefilein_b' )
levelB     = op( 'level_b' )

cues       = op( 'null_cues' )
path       = app.samplesFolder + '/Map/{file}'

def onOffToOn(channel, sampleIndex, val, prev): 
    return

def whileOn(channel, sampleIndex, val, prev): 
    return

def onOnToOff(channel, sampleIndex, val, prev):
    return

def whileOff(channel, sampleIndex, val, prev): 
    return

def onValueChange(channel, sampleIndex, val, prev): 
    
    # we're in deckB, change A 
    if deck > 0.5: 
        deckA.par.file         = path.format( file = cues[ int( val ), 'movie_file' ] ) 
        levelA.par.blacklevel  = cues[ int( val ), 'blk_lvl' ] 
        levelA.par.invert      = cues[ int( val ), 'invert' ] 
        attr.par.value0        = float( 1 / cues[ int( val ), 'trans_time' ] ) * - 1

    else: 
        deckB.par.file         = path.format( file = cues[ int( val ), 'movie_file' ] )
        levelB.par.blacklevel  = cues[ int( val ), 'blk_lvl' ] 
        levelB.par.invert      = cues[ int( val ), 'invert' ] 
        attr.par.value0        = 1 / cues[ int( val ), 'trans_time' ]
    
    return

Uhhhhhhh… wait. What?

Okay. First off we just define a set of variables that we’re going to use. This makes our code a little easier to write, and easier to read. Next all of the action really happens in our onValueChange function.

We’re going to do all of this in a little logical if statement. If this thing, do that thing… in all the other cases, do something else.

First we check to see what our deck position is… which means that we check to see which output we’re currently seeing more of. If our cross TOP’s index is greater that 0.5 we know that we’re closer to 1, which also means we’re seeing more of deck B than deck A. That means that we want to make changes in deck A before we start transitioning. First we change our file, change all of our settings, then finally set a value in our constant CHOP. But why 1 / that value? And why multiplied by -1?

A default network runs at 60 fps. A speed CHOP fed by a constant with a value of 1 will rise a count of 1 over 60 frames. Said another way, an input value of 1 to our speed in a default network will increase by a count of one every second. If we divide that number in half we go twice as slow. A value of 0.5 will increase by a count of 1 every 2 seconds. 1 / our table value will let us think in seconds rather than in fractions while we’re writing our cues. Yeah, but what about being multiplied by -1?! Well, if we want to get back to the 0 index in our cross TOP we need a negative value feeding our speed CHOP. Multiplying by -1 here means that we don’t need to think about the order of cues in our table DAT, and instead our bits of Python will keep us on the rails. Our else statement does all of the same things, but to our B deck. It also uses a positive value to feed our speed CHOP – since we need an increasing value.

There you have it, a simple cuing system.

simple cues.gif

This is great Matt, but what if I want to tween settings on that level TOP? Or any other set of complicated things?! Well, I’d be that at this point you’ve got enough to get you started. You might use a column to indicate if you’re transitioning to a totally new cue or just to new values in the same source image. You could also choose to put your parameter values in CHOPs instead so you could manipulate them with other CHOPs before exporting them to your decks.

What if I don’t want linear transitions?! A speed is just a linear ramp! That’s okay. You might choose to use a lookup CHOP and a more complicated curve. You could even make several types of curves with animation COMPs and switch between them. Or you could use a lag  CHOP to change your attack and release slopes. Or you could use a trigger CHOP, or a fileter CHOP. There are lots of ways to shape curves with math, now it’s up to you to figure out exactly what you’re after.

Happy programming!

pull it apart

Pull apart the example in 088
Pull apart the example in 099

Maintaining Perspective with Multiple Cameras | TouchDesigner

File this away under “interesting theoretical concepts that I’ll never use… or will I?”

At some point while making realtime generative art for massive installations you may find that you’re beyond the capabilities of traditional realtime rendering in Touch. Let’s say, for example, that you need to render 12 hd outputs for a 3 x 4 array of screens – a resolution of 7680 x 3240 certainly can be done with a single render TOP, but delivering that final texture is a little more tricky.

I’m well aware that there are a number of possible solutions to this problem but before you find yourself composing that email to me about how to hack a way to a solution… what if it wasn’t 12 ouputs, what if it was 120? 200? What if every output was 4k? The answer we’re really after here is how to draw a scene with consistent perspective across multiple machines… because at some point you’ll have to use multiple machines. So, what do we do?

Forget what we do… what does that even look like? I’m still so confused.

Okay, so let’s first look at some examples of what it looks like as reference.

In this example we can see one large canvas that spans multiple screens. This is great – it’s huge and beautiful. This example shows a large desktop, which is also great… but what if we’re after some real-time rendering? This is a great illustration of the problem we might encounter. What if these displays were all 1920 x 1080. It’s a 7 x 4 array, so that’s going to be a definite challenge for a complex scene on a single machine. At this point we probably can’t realistically produce a single pixel to pixel texture for this array on a single machine. Instead we’d have to have a system of distributed rendering machines. Okay, that’s pretty straightforward and we can do some hip flat rendering that’s all orthographic no problem. What if we want perspective? If you want perspective in your real time rendering you need a way to conceptualize what the entire “screen” is, and then how to selectively render just a portion of that larger scene.

Huh?

Consider the beautiful work of Refik Anadol. I can’t speak to exactly what technique is being used here, but it’s a good illustration of the same challenge. How can you maintain the illusion of perspective if you need to render your generative art on multiple machines? That’s the real question we’re trying to answer… and now we can look at some ideas to help us better understand that challenge.

The process and methodology described below aim to solve that problem. For this example I’m going to work in a scale that’s unrealistic… but will allow those without a commercial license to play along from home. If you have a commercial license feel free to turn up the resolution as long as you keep mathematics involved in mind.

First things first, let’s build out a simple proof of concept that will make sure we understand this problem more completely.

Let’s imagine that we have a large composition that we need to cut up (for the sake of rendering) into 4 smaller slices. That might look something like this:

multi_perspective

Remember, this is just a proof of concept so we’re going to start with a very easy implementation first, before we start to dig into the more complex questions. An important lesson to consider when it comes to programming is to start by reducing the problem to its basic elements, then when you have a foundational understanding of the issue start to scale up – don’t worry, we’ll get there we just have to start small.

Okay, so we’ve got our 2 x 2 array that we want to render. Let’s see how we can set up some cameras to render just one of those squares a piece, but all from the same point of view.

Wait! Why do they need the same point of view? We’re after the same point of view so we can maintain perspective. It’d be easy enough to use four different cameras that were translated into positions to only see their section of the larger quad, but the results from lighting and perspective calculations wouldn’t match. You can use this multi-camera transformation technique if you’re doing orthographic rendering with emissive lighting, but not if you want to maintain perspective and use non-emissive lighting. It’s okay if you don’t believe me – I didn’t believe me either, and I had to set it up and test it bunch of times before I really understood what’s happening.

What’s that going to look like? Well, for the perspective of a single camera that can see the whole scene – the effect we want to recreate eventually – we might see something like this:

full_camera_view.PNG

From the vantage point of a single camera, we want to be able to zero in on just a single quadrant in our scene, something like this:

single_quad.PNG

Eventually, we want to reassemble the view of four cameras to look like our original single camera view.

Matt, I still don’t get it. That’s okay. Keep reading, and if by the time we get done with this example it’s still not a useful technique you can stop reading. If, however, you want a means to do perspective based illusions across multiple machines for massive installations, keep reading to the end.

We’re going to set up our test by by using a part of our camera COMP that you may not have used before. Specifically, when it comes to the view page, we’re going to use the Viewing Angle and Method called “Focal Length and Aperture.” I wish I could tell you exactly what this means to TouchDesigner – spoiler, I can’t – what I can help you understand is how these values relate to one another in order to achieve our particular illusion.

We’re going to start by setting up a simple example. Add a geo to your scene, and replace the torus inside with a grid. Set your sizex parameter to be 16/9. If you want to follow along step for step , change your grid to be a polygon, and add a noise SOP with a period of 0.02 and an amplitude of 0.5. Connect that to a facet SOP with unique points and computed normals. Connect your chain of operators to a null SOP and make sure that your display and render flag are turned on for your null. You should have a simple network that looks like this:

SOPs.png

Outside of your geo add two moviefile in TOPs. In the first you can use the supplied quad arrangement in the assets portion of the git repository that accompanies this post. It’s called multi_perspective.png.In your second moviefile in TOP select the FiledGuide.tif. Composite these two together with a composite TOP, and change the operand method to Add. Connect this to a null TOP, and finally assign this null to a phong material as the color map. When you’re done you should have something that looks like this:

texture_grid.png

Whew. Alright, now we can finally get to the interesting part. Let’s add a light and a camera to our network.

Now, in our camera comp let’s set the tz parameter to 10 units:

cam_tz.PNG

Next let’s move over the view page of the camera COMP. Here we’re going to leave the projection as perspective, but we’re going to change the viewing angle method to “Focal length and Aperture. Next we’ll change our Focal Length to 10 and our aperture to 16/9:

full_view_cam

This is our camera that can see the entire scene. Let’s add a render TOP to our network so we can see what our camera sees:

full_view_cam_view

If we were to bypass our noise SOP the view of this piece of geometry would fit exactly within our view-port. From here forward it’s going to get a little interesting. We want to maintain the perspective calculations from this vantage point, but we want only a single quadrant at a time to fill our view-port. It’s almost like zooming in and cropping to only a sub-section of our view. How do we do that?

Let’s copy our first camera, and then make a few adjustments. I’m going to call my new camera cam_single_p1. Next we’re going to leave our Focal Length and Aperture settings just as they were. We are, however, going to change our window x/y parameters to be:

winx -0.25
winy 0.25

We’re also going to change our window size to be 0.5.

cam_single_p1

Let’s render that camera and see what we get:

single_cam_p1_rendered.PNG

Woah!! That works just like we wanted! Thinking through our other cameras, we can quickly see that the combination of our windows size and offsets act as a zoom and translation mechanism. Try adding 3 more cameras with the following tx/y settings:

Cam2

  • tx 0.25
  • ty 0.25

Cam3

  • tx -0.25
  • ty -0.25

Cam4

  • tx 0.25
  • ty -0.25

If you render these cameras you should see something like this:

all_cams.PNG

Okay. That’s all pretty slick, but how do all of these parameters relate?

What does it mean?!

The meat and potatoes of this technique is to define a view-port’s aspect ratio, the number of vertical slices that a single window represents, and then to specify where the offsets sit that represent the center of a given window.

Huh?!

Let’s think through our simple example. We made our rendered quad a 1.7778 x 1 rectangle – a rectangle with a 16:9 aspect ratio. This was the same value we used for our aperture. We also set the distance of our camera from our geo to be 10 units, which was the same value we used to define our focal length. The window size is a ratio of 1 over the number of vertical sections… in our case we had two vertical sections, so 1 over 2 is 0.5. Our xy window offsets represent the center of our windows in our sections. That’s a little harder to wrap our heads around, and a better way to think about it is the UV coordinates of the center of a given window into our scene. Let’s break that out a little more.

Window Size

  • 1 / number of vertical windows that can fit within our viewport

Focal Length

  • the distance of our camera to our scene window

Aperture

  • the aspect ratio of our scene window

Win X

  • ( U * 2) – 1 ) / 2

Win Y

  • – ( ( U * 2) – 1 ) / 2 )

To really dig into the power of this technique we need to push beyond just a quad based set up we need a more abstract configuration of windows in our scene. Now that we understand the mechanics of our set up, let’s look at some arbitrary configuration of windows that might be spread across a large number of machines. What if our window arrangement looked something like the below:

multi_cam_perspective_output_map

What’s going on here?

Well,  let’s imagine that you’re working on a large format LED installation where you need to slice up your scene into uniform HD chunks that then feed an LED controller. In some cases those regions overlap even though only a single LED screen is outputting the content. Relationally, however, you still need to be able to control and designate the regions for the screens. Output 5 and 7 are a good example of these. The final shape of that screen is going to be the combined outline of the two displays, but the video feed to the LED controllers needs to be consistent HD cutouts. All of the dimensions in this example could probably be managed by doing a single rendering of the full scene then cropping to a given region – but at some point your ability to render the full scene and then use TOPs to crop out pixels is going to fail. This example has 7 outputs, but it’s not hard to imagine a project that had 20 or more – as reference, the need to understand this technique came out of working on an installation with 68 discrete outputs.

Okay, okay, okay… fine. So what are all these targets and numbers about?

We’ll remember in our first proof of concept example that we were able to take advantage of a simple 0.25 offset – that makes sense right?

If our geometry is placed with it’s bottom left corner at the origin (0,0), then the center of our first slice is going to be at (0.25, 0.25):

2016-12-13 13.35.19.jpg

That’s great Matt, but that doesn’t make any sense… following this logic, our slices would have been more like:

  • Slice1 ( 0.25, 0.75 )
  • Slice2 ( 0.75, 0.75 )
  • Slice3 ( 0.25, 0.25 )
  • Slice4 ( 0.75, 0.25 )

So what gives?!

Well, we have to remember that we set up our geometry to have it’s center at the origin. If we take this into account, what we see is more like:

2016-12-13 13.36.58.jpg

Looking at this, it should make sense why we used the translation coordinates that we did. The other interesting thing to understand in this look into our geometry is to consider the following.

If the full extent of our scene represents the bounds of a complete window, then we can begin to think about our winx and winy coords as being more akin to a UV – a normalized coordinate on our full window. We need to do some additional math to compensate for the translation of our origin, but that’s pretty straightforward.

If you’re still scratching your head, that’s okay. Let’s look at how to take our new window map and create a programmatic means of slicing up that full scene.

Thinking back to our proof of concept test we need a few things in place in order for this all to work as we expect:

  • We need the transforms for all our cameras to be the same – these are on the xform page: tx, ty, tz.
  • We also need several pieces for the view page:
    • Focal Length
    • Aperture
    • Winx
    • Winy
    • Winsize

Given that our cameras aren’t going to be doing any moving once we set up our calibration, we can safely use python expressions in our parameter fields. In terms of optimization, if an op does a lot of cooking it’s often better to use exports instead of expressions – expressions end up getting complied on demand and evaluated when op op cooks. Since we want fixed cameras looking into a moving world, we can use expressions for our cameras – it’s also going to be less of a hassle to set up, which is great.

For starters let’s add our reference template into our scene. We can start by dragging in our texture, connecting a Null TOP, and then assigning this to the color of a constant Material.

reference_plate.PNG

Next let’s add a geometry to our scene. We can replace the torus inside with a rectangle that has our source window’s aspect ratio, which in this case is 16:9 or 1.778 : 1.

scene_window.PNG

Next I’m going to use a Null COMP to hold the transforms of our camera system. I’m going to set this to have a tz value of 5, and otherwise leave this alone.

Null_pars.PNG

Let’s also add an object CHOP to help us with determining the distance between the null and geometry – to be clear, we don’t need to do this with an object CHOP, we could do this with a math CHOP, or with Python.

In the Object CHOP I’m going to set our null as the target object, geo1 as the reference object, and I’m going to set this to compute distance.

object_chop.png

I’m also going to add some tables that hold some reference information for us. I want to know the width and height of our scene, as well as the width and height of a given cut-out.

ref_values.PNG

We’re also going to need a table with all of our pixel space coordinates:

output_coords.PNG

Now that we have all our primary ingredients ready, we can build out a system to convert our pixel space coordinates into a set of winx and winy translation values.

Let’s start by looking at this process in general. We can first start with our coords in pixel space:

Pixel Space

output x y
output1 640 360
output2 205 130
output3 237 518
output4 525 130
output5 752 593
output6 1013 188
output7 1012 483

These values represent the actual center of these windows in the full pixel scene. I started by making this template in Photoshop, and then measured the location of the center of each given viewport.

Once we have these values, we need to convert them into a normalized values. In other words, how do these pixel coordinates translate to UV coordinates. This is a pretty straightforward calculation – the pixel value divided by it’s respective dimension for the full scene:

  • outputx / full_scene_x
  • outputy / full_scene_y

We can set up a quick eval DAT to do all of this for us:

convert_to_uv

The two python expressions that drive this in the table2 DAT are:

me.inputCell / op( 'table_scene' )[ 'scene_x', 1 ]
me.inputCell / op( 'table_scene' )[ 'scene_y', 1 ]

Our results from this can be found in the table below.

UV Space

output x y
output1 0.5 0.5
output2 0.16015625 0.18055555
output3 0.18515625 0.71944444
output4 0.41015625 0.18055555
output5 0.5875 0.82361111
output6 0.79140625 0.26111111
output7 0.790625 0.67083333

Now that we have the UV coords that represent the center of each window, we need to convert these values into a scale that takes into account that the center of our geometry is located at the origin. For our x values we multiply our value by 2, subtract 1, and divide by 2. For our y values we use the same operation and multiply by negative 1. We can use another eval DAT to do just this for us.

convert_to_winxy.PNG

The two python expressions that drive this in table3 are:

( ( me.inputCell * 2 ) - 1 ) / 2
( ( ( me.inputCell * 2 ) - 1 ) / 2 ) * -1

Now we have taken our original pixel coords and then converted them into our winx and winy transforms.

Converted into winx winy transfroms

output x y
output1 0.0 -0.0
output2 -0.33984375 0.31944444
output3 -0.31484375 -0.21944444
output4 -0.08984375 0.319444444
output5 0.08750000 -0.323611111
output6 0.29140625 0.238888888
output7 0.290625 -0.170833333

With these values set, we just need to make sure that we compute our window size, aperture, and focal_length. Looking back to the above, calculating these remaining values should be a snap.

Window Size

Window size is 1 over the number of windows that can fit into our full scene. This also means that our total scene’s width should be n x the width of a given window.

  • In our case a single window (measured in Photoshop) is 320 pixels, and our full scene is 1280 pixels.
  • 1280 / 320 = 4
  • 1 / 4 = 0.25
  • Window Size = 0.25

Focal Length

Focal Length is the distance between our point of view (in our case the null), and our full scene. We’ve used an object CHOP to compute this distance.

Aperture

Our aperture is the aspect ratio of our full scene.

  • 1280/720 = 1.778
  • Aperture = 1.778

I’m using an eval DAT to do the computation and organization of all of this:

cam_attr_calculations.PNG

The python for these looks like this:

1 / ( op( 'table_scene' )[ 'scene_x', 1 ] / op( 'table_render_attr' )[ 'width', 1 ] )
op( 'table_scene' )[ 'scene_x', 1 ] / op( 'table_scene' )[ 'scene_y', 1 ]
op( 'object1' )[ 'dist' ]

Python in touch is name dependent, so I’d recommend looking at this example network if you’re trying to replicate this effect.

Now we can set up our cameras to correctly crop out a given viewport of the entire scene. I’m going to rely on the digits of a camera to be correspondences to the output. So in my case output1 and camera1 should be the same thing. I’m also going to use the translation values of our null to set the location of the camera.

All of that said, our expressions for our camera should look like this:

camera_expressions.PNG

Again, all of our expressions are name dependent, so I’d recommend looking over how I’ve organized this in the sample file in order to make sure you know exactly what’s referencing what.

Now we can copy past our camera 6 more times – I’m using digits in many of these expressions to match the camera digit to the output digit. Looking over my results it looks like we’re right on the money.

view_ports.PNG

To really appreciate what’s happening here, I’m going to turn off the rendering on our calibration plate, and turn on a sample piece of geometry.

view_ports_real_geo.PNG

In geo2 you can see the entire geometry, and in each of our viewports we can see that we’re only rendering the region of our geometry that falls into single view.

“That’s a mess… I don’t get it.” You might well be saying. That’s okay. Let’s rearranged our TOPs to mirror more closely what’s in our template:

view_ports_real_geo_rearranged.PNG

Hopefully, doing this we should better be able to see how these various pieces work together.

At this point we now have a means of rendering a complex scene across multiple machines (or GPUs if you’re able to use affinity on Quadro cards), and maintain perspective. That unlocks a whole new avenue for realtime rendering that breaks you away from the limitations of single machine configurations or reliance on baked media for distributed realtime rendering.

Download this example from github


It’s always lovely to get an email from Derivative headquarters.

In this case I just got a lovely ping from Malcolm to let me know that the crop parameters on the render TOP can be used for the same functions described above.

Let’s look back at our first example to understand how that might work. In this case I’m going to use the same initial camera that we set up – our cam_full_scene COMP. I’m going to use this one already since I know that it’s correctly configured to capture the entire width and height of our reference plate. Next I’m going to add a render TOP, and under the CROP page I’m going to change my crop right and crop bottom pars to 0.5. For the sake of understanding the concept I’m going to leave this in a fractional unit space, but we could just as easily determine these values as absolute pixel measures. The result of this looks like this:

render_crop1.PNG

Next up, rather than using another render TOP I’m going to use a render pass – there’s lots of good reasons to use the render pass but one of the most important considerations here is that it’s a very efficient rendering operation. We do, however, need to make a few other adjustments. We need to target our render2 as our render TOP on the Render Pass page, and we also need to toggle on clear to camera color, and clear depth buffer:

render_crop2.PNG

On our render pass we’ll need to use the following crop parameters:

render_crop2_2.PNG

As we add additional render pass tops we need to target the previous render pass – a given render or render pass TOP can only have a single render pass assigned to it.

Our crop parameters should look like this:

renderpass3

  • crop left 0.0
  • crop right 0.5
  • crop bottom 0.0
  • crop top 0.5

renderpass4

  • crop left 0.5
  • crop right 1.0
  • crop bottom 0.0
  • crop top 0.5

All in all when we’re done we should have something that looks like this:

render_crop_full.PNG

Here the result is the same, the methodology going into it all is just a little different. Like all things TouchDesigner, there are multiple means of solving the same challenge and the “right” one ultimatly comes down to the choice that’s best for your particular installation.

Happy Programming everyone.

* The git repository and support files have been updated to reflect this additional material.

Building a Calibration UI | Reusing Palette Components – The Stoner | TouchDesigner

Here’s our second stop in a series about planning out part of a long term installation’s UI. We’ll focus on looking at the calibration portion of this project, and while that’s not very sexy, it’s something I frequently set up gig after gig – how you get your projection matched to your architecture can be tricky, and if you can take the time to build something reusable it’s well worth the time and effort. In this case we’ll be looking at a five sided room that uses five projectors. In this installation we don’t do any overlapping projection, so edge blending isn’t a part of what we’ll be talking about in this case study

stoner

As many of you have already found there’s a wealth of interesting examples and useful tools tucked away in the palette in touch designer. If you’re unfamiliar with this feature, it’s located on the left hand side of the interface when you open touch, and you can quickly summon it into existence with the small drawer and arrow icon:

pallet

Tucked away at the bottom of the tools list is the stoner. If you’ve never used the stoner it’s a killer tool for all your grid warping needs. It allows for key stoning and grid warping, with a healthy set of elements that make for fast and easy alterations to a given grid. You can bump points with the keyboard, you can use the mouse to scroll around, there are options for linear curves, bezier curves, persepective mapping, and bilinear mapping. It is an all around wonderful tool. The major catch is that using the tox as is runs you about 0.2 milliseconds when we’re not looking at the UI, and about 0.5 milliseconds when we are looking at the UI. That’s not bad, in fact that’s a downright snappy in the scheme of things, but it’s going to have limitations when it comes to scale, and using multiple stoners at the same time.

stoner

That’s slick. But what if there was a way to get almost the same results at a cost of 0 milliseconds for photos, and only 0.05 milliseconds when working with video? As it turns out, there’s a gem of a feature in the stoner that allows us to get just this kind of performance, and we’re going to take a look at how that works as well as how to take advantage of that feature.

stoner_fast

Let’s start by taking a closer look at the stoner itself. We can see now that there’s a second outlet on our op. Let’s plug in a null to both outlets and see what we’re getting.

stoner_nulls

Well hello there, what is this all about?!

Our second output is a 32 bit texture made up of only red and green channels. Looking closer we can see that it’s a gradient of green in the top left corner, and red in the bottom right corner. If we pause here for a moment we can look at how we might generate a ramp like this with a GLSL Top.

glsl_vuvst

If you’re following along at home, let’s start by adding a GLSL Top to our network. Next we’ll edit the pixel shader.

out vec4 fragColor;

void main()
{
 fragColor = vec4( vUV.st , 0.0 , 1.0 );
}

So what do we have here exactly? For starters we have an explicit declaration of our out vec4 (in very simple terms – our texture that we want to pass out of the main loop); a main loop where we assign values to our output texture.

What’s a vec4?

In GLSL vectors are a data type. We use vectors for all sorts of operations, and as a datatype they’re very useful to us as we often want variable with several positions. Keeping in mind that GLSL is used in pixeltown (one of the largest burrows on your GPU), it’s helpful to be able to think of variables that carry multiple values – like say information about a red, green, blue, and alpha value for a given pixel. In fact, that’s just what our vec4 is doing for us here, it represents the RGBA values we want to associate with a given pixel.

vUV is an input variable that we can use to locate the texture coordinate of a pixel. This value changes for every pixel, which is part of the reason it’s so useful to us. So what is this whole vec4( vUV.st, 0.0, 1.0) business? In GL we can fill in the values of a vec4 with a vec2 – vUV.st is our uv coordinate as a vec2. In essence what we’ve done is say that we want to use the uv coordinates to stand in for our red and green values, blue will always be 0, and our alpha will always be 1. It’s okay if that’s a wonky to wrap your head around at the moment. If you’re still scratching your head you can read more at links below

Read about more GLSL Data Types

Read about writing your own GLSL TOP

Okay, so we’ve got this silly gradient, but what is it good for?!

Let’s move around our stoner a little bit to see what else changes here.

pushingpoints

That’s still not very sexy – I know, but let’s hold on for just one second. We first need to pause for a moment and think about what this might be good for. In fact, there’s a lovely operator that this plays very nicely with. The remap TOP. Say what now? The remap top can be used to warp input1 based on a map in input2. Still scratching your head? That’s okay. Let’s plugin a few other ops so we can see this in action. We’re going to rearrange our ops here just a little and add a remap TOP to the mix.

remapTOP.PNG

Here we can see that the red / green map is used on the second input our our remap top, and our movie file is used on the first input.

Okay. But why is this anything exciting?

Richard Burns just recently wrote about remapping, and he very succinctly nails down exactly why this is so powerful:

It’s commonly used by people who use the stoner component as it means they can do their mapping using the stoners render pipeline and then simply remove the whole mapping tool from the system leaving only the remap texture in place.

If you haven’t read his post yet it’s well worth a read, and you can find it here.

Just like Richard mentions we can use this new feature to essentially completely remove or disable the stoner in our project once we’ve made maps for all of our texture warping. This is how we’ll get our cook time down to just 0.05 milliseconds.

Let’s look at how we can use the stoner to do just this.

For starters we need to add some empty bases to our network. To keep things simple for now I’m just going to add them to the same part of the network where my stoner lives. I’m going to call them base_calibration1 and base_calibration2.

calibration_bases

Next we’re going to take a closer look at the stoner’s custom parameters. On the Stoner page we can see that there’s now a place to put a path for a project.

stoner_path

Let’s start by putting in the path to our base_calibration1 component. Once we hit enter we should see that our base_calibration1 has new set of inputs and outputs:

base_capture1_added

Let’s take a quick look inside our component to see what was added.

inside_base1.PNG

Ah ha! Here we’ve got a set of tables that will allow the stoner UI to update correctly, and we’ve got a locked remap texture!

So, what do we do with all of this?

Let’s push around the corners of our texture in the stoner and hook up a few nulls to see what’s happening here.

working_with_calibration1

You may need to toggle the “always refresh” parameter on the stoner to get your destination project to update correctly. Later on we’ll look at how to work around this problem.

So far so good. Here we can see that our base_calibration1 has been updated with the changes we made to the stoner. What happens if we change the project path now to be base_calibration2? We should see that inputs and outputs are added to our base. We should also be able to make some changes to the stoner UI and see a two different calibrations.

working_with_calibration2.PNG

Voila! That’s pretty slick. Better yet if we change the path in the stoner project parameter we’ll see that the UI updates to reflect the state we left our stoner in. In essence, this means that you can use a single stoner to calibrate multiple projectors without needing multiple stoners in your network. In fact, we can even bypass or delete the stoner from our project once we’re happy with the results.

no_stoner

There are, of course, a few things changes that we’ll make to integrate this into our project’s pipeline but understanding how this works will be instrumental in what we build next. Before we move ahead take some time to look through how this works, read through Richard’s post as well as some of the other documentation. Like Richard mentions, this approach to locking calibration data can be used in lots of different configurations and means that you can remove a huge chunk of overhead from your projects.

Next we’ll take the lessons we’ve learned here combined with the project requirements we laid out earlier to start building out our complete UI and calibration pipeline.

Exploring Examples from the Forum| TouchDesigner

capture

https://github.com/raganmd/td_fb_forum_examples

New TouchDesigner tool for those of you looking to meander through some examples.

Pull this whole repo and find example_explorer.toe in:{your_local_directory}\td_fb_forum_examples\example_explorer

All the examples I’ve collected appear as a list on the far left, read me docs display in the middle viewer, and the example network shows up on the right.

For right now, use the “h” key to home the network when you load a new tox. Tracking down the right way to have that happen automatically – coming soon.

As a bonus this has a functioning list comp that you’re welcome to pillage.

Only a single example is loaded at a time to try and keep things optimized.

If you’re teaching a course and could benefit from re-purposing this network design for assignments or examples please feel free.