Category Archives: Python

TouchDesigner | Reusable Code Segmentation with Python

reusable-code-segmentation.PNG

Thinking about how to approach re-usability isn’t a new topic here, in fact there’s been plenty of discussion about how to re-use components saved as tox files, how to build out modular pieces, and how to think about using Extensions for building components you want to re-use with special functions.

That’s wonderful and exciting, but for any of us that have built deployed projects have quickly started to think about how to build a standard project that any given installation is just a variation of… a flavor, if you will, of a standard project build. Much of that customization can be handled by a proper configuration process, but there are some outliers in every project… that special method that breaks our beautiful project paradigm with some feature that’s only useful for a single client or application.

What if we could separate our functions into multiple classes – those that are universal to every project we build, and another that’s specific to just the single job we’re working on? Could that help us find a way to preserve a canonical framework with beautiful abstractions while also making space for developing the one-off solutions? What if we needed a solution to handle the above in conjunction with sending messages over a network?  Finally, what if we paired this with some thoughts about how we handle switch-case statements in Python? Could we find a way to help solve this problem more elegantly so we could work smarter, not harder?

Well, that’s exactly what we’re going to take a look at here.

First a little disclaimer, this approach might not be right for everyone, and there’s a strong assumption here that you’ve got some solid Python under your belt before you tackle this process / working style. If you need to tool up a little bit before you dig in, that’s okay. Take a look at the Python posts to help get situated then come back to really dig in.


Getting Set-up

In order to see this approach really sing we need to do a few things to get set-up. We’ll need a few operators in place to see how this works, so before we dig into the python let’s get our network in order.

First let’s create a new base:

base.PNG

Next, inside of our new base let’s set up a typical AB Deck of TOPs with a constant CHOP to swap between them:

typical-ab-deck.PNG

Above we have two moviefilein TOPS connected to a switch TOP that’s terminated in a null TOP. We also have a constant CHOP terminated in a null whose chan1 value is exported to the index parameter of our switch TOP.

Let’s also use the new Layout TOP in another TOP chain:

layout-top.PNG

Here we have a single layout TOP that’s set-up with an export table. If you’ve never used DAT Exports before you might quickly check out the article on the wiki to see how that works. The dime tour of that ideal is that we use a table DAT to export vals to another operator. This is a powerful approach for setting parameters, and certainly worth knowing more about / exploring.

Whew. Okay, now it’s time to set up our extensions. Let’s start by creating a textDAT called messageParserEXT, generalEXT, and one called jobEXT.

parser-general-job.PNG


The Message Parser

A quick note about our parser. The idea here is that a control machine is going to pass along a message to a set of other machines also running on the network. We’re omitting the process of sending and receiving a JSON blob over UPD, but that would be the idea. The control machine passes a JSON message over the network to render nodes who in turn need to decode the message and perform some action. We want a generalized approach to sending those blobs, and we want both the values and the control messages to be embedded in that JSON blob. In our very simple example our JSON blob has only two keys, messagekind and vals:

message = {
        'messagekind' : 'some_method_name',
        'vals' : 'some_value'
}

In this example, I want the messagekind key to be the same as a method name in our General or Specific classes.

Pero, like why?!

Before we get too far ahead of ourselves, let’s first copy and past the code below into our messageParserEXT text DAT, add our General and Specific Classes, and finish setting up our Extensions.

The General Code Bits

In our generalEXT we’re going to create a General class. This works hand in hand with our parser. The parser is going to be our helper class to handle how we pass around commands. The General class is going to handle anything that we need to have persist between projects. The examples here are not representative of the kind of code you’d have your project, instead they’re just here to help us see what’s happening in this approach.

The Specific Code Bits

Here in our Specific class we have the operations that are specific to this single job – or maybe they’re experimental features that we’re not ready to roll into our General class just yet, regardless, these are methods that don’t yet have a place in our canonical code base. For now let’s copy this code block into our jobEXT text DAT.

At this point we’re just about ready to pull apart what on earth is happening. First let’s make sure our extension is correctly set-up. Let’s go back up a level and configure our base component to have the correct path to our newly created extension:

 

reusable-ext-settings.PNG

Wait wait wait… that’s only one extension? What gives? Part of what we’re seeing here is inheritance. Our Specific class inherits from our General class, which inherits form our MessageParser. If you’re scratching your head, consider that a null TOP is also a TOP is also an OP. In the same way we’re taking advantage of Python’s Object oriented nature so we can treat a Specific class as a special kind of General operation that’s related to sending messages between our objects. All of his leads me to believe that we should really talk about object oriented programming… but that’s for another post.

Alright… ALMOST THERE! Finally, let’s head back inside of our base and create three buttons. Lets also create a panel execute for each button:

buttons.PNG

Our first panel execute DAT needs to be set up to watch the state panel value, and to run on Value Change:

change-switch.PNG

Inside of our panel execute DAT our code looks like:

# me - this DAT
# panelValue - the PanelValue object that changed# # Make sure the corresponding toggle is enabled in the Panel Execute DAT.
def onOffToOn(panelValue):
    return
def whileOn(panelValue):
    return
def onOnToOff(panelValue):
    return
def whileOff(panelValue):
    return
def onValueChange(panelValue):
    message = {
        'messagekind' : 'Change_switch',
        'vals' : panelValue } 
    parent().Process_message(message)
    return

If we make our button viewer active, and click out button we should see our constant1 CHOP update, and our switch TOP change:

switch-gif.gif

AHHHHHHHHH!

WHAT JUST HAPPENED?!


The Black Magic

The secret here is that our messagekind key in our panel execute DAT matches an existing method name in our General class. Our ProcessMessage() method accepts a dictionary then extracts the key for messagekind. Next it checks to see if that string matches an existing method in either our General or Specific classes. If it matches, it then calls that method, and passes along the same JSON message blob (which happens to contain our vals) to the correct method for execution.

In this example the messagekind key was Change_switch(). The parser recognized that Change_switch was a valid method for our parent() object, and then called that method and passed along the message JSON blob. If we take a look at the Change_switch() method we can see that it extracts the vals key from the JSON blob, then changes the constant CHOP’s value0 parameter to match the incoming val.

This kind of approach let’s you separate out your experimental or job specific methods from your tried and true methods making it easier in the long run to move from job to job without having to crawl through your extensions to see what can be tossed or what needs to be kept. What’s better still is that this imposes minimal restrictions on how we work – I don’t need to call a separate extension, or create complex branching if-else trees to get the results I want – you’ll also see that in the MessageParser we have a system for managing elegant failure with our simple if hasattr() check – this step ensure that we log that something went wrong, but don’t just throw an error. You’d probably want to also print the key for the method that wasn’t successfully called, but that’s up to you in terms of how you want to approach this challenge.

Next see if you can successfully format a message to call the Image_order() method with another panel execute.

What happens if you call a method that doesn’t exist? Don’t forget to check your text port for this one.

If you’re really getting stuck you might check the link to the repo below so you can see exactly how I’ve set this process up.

If you got this far, here are some other questions to ponder:

  • How would you  use this in production?
  • What problems does this solve for you… does it solve any problems?
  • Are there other places you could apply this same idea in your projects?

At the end of the day this kind of methodology is really looking to help us stop writing the same code bits and bobs, and instead to figure out how to build soft modules for our code so we can work smarter not harder.

With any luck this helps you do just that.

Happy Programming.


Take a look at the sample Repo for this example on Github:
touchdesigner-reusable-code-segmentation-python

TouchDesigner | Switch Statements in Python

python-switch-statements.PNG

Hang onto your socks programmers, we’re about to dive deep. What are we up to here today? Well, we’re going to look into switch statement alternatives in Python (if you don’t know what a switch statement is don’t worry we’ll cover that bit), how you might use that in a practical real-world situation, and why that’s even an idea worth considering. With that in mind let’s dig-in and start to pull apart what Switch Statements are, and why you should care.

From 20,000 feet, switch-case statements are an approach to handling different situations by way of a look-up table rather than with a series of if-else statements. If you’re furrowing your brow consider situations when you may have encountered complex if-else statements where once change breaks everything… for so so much longer than you might want. Also consider what happens if you want to extend that if-else ladder into something more complicated… maybe you want to call different functions or methods based on input conditions, maybe you need to control a remote machine and suddenly you’re scratching your head as you ponder how on earth you’re going to handle complex logic statements across a network. Maybe you’re just after a better code-segmentation solution. Or maybe you’ve run into a function so long you’re starting to loose cycles to long execution times. These are just a few of the situations you might find yourself in and a switch statement might just be the right tool to help – except that there are no switch-case statements in Python.

What gives?!

While there aren’t any switch-case statements, we can use dictionary mappings to get to a similar result… a result so powerful we’re really in for a treat. Before we get there though, we need to look at the situation we’re trying to avoid.

So what exactly is that situation? Let’s consider a problem where we want to only call one function and then let that code block handle all of the various permutations of our actions. That might look like our worst case solution below.

Worst

To get started, what do we have above? We have a single function called switcher() that takes three arguments – the name of the function we want to call, and two values. In this example we have four different math operations, and we want to be able to access any of the four as well as pass in two values and get a result just by calling a single function. That doesn’t seem so hideous on the face of it, so why is this the worst approach?

This example probably isn’t so terrible, but what it does do is bury all the functional mathematical portions of our code inside of a single function. It means we can’t add and test a new element without possibly breaking our whole functional code block, we can only access these operations from within switcher(), and if we decide to add additional operations in the future our code block will just continue to accrue lines of code. It’s a naive approach (naive in the programming sense – as in the first brute force solution you might think of), but it doesn’t give us much room for modularity or growth that doesn’t also come with some unfortunate side effects.

Okay… fine… so what’s a good solution then?


Good

A good solution segments our functions into their own blocks. This allows us to develop functions outside of our switcher() function, call them independently, and have a little more flexible modularity. You might well be thinking that this seems like a LOT more lines… can we really say this is better?! Sure. The additional lines are worth it if we also get some more handles on what we’re doing. It also means we probably save some serious debugging time by being able to isolate where a problem is happening. In our worst case approach we’re stuck with a single function that if it breaks, none of our functions work… and if our logic got sufficiently complex we might be sifting through a whole heap of code before we can really track down what’s happening. Here at least there’s a better chance that a problem is going to be isolated to a single function block – that alone is a HUGE help.

All that said, we’re still not really getting to switch-case statements… we’re still stuck in if-else hell where we’ll have to evaluate our incoming string against potentially all of the possible options before we actually execute our actual code block. At four functions this isn’t so bad, but if we had hundreds we might really be kicking ourselves.

So how can we do better?


Better

Better is to remember that the contents of a python dictionary can be any data type – in fact they can even be function names, or Python objects. How does that help use? Well, it means we can look up what function we want to call on the fly, call it, and even pass in variables. In the example above our switcher() function holds a dictionary of all the possible functions at our disposal – when we call our switcher we pass in the name of the function with the variables that will in turn get passed to the function. Above our active_function variable becomes the variable that’s fetched from our dictionary, which we in turn pass our incoming variables along to.

That’s great in a lot of ways, but especially in that it gets us away from long complicated if-else trees. We can also use this as a mechanism for handling short-hand names for our methods, or multiple assignments – we might want two different keys to access the same function (maybe “mult” and “Multiple” both call the same function for example).

So far this is far away a better approach, so how might we make this better still?


Best

We might take this one step further and start to consider how we might address accepting an arbitrary number of vals. Above we have a simple way to tackle this – probably not what you’d end up with in production, but something that should hopefully get you thinking. Here the variable vals becomes a list that can be any number of values. In the case of both our Add() and Subtract() functions we loop through all of the values – adding each val, or subtracting each val respectively. In the case of our Multiply() and Divide() functions we limit these operations to only two values for the sake of our example. What’s interesting here is that we can return can think about error handling based on the array of values that’s coming into our function.


The above is great, of course, but it’s really just the beginning of the puzzle. Where this really starts to become interesting is how you might think of integrating this approach in your python extensions.

Or if vals is a a dictionary in it’s own right rather than a simple list.

Or if you can send a command like this over the network.

Or if you can start to think about how to build out blocks of code that are specific to a single job, and universal blocks that apply to all of your projects.

Next we’ll start to pull apart some of those very ideas and see where this concept really gets exciting and creates spaces for building tools that persists right alongside the tools that you have to build for a single job.

In the meantime, experiment with some Python style switch statements to see if you can get a handle on what’s happening here, and how you might take better advantage of this method.

Happy programming!


References

Looking for another perspective on this approach form a more pure Python perspective? Check out this post on Jaxenter.com.

TouchDesigner | Delay Scripts

It’s hard to appreciate some of the stranger complexities of working in a programming environment until you stumble on something good and strange. Strange how Matt? What a lovely question, and I’m so glad that you asked!

Time is a strange animal – our relationship to it is often changed by how we perceive the future or the past, and our experience of the now is often clouded by what we’re expecting to need to do soon or reflections of what we did some time ago. Those same ideas find their way into how we program machines, or expect operations to happen – I need some-something to happen at some time in the future. Well, that’s simple enough on the face of it, but how do we think about that when we’re programming?

Typically we start to consider this through operations that involve some form of delay. I might issue the command for an operation now, but I want the environment to wait some fixed period of time before executing those instructions. In Python we have a lovely option for using the time module to perform an operation called sleep – this seems like a lovely choice, but in fact you’ll be oh so sorry if you try this approach:

But whyyyyyyyy?!

Well, Python is blocking inside of TouchDesigner. This means that all of the Python code needs to execute before you can proceed to the next frame. So what does that mean? Well, copy and paste the code above into a text DAT and run this script.

time.sleep

If you keep an eye on the timeline at the bottom of the screen, you should see it pause for 1 second while the time.sleep() operation happens, then we print “oh, hello there” to the text port and we start back up again. In practice this will seem like Touch has frozen, and you’ll soon be cursing yourself for thinking that such a thing would be so easy.

So, if that doesn’t work… what does? Is there any way to delay operations in Python? What do we do?!

Well, as luck would have it there’s a lovely method called run() in the td module. That’s lovely and all, but it’s a little strange to understand how to use this method. There’s lots of interesting nuance to this method, but for now let’s just get a handle on how to use it – both from a simple standpoint, and with more complex configurations.

To get started let’s examine the same idea that we saw above. Instead of using time.sleep() we can instead use run() with an argument called delayFrames. The same operation that we looked at above, but run in a non-blocking way would look like this:

If you try copying and pasting the code above into a text DAT you should have much better results – or at least results where TouchDesigner doesn’t stop running while it waits for the Python bits to finish.

Okay… so that sure is swell and all, so what’s so complicated? Well, let’s suppose you want to pass some arguments into that script – in fact we’ll see in a moment that we sometimes have to pass arguments into that script. First things first – how does that work?

Notice how when we wrote our string we used args[some_index_value] to indicate how to use an argument. That’s great, right? I know… but why do we need that exactly? Well, as it turns out there are some interesting things to consider about running scripts. Let’s think about a situation where we have a constant CHOP whose parameter value0 we want to change each time in a for loop. How do we do that? We need to pass a new value into our script each time it runs. Let’s try something like:

What you should see is that your constant CHOP increments every second:

for-loop-delay

But that’s just the tip of the iceberg. We can run strings, whole DATs, even the contents of a table cell.

This approach isn’t great for everything… in fact, I’m always hesitant to use delay scripts too heavily – but sometimes they’re just what you need, and for that very reason they’re worth understanding.

If you’ve gotten this far and are wondering why on earth this is worth writing about – check out this post on the forum: Replicator set custom parms error. It’s a pretty solid example of how and why it’s worth having a better understanding of how delay scripts work, and how you can make them better work for you.

Happy Programming.