Author Archives: raganmd

TouchDesigner | The Big Badass Lister | Part 3/3

THE BIG BADASS LISTER

Part 3 of 3

Lister’s Dank Dark Secrets

Welcome back for the third and final episode of The Big Badass lister. This week’s knowledge store comes to you directly from Ivan himself. He gives us a peek into some of the trickier parts of lister, the power-user secrets, and some of the cooler things you can do with lister when you use a little python. This section stands on its own, but if you’re brand new to lister it might be worth giving the other parts of this long form post a read through. 

And now… none other than Ivan himself:

Notes From The Developer

As the main developer of the lister custom component, I’m excited that it’s getting more use and specifically I’m excited to get to collaborate with Tutorial-master Ragan on this blog post. I’d like to start with a brief history of lister’s development. It’s something I have fond memories of, as this was the very first project I would work on with Derivative.

When I started on it, the listCOMP, the operator lister is built on, was brand new and had only been used internally by Derivative. I was given a very primitive version of lister that had already been through three developers and had no associated design plan. It was built mostly using a Python extension, which was also a relatively new concept in TouchDesigner. I started adding features that were immediately needed, and the lister quickly became an incredibly valuable ui element. We needed more and more features and I kept piling them on as requested. Eventually I realized that I had created a bit of a monster. The lister is incredibly powerful and has a number of great features but, full confession, it does reveal my inexperience with TouchDesigner when I developed it. Some day, there will be a lister 2.0, but until then it’s worth understanding lister’s shortcomings when working with it.

Most importantly, lister is a little slow. Its heavy reliance on Python and listCOMP’s inability to insert single rows and columns means that certain operations, especially full refreshes, can drop frames. While this doesn’t hinder lister as a development tool, it can limit its use in live performances. The second main caveat with lister is that moving beyond simple operations will often involve some Python programming. I created a large number of callbacks in order to facilitate this, but managing lister can still be complicated and will sometimes involve working directly with the listCOMP itself. In fact, I would recommend learning how the listCOMP works by itself to anyone doing heavy lister work. The final disclaimer I’d like to make is about the lister extension itself. Because of the way it was developed, it is overly large and rather confusing, and generally not a great example of clean TouchDesigner Python code.

All that said, lister is still a powerhouse for displaying and working with lists of data. Let’s have some fun and get into the deep features.

The Column Definition Table 

You can do a ton of fancy things with lister using just the column definition table. As you already know, this table is where you set up the big picture layout of your list. There are also a number of hidden gems that you might miss just by skimming the wiki.

Source Data and Source Data Modes

lister Pars

For this example (sourceDataModes.tox), I’ve set up a simple table input with Input Table Has Headers on. For starters, compare the input table, the column definitions, and the lister output. Notice how many more columns the lister has than the input table. That’s because you can use the same source data multiple times in different columns by using the same header string in sourceData. You can also rearrange the columns however you want because when Input Table Has Headers is on, the lister looks up the data by table header, not by column number.

input Table DAT
colDefine Table
Styled Lister

Modes: int, float, string, version

The simplest source data modes are int, float, string, and version. They do not filter or change the text in any way, they are only used for sorting. To see this, click on the column header to sort by that column and notice that the same data is sorted differently depending on the source data mode. Any data that cannot be converted to the chosen mode is sorted alphabetically.

Special Modes: blank, rowNum, constant, repr

The blank mode keeps the text data internally, but doesn’t display it in the cell. This is extremely useful in conjunction with topPath and cellLook methods to show information graphically instead of with text. 

The rowNum mode also keeps the text data, but only displays the row number. This is just a convenient way to display row numbers without coding.

The constant mode puts the literal value in sourceData into each cell in the column. This is mostly used as a quick and easy way to make buttons. For example “X” for delete.

The repr mode displays the sourceData using Python’s repr function instead of converting the object to a string. When using tables (which hold only strings) this just means that the data will have quotes around it. When using objects, lists, or dictionaries as your source data rows, this can provide more detailed information about Python objects. 

Super special mode: color

The color mode is a shortcut for creating colored cells. In order to use this feature, the data in a color column must be formatted properly. Specifically, it must be a list of four numbers, followed by an optional string. The first four numbers will be used to define the cell’s color. The string will be used as text drawn over that color. The example lister shows a number of examples. If the data is formatted incorrectly, it will be used directly as text, with no colors applied. If you have incoming data that you want to color in the lister, the onConvertData callback is probably the best place to change that data into this r, g, b, a, text format.

Expressions in the Column Definition Table and Stretchy Columns

There are a couple places where you can use expressions in the column definition table to avoid using callbacks at all. This powerful technique is illustrated in the sourceDataModeEvalAndHelp example. This example uses a list of operators as its raw data, and its column define and output are as follows:

colDefine Table
styled Lister

The first place you can use an expression in the column definition table is to process your source data. If you set sourceDataMode to eval, the sourceData will be treated as an expression, and your row object will be available in the variable object. In the example, you can see type(object) displayed in the middle column, and an even cleaner version of that, object.__class__.__name__, in the right column. This powerful method lets you display just about anything in a Python object.

There is also some secret sauce for the help row, which defines popup help text for a column’s cells. An asterisk by itself will make the popup help display cell contents exactly. This is useful when a column might not be wide enough to show the cell contents because it allows you to hover over it for the full text. If the asterisk is followed by text, that text is used as an expression which will be evaluated and displayed in popup text. That expression gives you access to object just like in sourceData, and also provides text, which is the cell’s text. In the example, the middle column help will show the text representation of the row object, and the right column will display the cell text prepended with “type: “.

One last thing to notice in this example is that the left column is set to stretch. This means it will take up any available horizontal space in the lister display. If more than one column is set to stretch, the available space will be divided equally. That’s pretty obvious stuff, but what you might not know is that the width of the column will be used as the minimum width for stretch columns. 

Editable, selectRow, and clickOnDrag

The last few column definition tricks are pretty simple but worth going over because when you need these features they’re easy. This section will use the editableTable example and is particularly worth trying in TouchDesigner because the features are interactive. Also notice the columns are rearranged from the source table because I don’t want you to forget that great feature!

Lister Pars
sytled Lister
colDefine Table

The editable setting in colDefine is pretty straightforward. A 1 means single-click to edit a cell and a 2 means double-click to edit. What’s so great is that if you use this in combination with the Input Table Has Headers and Auto-sync Input Table parameters, you have an instant table DAT editor with a bunch of nice features. It’s worth noting here that if you are not using auto-sync features, you will have to explicitly propagate changes from your lister into your data by using Python callbacks. For an example of that, see the onDrop Callback section below.

Output DATs and Selection Features

The next two settings we’re going to look at have to do with the lister’s “selection” features. The selection is most easily seen in the output, so it’s a good time to look at the two outputs ports of the lister component: out_table and out_selection

The out_table DAT output is a table of all the text in the lister. This is often very similar to an auto-synced table, but is not exactly the same because of various data permutations you can do inside lister. This table can also be useful for reacting to data changes in the lister. 

The other output is the out_selected DAT, and only displays the data from the “selected” rows in the lister. 

In the example, click on cells in the left column to select them and see this in action. 

The reason you can only click on the left column in this example, is the selectRow setting. As you may have guessed, clicking a column will only select rows if this is set to 1. Notice when you click on the middle or right columns in the example, the selection does not change.

The clickOnDrag setting is a little more subtle. To see what this does, click on the left column in the example and drag the mouse up and down to other columns. Each row is selected as you drag because this setting is on. This is useful for things like presets or colors because you can set up a column of choices and drag across to browse through them quickly. This also triggers onClick callbacks for each cell as you drag across them.

Parameter Tricks

The lister component exposes a number of complex list interaction features that are commonly needed. Whenever possible, these are accessible via parameter. Let’s take a look at the possibilities. This example uses the editableTable tox used in the previous section. These examples are worth playing with to understand the exact effects. Let’s walk through them one by one.

lister Pars
  • The Header parameter simply turns on and off the header row display. 
  • Clickable Header turns on and off the ability to sort rows by clicking on the column header. In later versions of lister, ctrl-click will turn off this sorting. 
  • Selectable Rows controls whether rows can be selected with mouse clicks. As explained above, selectRow must be on in the column define table as well.
  • Multiple Row Select lets you select multiple rows using the common modifiers in file explorers. That is, holding down ctrl adds or removes selections and shift will select a section of rows.
  • Drag To Reorder Rows enables dragging selected rows to new positions in the list.
  • Arrow Keys lets you move the row selection by pressing up and down arrows.
  • Delete Key lets you delete all selected rows by pressing Delete.
  • Highlight Rollover enables/disables highlighting of the row that the mouse is currently hovering over.
  • Row Striping lets you differentiate rows with two different styles of highlight, either thin lines between rows or alternating highlights. The settings for these two colors are found in the define DAT in the lister’s config component.

Other Interaction Techniques: Buttons, Toggles, Menus

Toggle Pars
styled lister
Table

Now that we’ve looked at how easy it is to set up editable text, let’s explore some other common interaction features you might want to use in your lister. The example lister references a table with rows of “Yay” or “Nay” votes. You can right-click the text to get a menu or push any cell to toggle votes back and forth. The input table setup is the Auto-sync type you’ve seen before in this tutorial, but with the row selection features turned off. To set up the actual interactions, we’ll be using some simple Python callbacks. 

Setting Up Button and Toggle Looks

colDefine Table

We’ll go into how each column actually changes the data in the Interaction Callbacks section below, but for now let’s look at how we set up the looks. Setting up the columns in this lister starts, as usual, with the column define table. They all use the “button” cellLook. This means that each cell in the column will use the text and color settings in the config TOPs that start with the word “button”. When setting up a cell look, you must have a TOP with the same name. You can also have an optional name + “Roll” TOP for rollovers and name + “Press” TOP for when the mouse button is down on the cell. In this example, we set up all three. You can have as many of these cell looks as you want in your lister, they just need unique names.

lister Button Styles

The “Button” and “Toggle” columns go one step further than cell looks by adding images and removing the text. To remove the text, we just use the “blank” sourceDataMode. To add the images, we add a name in the topPath row. The “Button” column does not reflect the data in the table, it is simply a push button that switches the “Vote”. It uses the topPath “btnImage” and has separate TOPs for the roll and press state as well. In this example I’ve used circleTOP, hsvadjustTOP, and levelTOP to create the images, but you can use any TOP techniques you like. Note how the transparent part of the “btnImage” TOPs will reveal the corresponding “button” cellLook tops.

The “Toggle” column uses the same technique as the “Button” column, but will use a different image for each data state. The topPath for this column is “toggle*” and the asterisk at the end indicates that the text in the column will be appended when looking for the TOP to display. This means the image “toggleNay” will be used when the cell’s data is “Nay”, even though you don’t see the “Nay” because the sourceDataMode is “blank”. When using this asterisk mode, the “Roll” and “Press” will still be appended. In this example, we only set up a “Press” state, but notice that the background is still highlighted when rolling over these cells because of the “buttonRoll” cellLook. These toggles were created with the rectangleTOP but again, you can use any TOPs you want, including movieFileIn if you want external files.

Interaction Callbacks

(This section requires some basic Python knowledge) 

def onClick(info):  
    inputTable = info['ownerComp'].par.Inputtabledat.eval()
    cell = inputTable[info['row'], 'Vote']
    if cell.val == 'Yay':
        cell.val = 'Nay'
    else:
        cell.val = 'Yay'

The first callback we’ll look at it is onClick. This is called when the mouse is pressed and released on any non-header cell. Notice that when you click on any cell in the lister, it switches the “Vote” for that row. Because we will be working directly with the input table, the first thing we need to do is get it from the lister’s parameters. All lister callbacks will have the lister itself in the “ownerComp” key of the info dictionary. The first line in the callback gets the DAT stored in ownerComp’s Inputtabledat parameter and stores it in inputTable. The next line gets the specific cell in the inputTable by cross-referencing info[‘row’], which holds the number of the row clicked on, and the “Vote” column of the table.

Now that we have the tableDAT cell, we just need to test if its value is “Yay”. If it is, we set it to “Nay”. If it’s not, it must be “Nay” already so we set it to “Yay”. Again, note that this same code works for every cell in the lister, no matter which column, because they all change the same value in the input table. Because the “Vote” and “Toggle” columns in the lister are based on the input table’s value, they automatically update when we make this change.

Using Popup Menus With Lister

Using popup menus with a lister is an incredibly useful technique. It basically allows you to perform any number of custom functions on a per-row or per-cell basis. This example will be one of the simplest possible cases, but the method can be expanded easily. Because this is a lister tutorial and not a popup menu tutorial, we won’t dive deep into the many features of the TouchDesigner popMenu but the wiki has some good examples and it’s fairly straightforward once you get the basics.

popMenu = op.TDResources.op('popMenu')

In this example we use the system popMenu, which is accessed via the global shortcut op.TDResources. If you want to customize the look of your popup menu, you can get a unique one from the palette UI section. If you don’t mind the generic look, it is generally safe to use the shared system one because there will only ever be one popMenu open at a time.

def onClickRightVote(info):
    popMenu.Open(items=['Yay', 'Nay'], callback=selectVote, 
                    callbackDetails=info)

The next step in using a popup menu is opening it, which we do in the onClickRightVote callback. This is an example of the special cell callback system in lister, where we use the generic name of the callback (onClickRight) followed by the name of the column from the column definition table (Vote) to create a callbacks that is only called when that column is clicked. 

The Open function itself has a number of arguments, but in this case we only use three. The items argument is a list of choices in the menu. For this example, you can select “Yay” or “Nay”. The next argument, callback, is a function that will be called when the user makes a choice. In this case, we call it selectVote and we’ll define that function next. The callbackDetails argument will be passed into the chosen callback to provide extra info. A simple and powerful technique is to just pass the entire info dictionary in this argument, so that the popup menu’s callback will have access to everything that this lister callback knows.

When the Open function is called, the popup menu appears, and TouchDesigner runs normally in the background, waiting for the user to either select something or click somewhere not on the popup menu. If the user clicks somewhere else, the menu closes and the callback function selectVote will never be called. The last step is to define what the system will do if the user does click a selection from the menu.

def selectVote(menuInfo):
    clickDetails = menuInfo['details']
    row = clickDetails['row']
    col = clickDetails['col']
    lister = clickDetails['ownerComp']
 
    lister.SetCellText(row, col, menuInfo['item'])

The selectVote function is a callback from the popup menu, and like lister’s callbacks it takes a single argument that will be filled with a dictionary of information. For the sake of clarity, we’ve named it menuInfo instead of info but you can name it whatever you want. From the menuInfo dictionary we grab the “item”, which is the choice that was clicked on (“Yay” or “Nay”) and store that in menuChoice. We then get clickDetails from  menuInfo[‘details’], which is the original info dictionary in the onRightClickVote callback. Remember, we sent that explicitly using the callbackDetails argument in Open. Those clickDetails contain the row and column of the cell that was clicked on, and the lister itself, so we store those as well. It should be noted that you don’t have to store all these things in separate variables, but it’s a lot easier to read and explain if you do.

The last line is actually the only line in this function that changes anything. We use the lister method SetCellText to (did you guess?) set the cell text at the row and col that was clicked on. We set it to whatever string was clicked on in the menu, which can be found in menuInfo[‘item’]. Because we have Auto-sync on for the lister, setting the cell text will cause the input table to be updated automatically. This in turn causes the Toggle column to be updated automatically. Easy-peasy! So much power with so little code. You can do a whole lot more things with this technique just by adding more choices to the menu and writing different reactions depending on the “item” selected.

Drag And Drop Between Listers

(This section requires some basic Python knowledge)

Creating drag-and-drop systems is never dead simple, but lister does a lot of the heavy lifting for you. In this example we’ll go over the setup and Python code for dragging and dropping between listers and from outside them. The basic setup is things you’ve seen before in this tutorial: two listers with input tables and auto-sync on. For convenience, they’re displayed side by side in the dragDropContainer component. The only significant change in the column definitions is that the columns are set to be draggable

The one interesting thing about the set up is that both listers share a single config comp, ddListerConfig, which means they have the same look, the same column definitions, the same callbacks etc. This config comp sharing is not necessary for dragging and dropping between listers, it’s just convenient because both listers in this example work exactly the same. This requires a little extra attention to the callbacks, because they have to be written in a generic way that works for either lister. The good news is that if you stick to always using data from the info dictionary, it will always work. For example info[‘ownerComp’] will always contain the lister component you are working with.

To see the listers in action, just drag rows back and forth. They contain paths to actual nodes in the network. You can also drag a node from the network into either lister to add it to the list. You’ll also notice a line in the list showing where the dropped row will appear. This is controlled by the Drop Highlight parameter on the listers, and the ugly color I used to make it stand out is controlled by the define table inside the config comp. Try the other Drop Highlight settings, just to get a look at them. They can all be useful, depending on how you set up your drag/drop system.

The onDropHover Callback

def onDropHover(info):
    debug('onDropHover', info)
 
    for item in info['dragItems']:
        # make sure all dropItems are operators
        if not isinstance(item, OP):
            return False
    return True

This part of our drag/drop system is really simple. First, we have a debug statement for convenience so you can open a textport and see what info is available. In the info dictionary, we check dragItems and make sure everything in the list is an operator (OP) using the isinstance function. If we find an item that is not an operator, we return False to indicate that it is not an acceptable drop, otherwise we return True. Now, the listers accept either operators or rows from the other list, so why do we take any operators? Because when you pass lister data, the dragItem will still be the lister operator itself. There is more specific drag data elsewhere in the info dictionary, but we don’t care about that in the onDropHover callback. We’ll check that next.

The onDrop Callback

def onDrop(info):
    debug('onDrop', info)
 
    ownerComp = info['ownerComp']
    dropRow = info['row'] # row that was dropped in
    # special cases of dropRow
    if dropRow == 0:
        # dropped on header, put it in row 1
        dropRow = 1
    elif dropRow == -1:
        # was dropped on part of lister without a cell, drop it at bottom
        dropRow = len(ownerComp.Data)

In the first half of the callback, we have a debug statement again, we store the lister in ownerComp, and then we do a little setup to decide where to drop a row. As usual with lister callbacks, the row under the mouse is in info[‘row’] and we store that in dropRow. There are a couple of special cases, though. If dropRow is zero, that’s the header row. We don’t want to drop above that, so we’ll just drop into row 1 instead. If dropRow is -1, that means the area of the lister without any cells, and we’ll just drop it after the last row. To get the last row, we find the length of ownerComp.Data

The Data member of lister is worth examining a bit. It’s a Python list with an item for each row in the lister. Each item in the Data list is an ordered dictionary with a key for each column in the list. Those keys correspond to the column row in the column definition table. The value in each key is the text stored in the corresponding cell of the lister. There is one more key in the dictionary: rowObject. The rowObject can be either a list of strings from an input table, or the row’s Python object as provided in the onGetRawData callback or the Raw Data parameter.

    for dragItem in info['dragItems']:
        # handle multiple items being dropped

Now that we know where we want to drop, we need to go through each of the dragItems and create a row for it.

        if dragItem.name in ['dragDrop1', 'dragDrop2']:
            # it's one of our example listers, copy the info from source
            listerInfo = info['fromListerInfo'] # dragged cell
            sourceRow = listerInfo[0] # dragged row
            sourceData = dragItem.Data[sourceRow] # full row data
            ownerComp.Data.insert(dropRow, sourceData) # insert data
            ownerComp.DataChanged() # required after changing lister.Data

This first case is if the dragItem is one of our two listers. This means we’re dragging a row from the lister. The source cell is stored in info[‘fromListerInfo’] and from that we can get the source row number and thus the full Data item. We then simply insert that Data item into our lister at the dropRow we figured out before. One last thing you have to do when changing the Data list directly in Python is call the DataChanged function.

There are a couple extra things worth noting here. One is that the system can get confused between the lister operator itself and a cell from that lister. This will be fixed in later versions of listCOMP, but for now rest assured that it will never be an issue in practice. The other thing it’s important to understand is that our life is made very easy in this case by the fact that the Data item is identical in both listers. When adding a row into Data in more complicated situations, you will often want to use lister’s helper function RowObjectToDataRow, which takes a row object and creates an item that is properly formatted for the Data list. 

            # remove row from source
            if not ownerComp.panel.ctrl:
                dragItem.DeleteRows([sourceRow])

Now that we’ve added the rows to the lister that was dropped on, we need to remove it from the one that it was dragged from. I snuck in a copy feature with the if statement: if ctrl is being held down, we don’t delete the row from the source, which means it will now be in both listers. Also note the way that you can check if ctrl is being held down in onDrop is to check ownerComp.panel.ctrl. This works for shift (and any other modifier/panel value) as well.

        else:
            # it's a node from a network, add it via data table
            inputTable = ownerComp.par.Inputtabledat.eval()
            inputTable.insertRow([dragItem.path, dragItem.__class__.__name__],
                                    dropRow)

This else clause is for when an operator is dropped in the lister from a network. In this case, we use the technique of changing the input table instead of the Data list directly. Either way works, and it’s really up to you to decide which you prefer. The code here is using standard TouchDesigner tableDAT operations. First we get the DAT itself from the lister’s parameters, then we use the DAT’s insertRow function to create a row with path and class name at the location of dropRow.

Wrapping Up

It’s great to finally spell out some of these lister tricks! I’ve been meaning to write something like this for a long time. I hope it shed light on some new useful stuff and that you find lister helpful in your projects. Feel free to tag me directly in the TouchDesigner forums if you get stuck or you have ideas for new features.

You can download the examples from this post here.

Thanks for sticking with us over the last three weeks, and we hope you’ve learned a bit more about lister.

TouchDesigner | The Big Badass Lister | Part 2/3

THE BIG BADASS LISTER

Part 2 of 3

Well hello there! And welcome back for another installment of “I didn’t know how much I needed lister in my life.” Last week we looked at the basics for lister, getting started, and finding your bearings. This week we’ll take a closer look at some of the more nuanced pieces of lister. How you can add buttons to your lists, substitute images, create a roll-over button table, and work with data structures beyond just tables. If Part 1 was our beginner’s look at lister, this is our intermediate adventure in working with this component. 

Some caveats this week – in Part one all of the work we did with lister didn’t require any familiarity with Python. For some of the more intermediate use cases we do end up doing a little scripting to get exactly the results we want. Don’t worry – you don’t have to be a Pythonista to make these little bits of magic work, a little Python can go a long way in TouchDesigner especially with lister. 

Working with Tables

There are lots of ways we can work with tables with lister – and before we start to look at the more complex examples, let’s begin with some simple and practical examples. We’ll look at adding a button to our to our lister’s rows so we can remove an item, how to create an array of buttons, how to work with a TOP path to set a static button look, how to dynamically update our buttons in a single toggle style, and how update our buttons based on their toggle state.

Adding a Button Column

We’re going to use many of the techniques we’ve already learned to style our lister, so many of our first steps should feel very familiar. We’ll also start with the same simple table DAT we used previously.

Input Table DAT

This time we’re going to use an extra parameter on the lister COMP so we can use a referenced input table DAT rather than connecting our table directly to lister. We’re also going to set lister so it will automatically update the contents of our input table. 

Update Table Par

Looking at our colDefine Table, the first two columns should look very similar to our pervious configurations. In this case our third column is the one to pay closer attention to. Here we’re adding a column that doesn’t currently exist in our input table, setting it’s contents to be a constant value of “X” and setting the sourceDataMode to be “button”. 

colDefine Table

The look of our button is going to be derived from the three TOPs labeled “button” “buttonRoll” and “buttonPress”. These TOPs hold all of the display information for what will happen to our column in it’s default display mode, when a mouse rolls over a row in the column, and when we press on a row in that column. 

Button Looks

Our resulting lister with the configuration above should look like this:

Styled Lister

This is great, but what can we do with this configuration? There are some exciting pieces to look at right away. First, if we look at our colDefine Table we’ll see that for column 2 we set this to be editable with the value 2 – this means if we double click on one of those cells we can update the value. We also turned on the “Auto-sync Input Table” parameter, which means if we update any values in our table those changes will be pushed back to our input table. 

This is great, but let’s do something with this lovely button we’ve added to our lister. We can look at some simple actions that we might accomplish here by opening up the lister’s callbacks. We can open up the callbacks by clicking on the parameter on lister called “Edit Callbacks”.

Edit Callbacks

Here in the lister callbacks we’ll make a few changes. What I want to do is to delete a row from our lister when we click the “X” button. To do this we need to know a few things – where is the lister that holds all of the data, which column have I clicked on, and which row do I want to delete. It’s not too hard to figure out these pieces. We can determine which lister we’re working on by looking at which component our listerConfig base is docked to. We can then find the row and column from the info object that’s passed to the lister callbacks. Finally we can use the DeleteRows() method from the lister extension to remove a row from our lister. Here’s the handful of python we’ll need to do this:

def onClick(info):

	# find our row and column
	clickedCol 	= info.get('colName')
	clickedRow 	= info.get('row')
	owner 		= info.get('ownerComp')

	print(info)

	# only if we've clicked the column that's named 'close'
	if clickedCol == 'close':
		# delete the row
		owner.DeleteRows([clickedRow])
		
	else:
		pass

The result should be a lister than we we click the “X” it removes the selected row:

Updating Lister

A quick overview of what’s happening in the callback – here we’re using the “onClick” method from the available callbacks. Unlike TouchDesigner’s built in ops, that start by showing you all of the available callbacks, for lister you need to check the documentation to see all of the available callbacks. When we click on a cell in lister we start by setting a few variables: our owner COMP, what row was clicked, and what column was clicked. Next we do a quick test to make sure the column that the command came from was the close column. This name comes from our colDefine Table. If the cell we clicked on is in the close column – which is what holds our buttons, the we use the provided method with lister “DeleteRows()” to remove. 

Creating a Button Table

Similar to the technique for adding a button as a single column, we can also think of our entire lister as being a set of buttons. For this example we’ll work with a slightly different input table DAT:

Input Table DAT

Our lister pars are also a little different for this example:

Lister Pars

Our colDefine table will hold a large portion of our configuration information for this approach. The set-up here looks like:

colDefine Table

I took some time to style my button TOPs – though this works even if you don’t style them:

Button Looks

Our resulting lister of buttons looks like this:

lister of Buttons

Let’s add one extra step so we can change a table when we press a button. Let’s add a table DAT to our network and call it table_info. Let’s set it up to have three rows and two columns. The rows should be named: row, col, text.

Adding an info table

We’ll open up our callbacks again, and this time we want to update our table_info with the row, column, and text that we’ve clicked on. We can get this information from the info object. Here’s a look at the python we’ll need to make this work:

targetTable = op('../table_info')

def onClick(info):
	clickedRow = info.get('row')
	clickedCol = info.get('col')
	cellText = info.get('cellText')

	targetTable['row', 1] = clickedRow
	targetTable['col', 1] = clickedCol
	targetTable['text', 1] = cellText

Now when we click on a cell in our lister our table updates with the changes.

TOP Path Example 1 – Press and Roll Changes

So far we’ve just used the button cellLook. We can achieve a similar effect by using a setting a topPath for the look of our cell. The idea here is that we have an image in a TOP that we’d like to place in a cell based on a set of possible variables. The easiest would be to change the image based on if the mouse is over the cell, or if we’ve pressed on the cell. That can be a little hard to write our minds around, so let’s look at an example to see this in action. 

We’ll start with a simple table DAT that has some information we’d like to display:

Input Table DAT

Next, let’s think about what kind of button interaction we want to create. For this example, let’s just look at how we might change the view of our button when we interact with lister – just a visual cue that we’ve clicked on a button. 

So just like with our previous example that used a TOP Path to change the contents of a cell with the provided TOPs, we can create our own TOPs for this. Let’s start by creating a button look, a roll, and a press look. Here’s what my network looks like:

Button Looks

Here that’s a top called “btnImage” along with tops that start with that same name and are appended with “Roll” and “Press.” This is important, because lister relies on the names of our TOPs to order to know which to display at what time. 

Next let’s look at the colDefine Table:

colDefine Table

Here we can see that our sourceDataMode is empty – which is exactly what we want in this case. We don’t want to display anything in particular, just our image. Next in our topPath row we can see that ‘btnImage’ is what’s describing which TOPs lister is going to use to fill in these cells. One final note on formatting here – when working with TOPs that have images in them I generally prefer to make sure that I don’t allow my cells to stretch. This can make sure that you know the exact dimensions of your cell, and can format your texture operators accordingly. In this case, the dimensions for our cells are going to be the height of the top called master – which is used to define a number of characteristics about our lister, and from the colDefine table row called “width”. 

The result from our configuration here is a lister whose buttons change color when we mouse over them and when we click them:

roll and press mechanic

But what if you don’t want to just use Roll and Press?!

That’s a great question, and there’s a solution here. Let’s look at a static example first, and then we’ll start to update our lister’s images.

One approach is to think about putting data in a column in a DAT table that you don’t need to display, but can instead use a reference for the state of your row. For example, let’s look at a table where we’ve added another column called ‘state’. In this column I’m just going to set state to be True if col2 is greater than 3.

input Table DAT

In the colDefine table we’re going to make a few adjustments. Here we’ll again use a topPath but we want to make sure that our sourceDataMode is ‘blank.’ This will keep the lister cell empty, even though it holds some data that we’re going to use. Next in our topPath we’re going to use ‘btnImage*’. In this case the * is going to act as a pattern matching mechanism; this matches the name of our tops to the contents of that column. Our state column had values that were “True” and “False” – this means that if I create TOPs with the names ‘btnImageTrue’ and ‘btnImageFalse’ they’ll be matched to display in the cells.

Let’s first look at the TOPs we need in our listerConfig:

Button Looks

Between our colDefine table and our additional TOPs, what we end up with should look like this:

Input Table DAT and lister

This is great for a scenario where you have some internal logic that’s updating a table that you want to see in a list, but don’t want to change directly. This could be online / offline status, or any number of other kinds of information. 

But… what if you do want your interaction with the table to update it’s display? Let’s look at that next.

TOP Path Example 2 – Updating a Single Cell

In some cases you just want to toggle the state of a single cell. We already have all the ingredients we need for this spell, we just have to put our magic words in the right order. 

We’ll start by setting up our input table:

Input Table DAT

Next let’s add some additional TOPs in our listerConfig to act as our buttons:

Button Looks

Next let’s make sure our colDefine table is setup correctly:

colDefine Table

Our last step is to fix-up our callbacks. We have all the ingredients in place, now we just need to set-up the logic that will handle how our buttons change. Our callbacks should look like this:

def onSelectRow(info):

	# find our input table DAT
	targetTable = info.get('ownerComp').par.Inputtabledat.eval()
	
	# find our row
	row = info.get('row')

	# check the contents of our cell - and set them to be the 
	# opposite of their current state
	stateVal = targetTable[row, 'state'].val

	if stateVal == 'True':
		updateVal = 'False'
	else:
		updateVal = 'True'	

	targetTable[row, 'state'] = updateVal

The big idea is that we check to see if the contents of our target cell are “True” or “False” and then swap them accordingly.

Toggle buttons in lister

This approach works great for a toggle style system, where any given row can be on or off. But… what about if only one row can be on? We’ll use all the same mechanics, and look at that next! 

TOP Path Example 2 – Updating a Whole Column

For this example you can set-up your input table and your colDefine table the same way. I’m going to change up the look of my buttons, but feel free to use the styles of your choice:

Button Looks

The real fun will come with our callbacks:

# find the intput table DAT
inputTable =  info.get('ownerComp').par.Inputtabledat.eval()

# find the data from the column called 'state'
stateCol = inputTable.col('state')  

# replace the contents of the colum
newStateData = [True if eachCell.row == info.get('row') else False for eachCell in stateCol[1:]]

# insert the header into the column
newStateData.insert(0, 'state')

# overwrite the column in the input table
inputTable.replaceCol('state', newStateData)

Unlike our previous approach where we just changed one cell, here we pull the whole state column. We then make sure the only cell that has a “True” value is the one we clicked on, then we overwrite the column in the original table. This wouldn’t be great for DATs with lots of rows, but for small – medium sized lists this might help you map out that “only one active” use case.

Single row toggle

Working with Lists…

Working with tables is cool, but part of what makes lister particularly powerful is that it can also work with lists of other objects. Lists of lists, lists of dictionaries, even lists of python objects. We’re going to stay out of the deep waters here, but it’s worth a little exploration to see how you can use python lists with lister. 

A Single List

Before we get started, let’s look first at how we can pass lister a regular python list as a data structure. 

On lister’s Advanced page there’s a parameter called Raw Data:

Raw Data parameter

We can fill that parameter in directly, or we can write a little script to fill in that parameter. For practice, let’s write a little script that’s going to do that work for us. In a text DAT let’s write the following:

A simple python list
myList = [ 'Monday',
	'Tuesday',
	'Wednesday',
	'Thursday',
	'Friday',
	'Saturday',
	'Sunday']

op('lister').par.Rawdata = myList

Here we start by making a simple list, and then we set the Rawdata parameter of our lister to be this list. Easy. 

Let’s set-up the colDefine table for our lister to work with our python list:

colDefine Table

Because our raw data doesn’t have a header, here in our colDefine we can set an internal name for this column of data, as well as a display label for this column. We should now see a lister that looks like this:

simple list in lister

A List of Lists

This is a great start, but what if you wanted more than one column of data? With lister you can also structure your raw data as lists of lists. This is a little different than our last example, but as we take this apart this idea it should all make sense.

Let’s think about tables for a moment. We often think of tables as a 2D array, in fact in TouchDesigner we use this idea all of the time to describe how we want to either retrieve or set contents in a table – we need to know the cell’s address as a row and column location. If we need to transform the contents of a table into a python object we might start to pull apart the best way to structure this. As a data structure, lists have an inherent order – so we might use that idea transforming a table into something more pythonic. One way to organize this data would be to make a list of rows – where each row is also a list. 

Let’s look at that as an image rather than just words. The idea here is that a table organized like this in TouchDesigner:

A simple table

Could be arranged like this as pure python:

A table as a list of lists
listOfLists = [['1', '2', '3', '4'], 
		['5', '6', '7', '8'],
		['9', '10', '11', '12']]

Okay… so with that little bit of data organization unpacked, we can set up lister with a list of lists as a data source. Let’s use a little script again to set the parameter for Raw Data in lister:

Setting our Raw Data par
listOfLists = [['1', '2', '3', '4'], 
		['5', '6', '7', '8'],
		['9', '10', '11', '12']]

op('lister').par.Rawdata = listOfLists

Next let’s set up our colDefine table. This is where things start to get a little different from our previous work. Since our list of lists doesn’t have a “header” like our table might, we need to do a little more work to make sure that this will behave correctly. We need to specify the names for our columns in our lister, and we should set the sourceData mode for our lister. Lister will do it’s best to figure this out for us, but we’ll ultimately be happier if we set these manually as we’re learning. When working with a list of lists the sourceData is the index in our lists of rows that we want to display – it’s like a column in our constructed array of data.

colDefine Table

The resulting lister should look like this:

list of lists in lister

A List of Dictionaries

So far we’ve looked at using a single list as raw data, or a list of lists. But what if we had a list of dictionaries? Unlike lists, dictionaries don’t have an explicit order. Instead, dictionaries work as key and value pairs. We’re going to use this concept when we work with a list of dictionaries for our lister. Let’s start by first getting our data ready. 

A list of dictionaries
listOfDicts = [
	{'fruit': 'apple', 'quantity': 10, 'icon' : 'Apple'},
	{'fruit': 'pineapple','quantity': 5, 'icon' : 'Pineapple'},
	{'fruit': 'grape', 'quantity': 2, 'icon' : 'Grape'},
	{'fruit': 'cherry', 'quantity': 6, 'icon': 'Cherry'}
]
op('lister').par.Rawdata = listOfDicts

Here all of the dictionaries contain the same keys. We’re going to use the Keys in our dictionary to be the columns for our lister, and the values that correspond to those keys as the contents of each cell in the rows of our lister.

I want to have little icons for my example, so I’m going to add a few extra TOPs that will act as buttons in my listerConfig:

Button Looks

Finally, let’s update our colDefine table for our lister. Here it’s important to note that the sourceData for our columns refers to a key in the dictionaries of our list’s items:

colDefine table

The result should be a lister that looks like this:

A list of dictionaries in lister

A List of Python Objects

Finally, this brings us a list of python objects. If you’ve gotten this far, I’m going to assume that you have a handle on using classes in Python. If that sounds unfamiliar, that’s okay – first take a moment to do a little googling about python classes, and look at extensions in TouchDesigner. Both of those ideas are important to understanding this next example.

We can accomplish this example by using modules on demand in TouchDesigner, but this is likely a better candidate for working with an extension. The big idea is that we’ll have a class that acts as the manager of our list objects. Those list objects will all be instances of a class themselves. We’ll create an instance of our manager class with another text DAT so we’re not continuously re-initializing the class when we interact with lister. 

My objects are going to have a handful of attributes – I’m going to define these in a dictionary for now, but you might construct these on the fly. This might be looks, presets, scenes, or any other type of collection of data.

Looking at this we’ll see that there are two classes – our first class is our MangerMod – this holds a list of our objects, and does the work of filling the list of objects. Next we can see the myObj class. Right now this only sets the attributes of the objects. 

A list of python objects
refData = [
	{'name' : 'apple',
	'color' : (0.454, 0, 0, 1),
	'top' : op('iconApple')}, 
	{'name' : 'pineapple',
	'color' : (0.719, 0.319, 0, 1),
	'top' : op('iconPineapple')}, 
	{'name' : 'grape',
	'color' : (0.6009, 0.827, 0.4697, 1),
	'top' : op('iconGrape')}, 
	{'name' : 'cherry',
	'color' : (0.258, 0.124, 0.5, 1),
	'top' : op('iconCherry')}]

class ManagerMod:
	def __init__(self):
		self.myObjs = []
		self.fillObjs()
		pass

	def fillObjs(self):
		for each in enumerate(refData):
			newObj = mod.myMOD.myObj(each[0], each[1])
			self.myObjs.append(newObj)


class myObj:
	def __init__(self, index, infoDict):
		self.myIndex = index
		self.fruit 	= infoDict.get('name')
		self.color 	= infoDict.get('color')
		self.icon = infoDict.get('top')

Next we create an instance of our manager class in a text DAT:

modules on demand

We can now target the myObjs member of our instance in the Raw Data parameter of our lister:

Python object list in the Raw Data parameter

Okay… now we’re ready to have some fun. Let’s look at our colDefine table in lister:

colDefine table

Notice how in sourceData we now use the name of the attribute of our python object. Our resulting lister looks like:

list of python objects

Wooooo!

At this point you might be rightly wondering why on earth you’d want to do this much work if you could just use a table. That’s a fair question, part of that is difficult to show in simple examples. That said, it’s not hard to imagine a situation where all of your critical data is stored in an extension, and rather than writing that to a table DAT (and then keeping it synced with your extension) it would be easier to just reference that in a lister directly. That’s part of the magic of this variation on using lister – you can use this to directly display pythonic pieces that might otherwise be more frustrating or time consuming to write out into a UI friendly data structure. 

This week we looked at a TON of variations in using lister – from simple alterations, to more complex uses of python. You can find the examples from list post here:

Big Bad Ass Lister Part 2 | Examples

Next week we’ll hear from the puppet master himself – Ivan – about the Dark Dank secrets of lister. 

Happy programming – and Lister…ing

TouchDesigner | The Big Badass Lister | Part 1/3

The Big Badass Lister

Part 1 of 3

This one is a different in part because we’ll be looking at something from the TouchDesigner pallet in depth, and also because Ivan from Derivative has very graciously volunteered to help demystify one of the coolest components that you’re not using enough in your projects – Lister. 

There’s a few things to keep in mind as you read through this post – there are two authors: Matthew and Ivan. They talked through what pieces would be great to share about lister, and then split up the examples and writing – that means that you’ll notice a slightly different voice in the writing, and you’ll get to see two different perspectives on how lister works. It’s also worth noting that Ivan is the developer at Derivative who has spent the most time working on lister, and the person who knows the Dark Dank Secrets of this component. There are also a handful of examples that go with everything written here, and we’re hoping that this post helps you find new and exciting ways to use lister in your projects!

Lister Basics

What’s a lister anyways? ! Lister is a shared component that both lives in the palette browser as its own comp, and is deeply integrated into the latest widgets UI kit Derivative has been cooking up. You can find both of these in the Palette browser on the on left hand side of the TouchDesigner UI, or by pulling up the palette browser with ctrl + l (win) / cmd + l (mac). 

Lister from the UI Folder in the Palette
Lister from Basic Widgets

Lister is actually built on top of the List COMP. The List COMP is a wildly powerful component, and replaces some of the pieces that you used to do with the Table COMP. If you haven’t used the Table COMP before, it’s amazing… but gets a little unwieldy pretty quickly. The Table COMP can be used to make radio buttons or other list style interfaces (the op create dialogue is a good example of the Table COMP in action), but it’s all set up with Table DATs, and can be a little hard to read / understand.

The list COMP

The List COMP does many the same things, but unlike the Table COMP, the List COMP has a deep set of python integrations with an internal callback DAT that allows you to think about UI building and operation from a Pythonic perspective rather than with Table DATs. That’s very exciting, but to make a truly reusable and extensible UI component you have to write a lot of Python… and this is where Lister comes into play. Lister is built on top of the List COMP and solves a large number of challenges you might typically encounter. It still has all the raw power of the List COMP, but gives you a huge headstart over trying to create a list UI element from the ground up.

So what can you do with Lister? Anything you might use a list for!

With that said, let’s actually look at how we can get started with lister, and some of the ways we can start to style and configure a lister for use in your projects.

Configuration

When it comes to working with lister, the config COMP is where you’ll actually find all of the most important secrets. The config COMP is docked below lister, and has a few easy to access helpers. On lister you’ll find a handful of pulse open parameters that will make configuration easier to set-up. 

Before we can really dig-in let’s set-up our workspace so we can see some of the interesting pieces of lister in action. To begin we’ll start with a Table DAT that holds a set of rows and columns that we want to turn into a UI element. 

We can start by creating a Table DAT that has two columns and a few rows. I’m going to give mine 6 rows, and name the columns “col1” and “col2”.

A Simple Table DAT

Let’s also add a lister from the palette – in this case we want the lister that’s in the UI Folder.

lister from the palette

To see just how fast lister can be to use, let’s start by just connecting our table DAT to our lister COMP:

lister out of the box

Right away we’ve already got a table up and running. This is a nice start, but we can do a little more to better configure our lister UI element. Some of the most important configuration pieces can be found in the docked base called “listerConfig”:

listerConfig

The listerConfig Base holds the sweet secrets for our lister – let’s take a closer look at how we might set a few configuration elements. Before we dive inside of the config, let’s first set a few of the parameters on lister.

On lister let’s first turn off the Auto-define Columns parameter:

Change pars on Lister COMP

Let’s also change the parameter “Input Table Has Headers” to True – since our input table has headers:

Table has header pars

Like it’s parameter name sake, Auto-define columns will attempt to set-up your lister based on the input table. For this example, we’re going to turn off that parameter, and then click on the “Edit Column Definitions” pulse parameter. This will open a floating window of the colDefine table DAT inside of the listerConfig:

colDefine Table

This table contains many of the configuration elements for our lister’s look. We can see that the table already has some elements set up. We’re going to replace these with our own values so we can see how lister will change. 

First let’s set the column name to be displayed as our header. Our starting table contains fruits in col1 and their quantities in col2 – So I want to change the column Row to be “Fruit” and “Quantity” respectively. 

Our lister’s data structure supports an internal column name that’s different from the display label – in some advanced cases you may want to use this. For now, let’s just set the columnLabel row to have a value of “*” which we can read as – “use the value from the column row.” 

Next let’s change the sourceData row to be the name of the input table’s columns that we’d like to use. My input table had columns called “col1” and “col2” – what’s important to consider here is that our lister’s columns don’t have to be arranged in the same order as our input table. This can be very helpful for more complex list UI elements. 

Let’s set our sourceDataMode to be “string” for both of our columns.

For now let’s style the width and stretch parameters for the column to be the same. Each column can have a width value of 100, and a stretch value of 0. At this point you should have a config table that looks something like this:

finished colDefine Table

Your lister should look something like this:

lister with Updated colDefine Table

Before we experiment with styling our lister anymore, let’s quickly take a look at some of the handy features that already exist. For starters we can see that there’s a hover highlight from our mouse position, and when we click on a row it highlights another color. 

lister with Highlighted rows

Lister also has two DAT outputs. The first lister DAT output is a table DAT version of exactly what’s in lister. The second output is the header columns from our lister, along with the selected row or rows from our lister:

lister output DATs

Let’s next look at how we might style our lister’s output a little more. 

In addition to the column definitions table, there are a host of additional operators we can change inside of our listerConfig DAT that are worth playing with. Let’s head back to our lister’s parameters, and this time click on the pulse parameter called “Edit Config COMP”.

Edit Config COMP

This will open a floating network window that looks into the listerConfig base:

listerConfig COMP

In addition to the column definitions table that we’ve been working with, there’s also a whole cluster of other TOPs. The first two columns of TOPs hold text TOPs that are used to define how our lister is styled. 

By changing these TOPs we in turn change the style for the lister’s columns. Let’s take a quick look at what we might change and the impact that will have on our lister. 

Let’s start on the text TOP called “header” on the font page. We can see that a number of the parameters are set to read-only – but there are a few on this page of parameters that we can change. Let’s start by turning the font x size up to 12 and turning on the “Bold” parameter.

changing the lister Header font size and bold attributes

Next let’s move to the Color page, and change the background color for our header. Here I’m going to make the Font Color bright white, change teh background color to a plumb, and set the bottom border of my TOP to be Border A. You’ll want to make sure that you also set the Border A Alpha parameter to 1 – it’s set to 0 by default. The result should look something like this:

changing the lister Header color attirbutes

Next if we look at our lister we’ll see those changes in action:

results from changes in listerConfig

This same idea also works with our rows – we just have to edit the first column of TOPs in our liserConfig base. I’m going to change my TOPs so that my lister uses this plumb / purple color palette. The only changes I have to make will be to the first column of TOPs:

styling lister rows

The resulting look for the whole lister is something like this:

styled lister

If you wanted to also change the row heights, you can achieve that change by adjusting the header TOP’s vertical resolution, and the master TOP’s vertical resolution. I’m going to change the header TOP’s resolution to 36, and the master TOP’s to 30. I’m also going to turn on row Striping on the lister COMP. The resulting changes look like this:

lister with height adjustments

This is just a start of the kind of customization we can achieve with lister. Now that we have a handle on some of the essential elements for manipulating the look of our lister, let’s take a deeper dive into customization and other lister features.

Auto Config and Copy Auto-Cols

For a simple input table this is a pretty straightforward approach, but sometimes we end up with a table that’s slightly more complicated. A good example of this might be when we’re converting SOP data to a table – this can be helpful in all sorts of situations, but setting up our lister for this many columns could take a  minute. Let’s set up a quick table that holds data converted from a sphere SOP.

We can start with a sphere SOP and change its primitive type to Polygon (this will start us off with fewer rows). Let’s add a null SOP (in case we want to add any other transformations), and then use a SOP to DAT to convert this in a table format. 

SOP to DAT

Similar to Houdnini’s geometry spreadsheet, this table holds all sorts of useful information about our SOP. This often works great for internal project uses, but if we want to create a styled interface for this data we would have to think about how to display this information. 

Geometry Spreadsheet

A lister here can be a great help. Let’s add a new lister form the palette to get started. Next we can connect our DAT to lister.

lister from Geometry Spreadsheet

Lister starts with the “Auto Define Cols” parameter on, which helps us get a starting point from our data right away. If, however, we want to style our lister a little more we’d need to repeat the steps we practiced earlier – going though and creating a column in our colDefine table for each column that we want to display. That’s all well and good, but it’d be oh so nice if there was a way to copy the current auto-config format into our colDefine table to give us a headstart. This gives us an opportunity to look at one of the handiest parameters on lister – Copy Auto-Cols to Config. 

For a table with this many columns it might be a little cumbersome to go through and set all of the colDeinfe elements manually – which makes this parameter incredibly helpful. So let’s go through an setup this lister so we can see the “Copy auto Cols To Config” in action. First, I want to turn on the parameter “Input Table Has Headers.” This SOP to DAT has header information already, and I want to make sure that the first row is treated as a header.

Input Table has Headers

Now we can use the pulse parameter called “Copy Auto Cols to Config.” A good trick to keep in mind here is that pulsing the “Recreate Auto-columns” parameter will adjust column widths to their contents when you press it. This would be helpful when you’re looking to have your columns auto size themselves. 

For this example we’re going to make some changes to our columns after we’ve captured them in the colDefine table. Let’s pulse the “Copy Auto Cols to Config” parameter. After we pulse this parameter, let’s turn off the “Auto-define Columns” parameter so we can make some changes to our lister.

Setting our Auto-define pars

Now let’s use the pulse parameter on lister to open our Edit Column Definitions table. We should see that lister has done all of the hard work of setting up our column definitions table, and now we can focus on just modifying the contents of this table.

The initial Auto-Defined columns

Now that we have the initial work done let’s go through and make some edits to our definitions. For starters, we might consider changing the labels for our columns. The names that come from our SOP to DAT are great when working in TouchDesigner, but aren’t always as descriptive as we might like for a user interface. We can update what’s displayed in the headers by making some changes to the first row of our colDefine DAT. I’m going to change a few names:

  • Index -> Point Index
  • P(0) -> Point.x
  • P(1) -> Point.y
  • P(2 -> Point.z
  • N(0) -> Normal.x
  • N(1) -> Normal.y
  • N(2) -> Normal.z

I’m also going to remove the point weight column, and the groups column. I also want the columns to stretch to fill the lister’s width, so let’s change the stretch row to be 1 for each of the columns. I want the Point Index values to be center justified instead of left justified, so we can change the justify cell in column 1 to be CENTER. Finally, I want the text in the first column to be Bold, so let’s change the fontBold cell to be 1. The resulting table should look like this:

Our adjusted auto defined cols

While we’re here practice using the font TOPs inside of the listerConfig COMP to style your lister. I’m going to adjust my row heights, and add a little color. After a few adjustments, my listerConfig looks like this:

listerConfig updates

And the final lister looks like this:

Geometry spreadsheet lister output

In our next installment we’re going to look at additional mechanics for working with tables, creating buttons in lister, using simple callback functions, and deeper styling with TOP paths. 

You can download all of the examples from this post here.

Happy programming… and lister-ing.

TouchDesigner | SOPs and Stamps

Usually when it comes to working with SOPs I tend to focus on thinking about instances instead of copies – but there are some really excellent use cases for the Copy SOP and one of the most powerful pieces of the copy stop is stamping your copies.

If your shaking your head wondering what Stamps are, you certainly aren’t alone – they’re one of the lesser used features of the Copy SOP (in my opinion) but one that has a wide range of use cases. Recently on the forum there was an excellent question about using the data from CHOPs to set the characteristics of SOPs – Multiple Circle SOPs from CHOP samples. One solution to that particular question is using the ability to Stamp copies, and with that in mind we can pull apart how stamping works.

Let’s start with a simple example. Let’s begin by connecting a Circle SOP to a Copy SOP.

A Circle SOP connected to a Copy SOP.

When we start there’s nothing particularly exciting that happens in this example. In order to make some changes we need to first change our number of copies:

Number of Copies set to 10

This still isn’t particularly interesting just yet, all of the copies of our original circle are all in the same place – all stacked on top of one another. Let’s translate our copies by some small number, and turn on our wireframe view of our geometry so we can better see what’s happening:

Copy Translate X par set to 2

The first copy is moved 2 units to the right, the second copy is moved 4 units to the right, the next 6, etc. This puts all of our copies neatly next to one another.

What if, however, we wanted to change the number of divisions in each copy… we might start with a triangle, and add one division for each copy. There’s no good way to do that out the gate – and it might be easy to think that you need to use a Script SOP to make this work.

Luckily, however, we can make this work with stamping. Let’s start by heading over to the Stamp Page of the Copy SOPs parameters:

The Stamp Page

We’re going to start by first turning on our Stamp inputs parameter. Next let’s figure out what we want to stamp each Copy with… if we want to change the number of divisions in a circle based on what Copy it is, we need a way to attach the copy index to that piece of geometry. Lucky of us there’s a handy python member that let’s us do just that. First let’s name our parameter index, and next we can use a simple python expression to attach our copyIndex to our geometry:

me.copyIndex

me.copyIndex in plain English means – “which copy of the geometry I am.” We can see that our copy index goes up to 9, and like all things programming, we should remember that our indices start at 0 – so our copy index are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9. What’s interesting about using Stamps is that we first Stamp our geometry, but then go back upstream in our network to use that stamp.

So to use our stamp we now need to go to our Circle SOP. The syntax of using Stamps looks like this:

fetchStamp(name, default)

We just used the name index when we stamped our copies, so we might first start with:

fetchStamp('index', 0)

Which means, find the Stamp called index and if you can’t find it, use the value 0. BUT, before we use this expression we need to remember that a circle needs a minimum of 3 divisions – less than 3 and we’ll just end up with a line. Let’s make sure we add 3 to our stamp value since our number of copies starts at 0:

fetchStamp('index', 0) + 3

What does that look like:

Copy Index to define the number of divisions

Okay… that’s pretty cool. But that’s really just the start of our journey here. Where this gets really interesting is if we start to use this new super power to look at other operators – after all we now have a way to index our copies.

Let’s now look at something a little more interesting.

We might instead think about using a Noise TOP to help describe how we’d like to displace the vertices of our copies. The idea in this example is that we’ll start by creating a grid – we’ll use a point SOP to displace our verticies, a tips SOP to fan our one side, and a copy SOP to create a spiral:

Our Point SOP is going to displace the vertices of our SOP by using the point Index to locate the x sample position, and the copyIndex to locate the y sample index:

The resulting Spiral has a noisy feeling that we can adjust with a combination of our noise TOP and copy SOP:

This isn’t a technique that’s well suited for changing in real-time, but it is a great way to think about creating fixed geometry that you might use for any number of environmental elements in your pieces.

TouchDesigner | Instancing Examples

Four years ago I started collecting examples on working with instances. What started as a rough collection of ideas has turned into a collection of over 40 different examples. In this revised set of techniques the format and presentation has also changed.Initially the user had to mouse around networks and look for readmes to learn about the examples. In this new version all examples can be explored through a purpose built UI:

Explorer screen shot

This new starting point allows the developer to quickly jump between examples without needing to open the TouchDesigner network. The interface is re-designed with three primary interaction regions in mind: The Navigator, The Readme, and The Liveview.

The Navigator

The Navigator organizes examples into sensible groupings. These high level containers can be collapsed by clicking on their heading. Each example also includes a button to jump to the example, or to open a TouchDesigner network view of the example. This allows you to both see the example rendered and see how it was built all at the same time.

The Navigator Pane

The Readme

The Readme pane has a short description of what you’re seeing. This set of documentation is intended to go along with each specific example and includes discussion about the particular techniques employed. Using the globe network icon on the Navigator will open the readme in your browser if you’d like a slightly larger window for reading through the documentation.

Right now there’s a small display bug on macOS that prevents the scroll bar from working correctly – that’s okay, you can always use the globe icon to open the readme in your default browser instead. You can follow along with the bug report if you’re a mac user here.

The Readme Pane

The Liveview

The Liveview let’s you see the output render from the network at a glance — making it easy to determine if this technique is one that you’d like to learn more about.

The Liveview Pane

Jump to Network

In addition to exploring the examples through the viewer, there’s also a button to open the network view for the example. This lets you jump immediately into the network and see how the instances were set-up.

The Explorer and an open network view

This project has been a labor of love to collect, organize, and compile materials for other passionate developers and educators. I’m hoping that enthusiasts, instructors, and creative developers alike will find this useful and interesting both as a tool and as a reference. 

Github: Clone the GitHub Repo

Happy Programming!


❤️ Thanks ❤️

…to Zoe Sandoval for their tireless work editing, play testing, and making all of these examples better. If it wasn’t for their handwork, and attention to detail this tool wouldn’t be nearly as complete or well constructed.

… to Ian Shelanskey for his passion and encouragement to keep making examples, and craft collect them into a single unified example.

… to Michael Walczyk for his mentorship, guidance, and constant enthusiasm. I would have been lost in the GLSL wilderness without Mike, and I always appreciate when I can ask him some crazy question about rendering and mathematics.

… to Elburz Sorkhabi and everyone at The Interactive Immersive HQ. Many of you were kind enough to test and look through these examples before posting this. The engagement and hunger for learning TouchDesigner in that community continues to inspire me, and is a reminder about why these kinds of examples are useful and important to share.

… to the Derivative Team for their toolkit that never stops being fun to use, and for their thoughtful feedback and support through building out this collection. From combing through the forum, to late night conversations with Greg and Ben I wouldn’t have gotten this far without them.

A Conversation with Robert Erdös

Recently I had the great honor of participating in Robert Erdös’s recent project TOP Players. Robert is currently working with Les Ateliers Nomad – you can see some of their work on Facebook. During the roller coaster of stay at home orders, and social distancing Robert’s been interviewing TouchDesigner developers and artists and has put together a casual format for having real conversations about work, artistry, and one’s ever evolving practice.

From Contraption:

The first series of “TOP Players” recordings is focused on revealing the people behind the TouchDesigner community video tutorials, people who showed great generosity and skill in helping others find their way. 

One of the primary goals of these conversations is for users to gain access to the most valuable resources that lead to their success in the interactive world.

The 2020 Lockdown has offered me a unique opportunity to reach these wonderful people and try to share a perspective on their distinctive personalities and amazing work skills.

https://contraption.pro/

It was a joy to talk with Robert, and I’d encourage you to both checkout the page he made for this talk, and to follow along with the conversations he’s having with other artists and developers. You can learn more about Robert by checking out his instagram, contraption.pro, or Les Ateliers Nomad.

Follow Robert on

YouTube
Instagram
Behance
Facebook
Les Ateliers Nomad on Facebook

A Conversation with Cocolab

Zoe and I were recently invited to participate in an open conversation about “Making the Best of our Current Situation with the Power of Creativity” hosted by Cocolab. Cocolab is a creative studio based out of Mexico City doing work of all scales – from intimate experiences, to enormous projection events they’ve done it all. Like all of us, they’re working to understand how to navigate this strange time, and have been hosting an ongoing series of talks that explore how we might understand and navigate this changing world. Our conversation was moderated by the ever charismatic and playful Elburz Sorkhabi.

You can see our talk at the facebook link below, and if you’re interested in listening in future conversations you should check out their Instagram feed for the next artist and date.

Learn more about MIR

TouchDesigner | The Object CHOP

The Object CHOP has long been one of the most challenging CHOPs for me to really wrap my head around. Following along with some conversations on the Facebook Help Group 1, it’s clear that I’m not the only one who has bumped their head against how to take advantage of this operator.

With that in mind, here are a few tricks and techniques that you might find helpful when working with the object CHOP.

Distance Between Many Objects Part 1

At first glance, it seems like the object CHOP can only perform calculations between single objects, but in fact you can use this operator to perform calculations between many objects provided that you format the input data correctly, and set up your object CHOP to account for multiple samples.

In a first example let’s say that we want to find the distance between several green spheres and a blue box:

First let’s collect our position information. I’ve used an object CHOP per sphere to find its distance, but you might also use a script CHOP, or a put positions in a table that you reference for the spheres, or drive them with custom parameters. How you position them doesn’t matter. What we need, however, is a single CHOP with three channels that hold the transformation information of those spheres. My trick in this network is to use object CHOPs to find their positions, then put them in sequence with a join CHOP:

Next we can use a single object CHOP that’s fed reference positions from this join CHOP, and a target Geometry COMP:

Other important pieces here are the start and end parameters on the channel page.

This is where we set how many samples the object CHOP will evaluate. This can be a bit confusing – here especially as the join CHOP has started at a sample index of 1 rather than 0. The devil is in the details, so it’s worth keeping a close eye for these kinds of oddities. Because of this we compensate in our start position by moving back one sample index.

Next make sure to set your object CHOP to output measurements, and distance. What you’ll then end up with is a single channel with a sample for each distance between your box and spheres. We can convert this to a table if we wanted to see the actual values:

Distance Between Many Objects Part 2

We may also want to measure distances between multiple blue boxes. Say, for example, that we had two different blue boxes and we wanted to know the distances of our spheres to both of those boxes?

Similar to our first exercise we’ll start by collecting all of the position information for our spheres. We also need position information for our boxes. In this case, however, we need to stretch a single sample to be 4 samples long – this is part of our data preparation step to ensure we correctly calculate distance.

Here a simple stretch CHOP has been used to make sure we have four samples of data for each box. Next we can join this data so all of our box position information is in a single set of CHOP channels:

Before moving on, we need to take a moment to adjust our sphere position data. In our first example we only collected the four positions… we need to set up the correct extend behavior for this series so that our CHOPs know what values to use when CHOPs of mismatched lengths are combined. We can use an extend CHOP set to cycle to do this trick:

Finally, we can then use an object CHOP to calculate the distance between our box and our spheres:

Distance Between Many Objects plus Bearing

If we also calculate the bearing between our boxes and spheres, we’ll end up with rotation information… what can we do with this? We could use this to calculate the correct rotation for a set of instances. For example:

Here each line is correctly rotated, scaled, and placed based on calculations from the object CHOP.

Bearing

You can also use the object CHOP to just calculate bearing – or rotation from one object to another. Here you can see how this might be used to rotate instances to sit flat on a sphere’s surface, or rotate an arrow to point towards an object:

Bearing and Distance

Or you might use the combination of bearing and distance to make some strange abstract art:

Collision

You can also use the object CHOP to simulate a kind of collision calculation where the distance you’re measuring can help you tell how close an object is to another and if they’re on top of one another:

GitHub

Clone the Repo to Follow Along

TouchDesigner | Reflection and Refraction

I can haz Reflections?! Refractions?

Zoe loves all things reflective and refractive and it was almost a year ago that they started looking into how to achieve compelling illusions of reflection and refraction. Then I went to Macau, then Chicago, then Zoe dove headlong into their thesis project… fast forward to 2019, and it was time for me to finally follow through on a long overdue promise to create some examples of reflection and refraction in TouchDesigner. It didn’t hurt that Zoe gently reminded me that it was time for more refractive rendering in life. Good places to start for these kinds of questions are to look at existing references in the world. 

Reflection 

Reflections are hard. In part because they often mean that we need to see the whole world – even the parts that our virtual camera can’t. We might know this intuitively, but the reach of this is easy to forget. When we point the camera in our smartphone at a mirror we see ourselves, the world behind us, above us, and and and. If we point a virtual camera at a virtual mirror we need the same things. That can be a wobbly bit to wrap your head around, and develop a better sense of this challenge I look a look at a reference book I picked up earlier this year – OpenGL 4 Shading Language Cookbook – Second Edition. This has a great chapter on reflection techniques, specifically generating them by using cube-maps. Cubemaps look like an unfolded box, and have a long history of use in computer graphics. 

One of the primary challenges of using cubemaps is that you need to also know the perspective of the object that’s reflective. In other words, cube maps can be very convincing as long as you move the camera, but not the reflective object. But what if we want the option to both move the camera, and the object? In this quick tutorial, we look at how we can use a cube map to create convincing reflections, as well as what steps we need to consider if want not only the camera to move, but the object itself. 


Refraction

The one and only Carlos Garcia (L05) has a great example posted on the TouchDesigner forum. This great example helps illustrate the part of what we’re after with this kind of work is the sleight of hand that hints at refraction, but isn’t necessarily true to the physics of light. Almost all realtime rendering tricks are somewhere between the Truth (with a capital T) of the world, and the truth (sneaky lower case t) of perception. We’ve all fallen for the perceptual tricks of optical illusions, and many times the real work of the digital alchemist is to fool observers into believing a half truth. Carlos’ example proves just that point, and helps us see that with a little tricksy use of the displacement TOP we can achieve a healthy bit of trickery. 

That’s an excellent start to our adventure, but we can dig-in a little more if we keep searching. Another post on the forum links over to an article on medium that showcases an approach for webGL that leverages the use of a UV map to “pre-compute” the direction of displacement of the light that passes through a transparent object. This is an interesting approach, and in fact there’s a middle ground between the webGL example and Carlos’ TOX that gives us some interesting results. 

In the following tutorial we can see how we remix these two ideas. The big picture perspective here is that we can leverage TouchDesigner’s real-time rendering engine to provide the same “pre-computed” asset that’s utilized in the webGL approach, and then use Carlos’ displacement TOP technique. We can also short-cut Carlos’ use of a rendering a second version of the object as a mask, and instead use a threshold TOP looking at our alpha channel to achieve the same effect. This isn’t a huge change, but it’s a bit faster in some implementations and saves us the wobbles that sometimes come with multiple render passes. Finally, a little post processing can help us achieve some more convincing effects that help sell our illusion as a true to the eye. 


Playlist


Individual Videos