Back in May I wrote a quick piece about how to use a camera as a mask in Isadora. This is a very powerful technique for working with live cameras, and no matter how many times I see it I’m still struck by how fun it is. This isn’t difficult to program, and one of the questions I wanted to answer this summer was how to create a network in TouchDesigner that could accomplish the same process. Before we get started it’s going to be important to make sure that you have a camera (a web-cam is fine) connected to your computer. While you can create this same effect using any recorded footage, it’s certainly more compelling and playful when you’re working with a live camera. You’ll also want to make sure that you know how the feedback TOP works in Touch Designer. If you’re new to this programming environment you might take a moment to read through how to work with the feedback TOP here.

Getting Started

We’ll start by creating a new project, and by selecting all of the standard template operators and deleting them. We can stay inside of the default container that TouchDesigner starts us off with as we create our network. Let’s start by taking a look at what we’re going to do, and the operators that are going to be involved.

Unlike other tutorials, this time we’re going to work almost exclusively with Texture Operators (TOPS). The effect we’re looking to create is to use the feed from a live camera as a mask to hide, or reveal, another layer that could be either a video or photo (in our case we’ll work with a photo today, though it’s the same process and the same TOP when working with video). To do this we need to first remove a portion of our video stream, we’ll create a little bit of motion blur with the Feedback TOP, next we’ll composite this mask with our background layer, we’ll finish by creating a final composite with a black background layer.

Some like it Hot

Without a kinect we’re really just approximating separation. Luckily, if you can control your light there are some ways to work around this. Essentially what we’re looking to create is an image where the hottest (or brightest) portion is the subject we want to separate from the background. In another post I’ll talk about some more complicated methods, for now lets look at what we can do with just a camera and some light.

We’ll start by creating a string of TOPs like this:

Movie In – Monochrome – Threshold – Feedback – Blur – Level – Composite

You’ll want to make sure that you’ve connected the Threshold to the second inlet on the Composite TOP, and assigned the Target TOP of the Feedback operator as the Composite operator in this first portion of the string.

Remember to adjust the opacity on the Level operator to a value less than .95 and greater than 0.8. You will also need to spend some time adjusting the parameters of the Blur Operator to fine tune the aesthetic that you’re after.

Threshold

The Threshold TOP is going to be your best friend in this string. This operator is going to help you control how much background you throw away. By adjusting the threshold value you’re controlling the pixel values that get passed as white and what values are thrown away and converted to alpha. This means that as long as you can keep light on your subject, and mostly off of your background, you’ll be able to isolate the subject from unwanted background pixels. This will take some adjusting, but it’s well worth your time and attention. If you need a little more fine grain control here, you insert add a Level TOP to adjust your image before it gets to the Threshold TOP.

Composite Composite Composite

The final component of this network is to composite all of our images. First we’ll need to add a Movie In TOP as well as a Constant for our backgrounds. Next we need to add two more Composite TOPs and finally a Null. It might be useful to think of this as a string that looks like three Composite TOPs ended in a Null with some additional operators at each stage. First our composite of our live camera and feedback string is going to combined with our Movie In TOP. In the Composite TOP’s parameters make sure that the Operand method is Multiply. This replaces the white values from our previous string with the pixel values from the Movie In TOP. Next we’re going to composite this string with a constant. In this case I’m using a constant black background. Depending on the venue or needs of a production you might well choose another color, you can do this by adjusting the parameters of the constant TOP. Finally we’ll end the whole string with a Null.

We’ve now used a live feed as a mask to reveal another video or image. Next you might think about where in these strings you might think about adding other operators to achieve different affects or moods. Happy programming.