I love a good challenge, and today on the TouchDesigner slack channel there was an interesting question about how you might go about getting the contents of the textport into a texture to display. That’s a great question, and I can imagine a circumstance where that might be a fun and interesting addition to a set. Sadly, I have no idea about how you might make that happen. I looked through the wiki a bit to see if there were any leads, and it’s difficult to see if there’s actually a good way to grab the contents of the textport.
What do we do then?!
Well, it just so happens that this might be another great place to look at how to take advantage of using extensions in TouchDesigner. Here our extension is going to do some double duty for us. The big picture idea is that we’ll want to be able to use a single call to either display a string, or print and display a string. If you only wanted to print it you could just use print(), so we’ll leave that one out of the mix for now.
Let’s take a look at the extension and then pull apart what’s happening in side.
'''
author | matthew ragan
web | matthewragan.com
The idea behind the display class comes from a question on the TouchDesiger
slack channel about how to handle displaying text for performance. While you
might choose to display the console, the challenge here is that you'll end
up with another window open on your output machine. Ideally, you only draw
a single openGL context - this results in the best possible performance
from TouchDesigner. With that in mind, how might you approach wanting to use
the textport in an artistic way for your project?
This example tackles that challenge with a set of methods designed to
print to a texture intended for display, as well as to the text port.
'''
class disp():
def __init__( self ):
''' our init method does a bit of set-up and starting organization.
We need a few things to be available to us in this method. It's handy
to have the target table, the target text TOP, a message with a placeholder
and some information for our start and end rows to be displayed.
'''
self.TextTarget = op( 'table_target' )
self.DisplayText = op( 'text_for_display' )
self.MessagePrompt = ">>> {msg}"
self.StartRow = 0
self.EndRow = 11
self.TextTarget.clear()
return
def Display( self, inputMsg="hello world" ):
''' The display method will populate a table which is used to feed a text TOP
The big idea here is that we need to fill in a table DAT that will then feed
a text TOP. Why a table DAT instead of a text TOP? Well, you can do whatever you
want. My idea here is that a table DAT is a little easier to think of in terms
of single rows that can fill up a text TOP. We can also do fancy things like use
a select to make sure that our text is always displayed even if our start and end
rows get away from us. A text DAT will just keep filling our text TOP, and
we might end up with squashed text we can't see.
In this method we format our MessagePrompt with the supplied text. We also do a
quick check to see if our text will fit in the display. If we have more rows than
can be displayed, we update our class variables to so that our last lines will
remain visible.
Finally, we return our formattedMsg - why? This is handy if we want to use another
method to actually print our message to the textport as well.
'''
formattedMsg = self.MessagePrompt.format( msg = inputMsg )
self.TextTarget.appendRow( [ formattedMsg ] )
if self.TextTarget.numRows > 11:
self.StartRow += 1
self.EndRow += 1
else:
pass
return formattedMsg
def PrintAndDisplay( self, inputMsg="hello world" ):
''' The PrintAndDisplay method will both add the message to the text TOP, and
print the same message to the console.
You may be wondering why we return the formatted message - the trick here is that
we can take advantage of the work that arleady happens in the Display method.
Why write that whole bit again, if we can just use the same logic and work
we've already done, and just add a final step of printing to the textport as well.
'''
forConsole = self.Display( inputMsg )
print( forConsole )
return
def ClearDisplay( self ):
''' Clear display dumps out the contents of our target table.
This method seems silly, but the idea is that you're likely to want
a fast and clean way of cleaning out the contents of your display window.
This method does only that simple little task. This will also reset our
start and end bounds to make sure that we're getting the correct selection
for our text TOP.
'''
self.TextTarget.clear()
self.__init__()
return
Okay, so what exactly are we doing here?!
The big picture is that we want a way to be able to log something to a text object that can be displayed. In this case I choose a table DAT. The reasoning here is that a table DAT before being converted to just a text DAT allows us to do some simple clean up and line adjustments. Each new entry is posted in a row – which makes for an easy way to limit the number of displayed rows. We can do this with a select DAT – which is where we use our StartRow and EndRow members.
Why exactly do we use these? Well, this helps ensure that we can keep our newest row displayed. A text TOP can accept a text DAT of any length, but at some point the text will spill off the bottom – unless you use adaptive sizing. The catch there is that at some point the text will become impossible to read. A top and bottom boundary ensures that we can always have something portion of our text displayed. We use a simple logical test in our Display() method to see if we’ve hit that boundary yet, and if we have we can update our members plus one… moving them both along at the same time.
You may also notice that we have a separate method to display and print… why not just do this in a single method. Well, that’s a great question. We could just use a single method for this with another argument. That’s probably a better way to tackle this challenge, but I wanted to use this opportunity to show how we might call another method from within our class. This can be helpful in a number of different situations, and while this application is a little too simple to really take advantage of that technique, it gives you a peak into how it might work.
One of the courses I’m taking in my first year at ASU is a course called Media Design Applications. This course is centered around the use of various media design techniques in specific relation to their application in a theatrical setting. One of the techniques that we discussed in class is architectural projection mapping. This form has quickly become popular for forcing perspective, and opportunity for complex illusion. The underling principal of projection mapping is to highlight and take advantage of physical from in order to create the illusion that the entire surface is, itself, a screen. There are a variety of techniques to achieve this illusion, some based entirely in software and others based in the process of generating the artwork itself. This is an essential and powerful tool for the media designer as it opens up a wide range of possibilities for the creation of theatrical illusion. Before I start to talk about the process, here’s the project description:
Project Description:
Component 2 – Geometry, Surface and Illusion
Unfortunately – or possibly fortunately – media designers in the theatre get a nice, rectangular, white screen to shoot at from a perpendicular, on-center angle. In this section, we will explore methods for dealing with odd angles, weird shapes, and non-ideal surfaces, as well as exploring special effects that are possible through the right marriage of projection, surface and angle. For this project, you may choose a building, sculpture or other built object in the ASU environment, then map its geometry using the techniques shown in class and create content utilizing that geometry to best effect. Final presentations of this project will be in the evening on campus.
I started by this process by first scouting a location. After wandering around campus several times I one of the buildings that I kept coming back to was a energy solutions building by a company called NRG. One of the larger draws of this building happens to be the location. Positioned directly across from one of the campus dormitories it seemed like an ideal location that would have a built-in audience. While there’s no guarantee that there will be many students left on campus at this point, it never hurts to plan for the best.
The face of the building that points towards the dormitories is comprised of abstract raised polygons arranged in narrow panels. These panels come in two varieties creating a geometric and modern look for the building. One of the productions I’m working on next year has several design elements that are grounded in abstract geometric worlds, and this seemed like a prime opportunity to continue exploring what kind of animation works well in this idiom.
In the Asset or In the System
The debate that is often central to this kind of work is centered around an approach that values building a system (or program) for creating the aesthetic, or to instead create the work as fixed media artifacts. In other words, do you build a something that is at it’s core flexible and extendable (though problematic, finicky, and unabashedly high maintenance) or do you build something rigid and fixed (though highly reliable, hardware independent, and reproducible)? Different artists prefer different methods, and there are many who argue that one is obviously better than the other. The truth of the matter, however, is that in most cases the right approach is really a function of multiple parameters: who’s the client, what’s the venue, what’s the production schedule, what resources are available, is the project interactive, and so on. The theoretical debate here is truly interesting, and in some sense calls into question what the skill set is most appropriate for the artist who intends on pursuing this practice. The print analogy might be, do you focus on designing within the limitations of the tools that you have or do you commit to building a better printing press so that you can realize the design exists only as an abstract thought?
Recent Arizona State MFA graduate Boyd Branch shared these thoughts about this very topic:
I don’t know if there is much of a debate between system building and design for production. Quite simply- every production demands aesthetics. The aesthetic is always the most important. The system is only useful in as much as it generates the appropriate aesthetic experience. It doesn’t matter how reliable, interesting, or functional a system is if it isn’t supplying an aesthetic relevant to production. A “flexible and extendable ” system is only useful if the aesthetic of flexibility and extendibility is ostensibly the the most relevant aesthetic. Interactivity is an aesthetic choice for performance and only relevant when ontology or autonomy are the dramatic themes. For theatre in particular, the system inevitably becomes a character, and unless that character is well defined dramatically, it has no business inserting itself into production.
The debate if any is internal for the designer and presented as a range of options for the producer/director. That debate is a negotiation between time and resources. A designer may be able to envision a system that can achieve an effect- but without sufficient experience with that system and the ability to provide a significant degree of reliability, such a system should not be proposed without articulating how the dramatic themes will inevitably shift to questions about technology.
Sometimes an aesthetic is demanded that requires experimentation on the part of the designer. A designer has to be knowledgable enough about their skill set to know how to explain the cost involved in achieving that aesthetic. And if that cost is reliability than it is incumbent on the designer to iterate that cost and explain how the production will hinge on the unpredictability of that system.
An unreliable system , however, is frankly rarely good for any production unless unreliability is the theme. If a production requires a particular aesthetic experience that seems to be only achievable with the creation of a new tool, then it must be recognized that that tool and the presence of that tool embody the major dramatic themes of production.
Avant garde theatre is one of the best environments for exploring the aesthetics of system building – but it is also the theatre that has the smallest budgets…
For this particular assignment we were charged with the approach of building everything in the asset itself. That is, building a fixed piece of video artwork that could then be deformed and adjusted with playback software (MadMapper).
AfterEffects as playground
Given the nature of this assignment it made sense that Adobe After Effects would be the tool of choice. AE is an essential tool for any media designer, especially as there are times when pre-rendered media is simply the best place to start. I spent a lot of time thinking about the direction that I wanted to move in terms of overall aesthetic for this particular assignment, and I found that again I was thinking about abstract geometric worlds, and the use of lighting in 3D environments in order to explore some of those ideas. As I’ve been thinking about the production I’m working on the in the fall it’s seemed increasingly important to take advantage of open ended assignments in order to explore the ideas and directions that feel right for that show. I’m really beginning to see that the cornerstone of successful independent learning comes from deliberate planning – what can I explore now that will help me on the next project? To that end, what kind of work do I want to be making in two years, and how can I set myself to be successful? Planning and scheduling may be one of the most under-stressed skills of any professional, and I certainly think that applies to artists.
In thinking about abstract animation with After Effects I knew that I wanted to explore four different visual worlds: flat abstract art, lines and movement, 3D lighting and the illusion of perspective, and glitch art. Each of these has it’s own appeal, and each also has a place in the work that I’m thinking about for the fall.
Worlds Apart
Flat and abstract
In thinking about making flat abstract art I started, as I typically do, by doing lots of visual research. One of the more interesting effects that I stumbled on was achieved by using AE’s radio waves plug-in in conjunction with a keyframed mask. YouTube user MotionDesignCommun has a great tutorial about how to achieve these particular visual effect. Overall the media takes on a smoke-like morphing kind of look that’s both captivating and graceful. A quick warning about this effect. This is definitely a render hog. The 30 seconds of this effect used for this project took nearly 7 hours to render. As a disclaimer I did render the video at a projector native 1920 x 1200. All of that aside, this effect was a great way to explore using morphing masks in a way that felt far more organic that I would have originally thought about.
Lines and Flat Movement
I also wanted to play with the traditional big-building projection mapping effect of drawn-in lines and moving shapes. In hindsight I think the effect of the lines drawing in took too long, and ultimately lost some of the visual impact that I was looking for. I also explored playing with transforming shapes here, and that actually was far more interesting and happened too quickly. My approach for this effect was largely centered around the use of masks in AE. Masks, layers, and encapsulated effects were really what drove this particular exploration. Ultimately I think spending more time to write an expression to generate the kind of look that I’m after would be a better use of my time. If I were to go back I think I could successfully craft the write formula to make the process of creating this animation easier, but it really took the effort of creating the first round of animation to help me find that place. One of the hard, but important lessons, that I’ve learned from programming is that sometimes you simply have to do something the hard way / long way a couple of times so that you really understand the process and procedural steps. Once you have a solid idea of what you’re trying to make, it becomes much easier to write the expression as an easier way to achieve the effect you’re after.
3D Lighting and the illusion of Perspective
Another projection designer’s magic trick that I wanted to play with was the idea of creating perspective with digital lighting of a 3D environment. By replicating the geometry that you’re projecting onto, you the designer can create interesting illusions that seem impossible. In my case I started in After Effects by positing planes in 3D space at steep angles, and then masking them so that they appeared to mimic the geometry of the actual building. To make this easier for myself, I worked in a single column at a time. I then pre-composed individual columns so that they could easily be duplicated. The duplicated columns only needed small changes rather than requiring me to build each of the 86 triangles from scratch.
Glitch Art a la After Effects
In another course I took this semester a group of students focused on using glitch art as a base for some of their artistic exploration. While they specifically focused on the alteration of i-frames and p-frames, I wanted to look at the kind of effect that can be purposefully created in AE, and then later modified. YouTube user VinhSon Nguyen has a great tutorial on creating a simple glitch effect in AE. More interesting than the act of making his glitch effect, is his approach. Rather than just adding in adjustment layers and making direct changes. Nguyen instead focuses on how to connect the attributes of a your effect to null objects, essentially making an effect that can be universally applied to any kind of artwork that you want to drop into your comp. This approach to working in AE was interesting as it seemed to start from the assumption that the effect one is making is something that should be easily applied to other works.
Put it all Together
With each section as its own comp the last step in the process was to create a final comp that transitioned between the different worlds, applied some global lighting looks and effects, and added a mask to prevent unwanted projector spill. This was also a great place to do some fine tuning, and to see how the different comps transitioned into one another.
In the ever growing list of tools that I’m experimenting with Derivative’s TouchDesigner is a tool that time and again keeps coming up as something that’s worth learning, experimenting with, and developing competencies around it’s work flow. TD is a nodal environmental called a network. Inside of the network nodes can be directly connected by by exporting parameters.
Nodes, also called Ops (Operations) are split into families specific to the characteristics of their behavior: CHOPS (Channel Operators), TOPS (Texture Operators), SOPS (Surface Operators), MATS (Materials), and DATS (Data Operators). Nodes from within the same families can pass data directly to one another through patch cords (similar to MaxMSP and Isadora). The output of nearly every node can be passed into other nodes by exporting parameter values. This process divides the process of passing data values into two distinct processes, one that’s centered around like to like processes and one that’s about moving from like to different.
TouchDesigner’s nodes are the most powerful when they’re connected. Like Max single nodes do little by themselves, and are the most powerful when they’re connected. Also like Max the flexibility of TD is it’s ability to build nearly anything, and with that comes the fact that little is already built. Similar to Isadora is the native ability to build user interfaces as a part of the very fabric of building a program / user experience.
One of the projects that I’m working on this semester is for a sculpture course. This course, called New Systems, is intended to address the link between media and sculpture. One of the areas that I’m interested in exploring is about collecting data from a circus apparatus and using that to drive a visualization in performance. I’m most interested in the direct link between how an apparatus is behaving and how that data can be interpreted in other ways. To that end this semester I set to the work of building an apparatus and determining how to parse that data. In my case I decided to use this opportunity to experiment with TouchDesigner as a means of driving the media. While I was successful in welding together a square from stainless steel, after some consultation from my peers in my sculpture course it was determined that this structure was probably not safe to perform on. Originally I had planned to use a contact mic to capture some data from my interaction with the apparatus, and after a little bit of thinking and consultation with my adviser (Jake Pinholster) I decided that gyroscope data might be more useful.
My current plan is to move away from this being a performance apparatus and instead think of it as installed sculptural piece that serves as a projection surface. For data I’ll be using an iPod Touch running Hexler’s touch OSC. Touch OSC passes data using UDP packets to communicate over a wired or wireless network using Open Sound Control (OSC). One of the many things that Touch OSC can do is pass the accelerometer data from an iOS device out to other applications. In my case Touch OSC is passing this information to TouchDesigner. TD is then used to pull this information and drive some of the media.
One of the challenges that my adviser posed in this process was to create three scenes that the media moved through. For the sake of experimentation I applied this challenge to the idea of working with containers in TouchDesigner. Containers are a method of encapsulation in TD, they’re a generic kind of object that can hold just about any kind of system. In my case I have three containers that are equivalent to different scenes. The time line moves the viewer through the different containers by cross-fading between them. Each container holds it’s own 3D environment that’s rendered in real time and linked to the live OSC inputs coming from an iPod touch.
The best way to detail the process of programming this installation is to divide it up into the component pieces that make it all work. The structure of this network is defined by three hierarchical levels: The Container, Control, and Final Output; the individual composited scene, the underlying geometry.
Want to work your way through these ideas one chunk at a time? Visit their individual Posts here:
Want to work you way through the whole process, keep on scrolling.
The Underlying Geometry
One of the benefits of working with TouchDesigner is the ability to work in 3D. 3D objects are in the family of operators called SOPs – Surface Operators. One of the aesthetic directions that I wanted to explore was the feeling of looking into a long box. The world inside of this box would be characterized by examining artifacts as either particles or waves with a vaguely dual-slit kind of suggestion. With that as a starting point I headed into making the container for these worlds of particles and waves.
Before making any 3D content it’s important to know how TouchDesigner processes these objects in order to display them. On their own, Surface Operators can’t be displayed as a rendered texture. In TouchDesigner’s idiom textures are two-dimension surfaces, and it follows that the objects that live in that category are called TOPs, Texture Operators. Operators from different families can’t be directly connected with patch chords. In order to pass the information from a SOP to a TOP one must use a TOP called a Render. The Render TOP must be connected to three COMPs (Compositions) in order to create an image that can be displayed. The render TOP requires a Geometry COMP (something to be rendered), a Light COMP (something to illuminate the scene), and a Camera COMP (the perspective from which the object is to rendered). In this respect TD pulls from conventions familiar to anyone who has worked with Adobe’s After Effects.
Knowing the component pieces required in order to successfully render a 3D object it’s easier to understand how I started to create the underlying geometry. The Geometry COMP is essentially a container object (with some special attributes) that holds the SOPs responsible for passing a surface to the Render TOP. The default Geometry COMP contains a torus as a geometry.
We can learn a little about how the COMP is working by taking a look inside of the Geometry object.
Here the things to pay close attention to are the two flags on the torus object. You’ll notice in the bottom right corner there is a purple and a blue circle that are illuminated. The purple circle is a “Render Flag” and tells TouchDesigner to render the object, and the blue circle is a “Display Flag” which tells TouchDesigner that this is the object that should be displayed in the Geometry COMP.
Let’s take a look at the network that I created.
Now let’s dissect how my geometry network is actually working. At first glance we can see that multiple objects are being combined into a single piece of geometry that’s ultimately being passed out of this Geometry COMP.
If we look closer we’ll see that here that the SOP network looks like this:
Grid – Noise – Transform – Alpha Noise (here the bypass flag is turned on)
Grid creates a plane that’s created out of polygons. This is different from a rectangle that’s only composed four points. In order to create a surface that can deform I needed a SOP points in the middle of it. The grid is attached to a Noise SOP that’s animating the surface. Noise is attached to a transform SOP that allows me to change the position of this individual plane. The last stop in this chain is another Noise SOP. Originally I was experimenting with varying the transparency of the surface. Ultimately, I decided to move away from this look. Rather than cutting this out of the chain, I simply turned on the Bypass Flag which turns off this single SOP. This whole chain is repeated eight times (for a total of eight grids).
These Nine planes are then connected so that the rest of the network looks like this:
Merge – Transform – Facet – Texture – Null – Out
Merge takes all of the inputs and puts them together into a single piece of geometry. Transform allows me to move object as a whole in space. Facet is a handy operator that allows you to compute the normals’ of a geometry, which is useful for creating some more dynamic shading. Texture was useful for another direction that I was exploring, ultimately ended up turning on the bypass flag for this SOP. A null, like in other environments, is really just a place holder kind of object. In the idiomatic structure of TouchDesigner, the Null is operationally an object that one places at the end of operation string. This is considered a best practice for a number of reasons. High on the list of reasons to end a string in a Null is because this allows easy access for making changes to a string. TouchDesigner allows the programmer to insert operations between objects. By always ending a string in a Null it becomes very easy to make changes to the stream without having to worry about re-exporting parameters. Finally all of this ends in an Out. While the Out isn’t necessary for this string, at one point I wasn’t sure if I was going to pass this geometry into another component. Ending in the Out ensured that I would have that flexibility if I needed it.
The Individual Composited Scene
There are always large questions to answer when thinking about creating an interactive work: Who is it for? What does it look likes? What are you trying to communicate? How much instruction do you provide, how little instruction do you provide? And on and on. As I started to think about how this piece was going to work as an installation rather than as a performance apparatus, I started by thinking about what kind of data I could use to drive the visual elements of this work. One of the sensors that I knew I could easily incorporate into my current sculptural configuration was a an iPod Touch. The Touch has an on-board gyroscope and accelerometer. After a conversation with my adviser (Jake Pinholster) we decided that this would be a direction of exploration worth pulling apart, and from there I went back to TouchDesigner to start thinking about how I wanted to incorporate live data into the piece I was making.
When dealing with a challenge like building an interactive sculptural system that has at least three different visualizations, it can be challenging to think about where to start. Different programmers are bound to have different approaches to addressing this question. My approach was to start by thinking about what kind of input data I had to work with. Because I was dealing with a sensor that relayed spatial information, this also help me think about how to represent that data. Next I thought about what different kinds of ways I wanted to present this information, and finally I addressed how to playback this experience for users. Some of my more esoteric and existential questions (why am I making this? what does it mean? what does it represent?) were addressed through the methodical programming process, and others were sussed out over contemplative cups of coffee. As much as I wish that these projects could have a straight line of execution, a checklist even, I’m discovering more and more that the act of creating and programming is often a winding path with happy (and unhappy) discoveries along the way.
My first step on this journey, however, was to address what kind of inputs I had to use. Hexler has an excellent app for sending UDP messages over wireless connections called Touch OSC. OSC, or Open Sound Control, is a communications protocol that uses UDP messages to send data over wired and wireless networks. It’s functionally similar to MIDI and has some additional flexibilities and constraints. In the case of touch OSC, one of the parameters that you can enable from your iOS device to send xyz data from the accelerometer. Getting Touch OSC up and running does require a few steps to get the ball rolling. First both the computer that’s receiving and the device that’s broadcasting need to be on the same network. Your broadcasting device will need the IP address of the receiving computer, and a specified port to send the data to (how to find your IP address on a Mac, and on a PC). Once this information is set on your broadcasting device, it’s time to add a Channel Operator to your TouchDesigner network.
In TouchDesigner, there is a CHOP called “OSC in.” This CHOP will allow you to receive OSC data over a wireless network. Once you’ve added the CHOP to your TD network you’ll have to specify the port that Touch OSC is broadcasting to, and then you should be in business. In my case once this was set up I could instantly see a stream of accelerometer data coming from my iPod Touch. In order to use these values, however, I needed to take some additional steps. The raw OSC data from Touch OSC comes in as a range of data from -1 to 1. Additionally, the data comes in from one CHOP. My flow of operators looks like:
OSC In – Select – Lag – Math – Null
OSC In is the data input. The CHOP Select allows you to select a single channel out of a bundle of channels. In this case I used this to separate my X, Y, and Z inputs into different streams. The Lag chop helps to smooth out the attack and decay rates of input data. In my case this ensured that my final values used to control another object where kept from being too jittery. The Math CHOP is tremendously powerful, in my case I wanted to be able to map the values of my raw data [ -1 to 1 ], to a larger range of values, say 0 to 200. Finally I ended my string in a Null. A null in this case is very useful in case I need to add any other operators into my string.
Before thinking about how to use these values, it’s important to take a moment to revisit how geometry is rendered in TouchDesigner. The geometry COMPs that are used to create the objects to be displayed can’t be visualized without using a render TOP. The render TOP requires three components in order to generate an image that can be seen. Render requires a source geometry, light, and camera. The Geometry COMP provides the location of surfaces, the light provides the necessary information about how the object is being lit. The camera COMP controls the perspective that the object is being rendered from. This is similar to an approach that one might use when creating 3D content in After Effects – an object to be rendered, a light so the object can be seen, and a camera to control the perspective the audience sees of the object. Because we need to think of rendering by combining multiple COMPs, that can inform how we use live data.
With some scaled values process and ready to export I was ready to think about how these values could influence the viewers perspective of the geometry. One of my initial thoughts was to render a cube that a user could look inside of. As the observer changed the orientation of the sensor, the virtual environment would also change in kind. While it’s possible to do this by rotating and translating the geometry itself, I instead decided to focus on the orientation of the camera instead. This has a few advantages. One important advantage is the ability to tell a camera to look directly at a specified geometry. This means that in translating the camera (left or right, up or down, in or out) the camera stays focused on the center of the target geometry. This makes changing perspective much simpler.
Initially I was thinking of rendering the entire 3D scene as a single geometry. In doing this, however, I was experiencing some challenges when thinking about the placement of lights and the overall organization of the geometry, and in applying texture to the surfaces. By using a Phong shader one can apply texture maps to the 3D geometry COMPs that have been created. By separating the interior and exterior pieces of the geometry and then compositing them after rendering I was able to apply different shaders to each geometry.
The portion of my network responsible for compositing the geometry looks like this:
Render 1, Render 2, and the Constant are the three source surfaces. Render 1 is the box, Render 2 is the merged set of waves, and the Constant is a black background. Another approach to this would be to set one of the camera background’s as black. These three flow into a Composit COMP. Next is the Transition COMP (this allowed for some small adjustments that needed to be made in order to help align the projection with the sculpture. Originally I made this string with a Null as the final output of this Component. I would eventually find that I needed an Out to pass this scene into another display module.
I used the same techniques as above for the other two scenes – starting with establishing my data stream, generating the geometry, rendering out layers to be composited and then passed out to the visual stream.
Are these pictures too small? You can see higher quality versions by looking at this Flickr Gallery: Graduate School Documentation
The Container, Control, and Final Output
In thinking about how to meet the objectives that I had for this piece, one of my central questions was how to make sure that I could move through three cued scenes – either with manual or automatic triggers. I knew that I had three different aesthetic environments that I wanted to move through. I explored several different options, and the one that ultimately made sense to me given my current level of proficiency (at this point I had only been programming in Touch for a total of three weeks) in TouchDesigner was to use a cross fading approach. Here’s what the whole network looks like:
In thinking about how to ensure that I was being efficient I decided to encapsulate my three different scenes in their own respective containers. You’ll notice on the left hand side that there are three containers – each holding it’s own 3D environment. These are joined through Corss fading TOPs though a final composite (for a mask) until ending in a Null that was used as the display canvas.
I spent a lot of time thinking about how this piece was going to be both interactive and autonomous. It needed to be interactive in that the user was able to see how their interaction with an object was driving the visual media; it needed to be autonomous in its ability to transition between scenes and then loop back to the beginning of the network. I don’t think I’ve totally cracked the nut that is the right balance of interactivity and self-directed programming, but it feels like I did make strides towards addressing this question. My solution to these issues was to allow the interaction with the projection to be centered around the control of perspective, but to drive the transitions through the scenes with time-line triggers.
Unlike some other interactive programming environments, TouchDesigner has a timeline built into the fabric of the control system. The Timeline is based in frames, and the programmer can specify the number of frames per second as well as the total number of frames for a given project. My Timeline triggering system was the following string of CHOPs:
Timeline – Trigger – Null
Timeline reports out the current frame number. The trigger CHOP can be set to trigger at a given threshold (or in my case a frame number). This in turn is passed to a null and exported to a Corssfade TOP as a rate for crossfade. The Crossfades are daisy-chained together before finally being attached to the null that’s output to the projector.
With the system working I also needed to make a mask for the final projection to ensure that I wasn’t displaying any empty grid onto the floor of the gallery where this was being installed. I would typically make a mask for something like this in Photoshop, but decided to try making this all in the TouchDesigner programming environment. My TOP operator string for this looked like:
Contsant – Transform – Blur – Composite
I started by creating a black constant that’s then passed to a transform so that it can be positioned into place. This is then passed to a blur to soften the edges, and finally to a composite to create a mask that contains a left, right, top, and bottom side. In hindsight I realize that I could use a single constant passed to four transform TOPs, to be a little more tidy. The mask as a composited object is then composited with final render stream before being passed to the Null that’s connected to the projector.
In the end I’m fairly happy with this project. t’s been a steep learning curve, but well worth the hassle, angst, and late nights. It’s no small thing to have made a piece of interactive media driven sculpture in a programming environment where that’s totally new to me. For as hard as all of this work has proven to be, I have to remind myself that I’m actively doing the work that I came to Graduate School to do. Everyday I realize that I’ve been changed by my time in the desert, and by my time with the gifted and brilliant artists and friends that I’ve found here.
Are these pictures too small? You can see larger versions of them here:
One of the courses I’m talking this semester is a Media Design course. ASU structures it’s courses into three classifications, A, B, and C Sessions. A Sessions course run during the frist half of the semester (the first quarter), B Session courses run the second half the of the semester (the second quarter), and C Session courses run the full length of the semester. The course is a B session course, and is just getting ramped up. The first project is structured around the need that designers frequently face in building assets that are in specific to known period of time. Copy art is one of the many skills that a good media designer needs tucked up his/her sleeve, and this assignment makes a strong case for learning that process. The project directions and results can be found below: Project Directions: For this project, please download the zipped folder of 4 images from the BB Assignmentssection. In this folder, you will find two images of daguerrotypes and two images shotwith Kodak Ektachrome film. Please follow the steps below to complete the project.
Examine the provided images closely. Research additional images that are also created in this format. Try to identify what features are inartistic to the image. How do these processes effect what subjects can / should be captured with this media?
Identify another medium that you will reproduce. Kodachrome? PixelVision? Silent movie stock? Repeat the above process for this additional medium.
Keeping in mind what you’ve learned about the interactions between subject and format, shoot 1 to three photographs that you will transform into faux versions of these three media.
You may neat tot spend some time researching photoshop tutorials online.
In the case of your self-chose third medium, please track your process, introducing why / how you chose this medium, how / where you researched it,why you think it would be useful, and the steps that yo have taken in the transformation (create a mini-tutorial).
