Interactive Action Painting

[johndalton]

A while back there were a few forum posts that discussed people's desire to bring back the global evolution paint synthesizer features in version 4. And in some recent forum discussions there's been some posts talking about interactive gestural interfaces for audio synthesis. Both of these discussions inspired me to get to try out a few ideas. What's falling out of these experiments is that there's really a whole new way to work with Studio Artist for painting that i had not thought all that much about before, which i'll call interactive action painting for lack of a better descriptive term.

So traditionally, you could setup a paint preset in Studio Artist and then press the action button and watch it paint. Or you could build something like an auto-draw interactive or particle pen mode paint preset and it would auto-draw as you controlled it in real time with a mouse or wacom pen in the main draw canvas. Synthetik has traditionally called this second approach intelligent assisted drawing.

Studio Artist 4 added a new approach to action painting, which is the live manipulation of individual paint parameters in real time as action painting is taking place. So you can press the action button and then start adjusting different paint synthesizer settings in the Editor window and watch the auto-painting change appropriately in real time as you make those adjustments. You can view this live interaction as a great way to edit paint presets since you can see the results of your editing changes in real time. But you can also view it as a different way to approach the overall process of building up a final painting. And notice that the live user interaction is taking place in a secondary interface area (as opposed to taking place directly in the main draw canvas).

Morphing between different paint presets in real time is an extension of this notion of live editing. And again, you can view it in different ways. It can just be a great way to interactively edit a preset while it's drawing as an aid to getting the look and feel of it to be what you want. Or, you can view it as a new feature for interactively building up a painting in real time. You're using gestural interaction in a window separate from the main draw canvas (a preset morphing window in this case) to influence auto drawing in the main canvas. So rather than your pen movements defining the actual paint strokes, the pen movements can be thought of as gestures in a separate control surface area that influence the appearance and behavior of the auto painting taking place in the main draw canvas.

So as i work to finish up these new features i thought i'd start up a forum discussion about this whole notion of using gestural interaction in a separate control surface area to control auto painting in the main draw canvas. So feel free to make your suggestions about how you would like to see these new features develop. Or if you do't understand what i'm talking about i can try and explain more.

I think that this whole notion of gestural control of automatic painting can be taken much further than my initial thinking. So as i finish up the live preset morphing features we can perhaps work out in this forum discussion what some other interesting gestural interactions might be.

What's great about digital art tools is that we don't have to be restricted to what has come before in traditional art media. And this new way of working with Studio Artist that i'm describing here is a great example of that.

[liveart]

Ok John already knows that I am very interested in the possibilities that could be opened here.

on my wish list:

Gestural control over some of the dynamic elements of the painting process, for example being able to direct multiple paint paths independently, or effecting interactive warps, adjustments or imops while painting with the pen and of course control of the color wheel.

But there could also be a way to map preset modulations to a touch control device or gesture recognition device, so for example a paint preset might be modified from a path shape of line to live extension, or from interactive multipen to particle paint. It is true that in fact one can do this already the difference is what could be done in live performance. real-time switches now take too long because of the menu system. I can not imagine programming a button for every option, but giving the user the choice of mapping a finite number 10? maybe, could make a huge difference.

Anything that moves beyond menus to bring control of the program's amazing abilities to the direct input of the user. The current favs strip is a great step, but maybe certain favorite presets or paseqs could be mapped to a gesture?

Another area to consider is the possibility of speed control, specifically slowing down some of the paints and paseqs via a interactive control would be awesome..


as a postscript

morphing between paint presets while loop running or even just while the action is running via an ordinary action cmd is brilliant, but does create instances that can not be repeated as certain aspects of the running preset persist for moment longer than others so for a period of time the action is a hybrid of which the settings are not captured, even if history recording is on. I have spent many an hour trying to recreate the look of some of these hybrids, I am usually not successful. I guess the moral is either live in the moment and enjoy what you see as a uniquely personal performance of SA for you, or always stream video while you do live morphs, the choice might depend on your HD space as much as your philosophy.

I would not mind this in a live performance for an audience, I usually only regret that no one else has or ever will see the effect.

[Thorrific]

wow you are blowing my mind
maybe some kind of shape recognition that you could also assign attributes to like if you make a certain simple shape with your brush or mouse (like a V or an O) then rotate the paint strokes around or expand in size or something? the possibilities are endless as if they weren't already!

[liveart]

yes simple gesture but not with the pen, keep the pen for drawing.. let the gestures be read separately. Scaling and rotating are good and obvious choices,

[Thorrific]

maybe the wacom mouse could be used in the gesture window and the pen used in the canvas window

[johndalton]

Wacom doesn't support 2 simultaneous tools like they used to in previous version of Intuos tablets. So we can't support the second device channel modulation like we used to.

[Thorrific]

I saw a new little wacom tablet at the store the other day (mine is from 1999!) and it has some kind of multi touch control functionality. Maybe touch can be used for the gestures? I would totally get one for that

http://www.wacom.com/bamboo/bamboo_pen_touch.php

[johndalton]

I don't think they pass anything back to the application from the multi-touch gestures. So the gestures would kind of be like a way to press a key or run a command menu by doing a gesture. If i'm wrong feel free to correct me, i haven't played with a bamboo tablet with multi-touch yet.

Intuos 4 has a 4 channel expression wheel built into the tablet. We'll probably support that at some point for parameter modulation.

[Thorrific]

you are probably right, the link doesn't really seem to imply that the touch feature does anything other than menu stuff.

[johndalton]

I've thought about picking up a windows 7 HP tablet computer with multi-touch to proto-type some multi-touch ideas for Studio Artist.

[Florence Artur]

Not sure this belongs to the topic, but there are two things I would love to have.

First, as I already mentioned elsewhere, the ability to have several actions running at the same time: for instance, start auto painting with the action button, then while this action is still running, paint on the canvas. Or run a paseq and paint at the same time.

Another thing I would like is to save a stroke, something like a paseq but it would be saved as a patch, so that I can repeat this stroke wherever I click on the canvas. Maybe something like that already exists though...

Florence

[johndalton]

Using dual mode paint is the way to do the first thing (ie: paint while another action is running at the same time).

You can also do the second thing now as well. Although we could probably make it a little easier to access. You can load a set of bezier paths into the internal bezier path memory in the pant synthesizer and then have those paths painted where you click on the canvas. So you draw the first path with the pen draw path record option flag turned on. then you'd record that generated bezier path into the internal bezier path memory. then you'd switch to the preset that draws with the internal bezier path memory. and if you export this preset it will also export the internal bezier path memory contents so when you load it again those path(s) will be there.

[Florence Artur]

I don't feel comfortable with the dual paint mode because it involves modes I never use, and it doesn't do what I want to do anyway: run two or more independent actions at the same time.

I use almost exclusively paint patches. MSG is beyond me, and I'm not very interested in image operations. So I have two modes of operations at my disposal: if I use auto paint, the whole canvas is painted in a short time. The action is all over the place but gets boring quickly. If I paint manually, things can be more interesting but the action is mostly limited to the part of the canvas where I am painting, although some patches allow me to have a wider range of action.

I would like to be able to start auto paint, then paint manually while auto paint is running. Or run a paseq (provided it doesn't run too fast - remember that discussion?) that paints on one part of the canvas, and paint at the same time so that I can have two things happening at the same time.

This is what I really want. The other thing can probably be done, as you said (I remember doing some tests but I forgot the details), but it's pointless anyway, unless I can run two actions at the same time.