Art Processing Sharing Blog
This video is an example of a latent diffusion generative ai drift session. The theme is 'the End of Time - a children's story'.
The imagery was generated using a LAION-400M latent diffusion multi-modal generative ai text to image synthesis model. The animation was put together in Studio Artist using Transition Contexts and Paint Action Sequence (PASeq) processing. You see the imagery presented in the order it was created in the live drift session. Drifting is an art process that we originated working with live mutation of procedural MSG (modular synthesized graphics) generative effects in Studio Artist. Here we are extending the live drift concept by working with generative ai image synthesis algorithms. The particular generative synthesis algorithm we are using here is a multi-modal latent diffusion algorithm, but you could just as easily be using a VQGAN or other generative image synthesis techniques.
Drifting is an example of a human artist and a generative ai system working together to create something over time. You can think of drifting as an interactive exploration of the adjustable parameter space of a generative algorithm or art creation system. The idea is to slowly change or adjust the parameter space of the system by some small amount, observe the results, and then continue that cycle. The changes for this particular drift being additional textural input added to or subtracted from the multi-modal text to image generative algorithm, as well as individual adjustments of parameters associated with the image synthesis model. Over time, you are slowly drifting through the generative algorithm's space of creative possibilities.
All of the imagery in this was automatically generated by Gallery Show. It is all derived from MSG presets that are being live evolved and mutated in the Evolution Editor as gallery show is running. I'm using the grab button in the Gallery Show Toolbar to grab output from gallery show that i like and save it to a folder.
Once the gallery show run is finished, then i keyframe the saved gallery show image grabs in the folder, which you can do with a s ingle menu command. You can then run a second menu command to time expand the timeline so there is space between the Transition Context keyframes.
You can then add additional action steps to your PASeq if you want to add additional adaptive image compression, sharpening, etc effects to polish up the transitioning grab image keyframes.
There are a lot of different approaches to working with restrictive sets of MSG presets. The default settings for MSG techniques in gallery show default to using the entire factory MSG preset set. This might be good if you want a lot of diversity in the output, or you are just using gallery show as a mechanism to explore what MSG can do (the potential MSG visual appearance space is super huge).
Under the Action : Gallery Show : Assets menus you can retarget the MSG cache location if you would prefer to restrict where gallery show is looking for MSG presets. So you can point it at a custom folder, or collection, or category, whatever you want.
You can also setup custom Favorites or Process folders you put together that contain specific MSG presets you want to work with.
Here, experimenting with Simple Texture & MSG.
A quick experiment: autopaint, displacement, Mass Abstractor im op.
music (w/o permission): Sia
This animation is part of my ongoing gallery series. It was created from three original paintings of Sweet Peas made with StudioArtist.
I started this new thing where at the end of every afternoon i take a folder of Studio Artist gallery show grab images i generated over the course of the day (usually during testing) and turn it into a transition movie using a transition context in Studio Artist. This is from earlier in the week and is composed almost entirely of images generated using the various 'line screen' effects in the image operations section of Studio Artist. All of the images were generated automatically using gallery show generative ai that automatically builds the effects.
Low frame rate, sketch edge ImOp, loose render, blur, geodesic growth. Video source?: a screen capture while playing around in Drop Dots app.
Experimenting with time particles & displacement w radio telescope audio
All of the visual effects in this were created by Studio Artist gallery show generative ai (no pre-built presets involved).
The visual effects are Ip Op - Vectoizer hybrids. So they are all vector based, and could be scaled to higher resolutions for printing if one so desired.
I used the Grab button in the Gallery Show Toolbar to grab specific images gallery show was generating that i liked. Those get stored in a folder.
I then used a Transition Contet to create thethe transitioning animation. I'm using the Flow algorithm option in the Transition Context.
My attempt at an egg tempera paint look generated in Studio Artist V5.5. Seems to work well for both still images as well as video processing. Like all good ideas, i was trying to do something else when i stumbled upon it yesterday. So i went with the flow rather than being pissed off that my original idea was not quite panning out as i had hoped.
I built this with 2 different layers, but there is no stacked layer compositing involved. I have a recursive processing loop going on in each layer. It's a very small amount of recursive mixing of the previous output of the layer with the current output of the layer. Enough to subtly blend, but not so much that you see ghosting.
I used the Sketch Mass IpOp in the first layer to build the sketch hatching. Along with the Threshold IpOp to punch through a little bit of the low end of the mass rendition.
I used the output of the first layer for the luminance of the Colorize IpOp effect, colorized by the source. This is happening in the second layer. Smart Blu IpOp is used in both layers. Several passes of MultiOctave Gradient Lighting IpOp with different settings is used in the second layer, along with some low Mix Smart Contrast, to build the shading.
Hoping John will weigh in with some good suggestions on this on this question!
The pieces I've been working on lately have become more complex and my need to generate and retain selections, to use as stencils or masks for painting or applying effects, warps and operations, has become more critical.
The new GenStrategy option in v5.5 is fantastic (thank you!) and I find myself generating all kinds of new selections to use when working. I extensively use the "traditional" or classic options built-in to SA as well in conjunction with the new selection options.
I really would like some way to "retain and store" these selections as they are generated so they can be utilized throughout the development and refinement of a piece. Is there some way or place or strategy that I'm missing where I can "park" a selection or selections as they are generated? I am almost always in Paint Synthesizer Classic for the bulk of the work I do. I'll drop into warp, adjust and image ops too. Often I'll rotate the canvas and it is frustrating to have the selection disappear (the selection resets to full off). For example, I'd like to be able to retain a selection(s) and apply it to the piece after it is rotated to a new orientation.
I vaguely recall a discussion from the distant past about a way to put the selection in a layer to use it later. I think it was a kind of work-around or kludge though I could be mis-remembering this. Overall it would be nice to see a more comprehensive approach to selection operations (a stack like the layer stack for instance where you could blend layers, reorder them and so forth) though I doubt it's high on the list of must-haves right now.
Appreciate your ideas/suggestions.
Prototyping a new transition effect in Studio Artist V5.5.5 using Warp Contexts in conjunction with a factory movie processing preset for the stylized paint look of the output. The transition rolls into chaos, then rolls back into form over and over again. The movement could be different each time, i kept it simple for prototyping purposes.
Working with some simple but effective vector paint effects in Studio Artist V5.5 to build a paint animation. I'm using the paint synthesizer's vector paint shadowing options to build something that works visually like a gradient lighting effect, but is all just built using vector paint. You flip the orientation of the vector diffuse shadowing by 180 degrees (some slop in this is good sometimes), and darken the shadow on one side, and then lighten the shadow on the other side (think about what gradient lighting does to the image). The nice thing about building your lighting effects as a direct part of the vector paint is that they then avoid introducing any resonance energy over time that a normal raster gradient lighting effect adds to a recursive paint animation process.
Watercolor paint animation experiment created in Studio Artist V5.5. I've been trying some experiments recently where i just go into the factory manual paint presets, grab one at random that looks interesting, and try working with it in a Paint Action Sequence (PASeq) to build an animation. I'm working with a Transition Context to build a virtual source based on multiple source movies that are transitioning between one another (algorithm 4) as the animation progresses.
A bit of context: I am a composer and visual artist. I am new to StudioArtist. The art I have created so far with StudioArtist has been based on original photography and watercolours. As a composer I often write music first then create visual art from that inspiration, although art has come first from time-to-time. I am looking forward to being able to input audio into the paint synthesizer to create art on the fly. I hope this feature will be able to work with multilple input channels. I work in quadraphonic (a bit of a relic) but also in full surround-sound and Atmos. Comments and suggestions are welcome. In particular best practices to rotoscope video into smooth animation. The video linked to this post (Travellers) was created from a photograph of tulips taken at the Royal Botanical Gardens.
Simple bezier path embedded keyframe paint animation created in Studio Artist V5.5.5. I started with a single vector shadow paint synthesizer preset that painted bezier vector paths generated by an embedded Ip Op effect. I had 4 different source images key-framed into a Transition Context that feeds the source area with its output. They were key-framed 100 frames apart over a 300 frame timeline. I then ran a menu command that embedded the paint synthesizers draw paths directly into a second paint action step where every bezier path was directly stored in the embedded auto paint keyframes (as opposed to the original paint preset that auto-generates the bezier paths using an Ip Op effect every time it is run).
Automatically generated keyframed embedded bezier path vector paint animation. Everything is derived off of vector bezier paths, so you can rescale any frame to any output size you want.
This particular movie is interesting because it is an example of proto-typing a V6 effect using V5.5.5 features. I actually do this a lot, try out a potential new feature for the next version using existing Studio Artist features cobbled together in a Paint Action Sequence (PASeq) to implement what we ultimately end up turning into a single action step new visual effect. In this simple example we basically re-conceptualizing 'old school raster effects' color gradient and color palette mapping effects as integral vector effects.
Once i had a mechanism for generating the bezier paths and painting them using the paint synthesizer, i then used a Transition Context with virtual sub-nest keyframes set to 15 and a Path menu command to auto convert each Transition Context virtual keyframe into an embedded bezier atuo paint action step in a PASeq.
Then i used two Embedded Bezier Frame menu commands to auto-normalize the number of bezier paths in each keyframe and then re-order their index positions to lead to a smoother animation effect.
So 3 menu commands generate the entire keyframe paint animation automatically after you build the initial PASeq for processing a frame and setup the timing of the Transition Context virtual keyframing.
Please, subscribe to get an access.
This video is an example of a latent diffusion generative ai drift session.…
All of the imagery in this was automatically generated by Gallery Show. It is all derived…
Here, experimenting with Simple Texture & MSG.
Please, subscribe to get an access.