AI algorithm

[yola]

hello

I've made a short film using Artist Studio:

https://www.oneeyedpixel.com/work/mrsd

it got some attention for a digital platform and they are asking questions.

what could I say about a process to sound inteligent?I mean what could I say about the algotihm we are using here? I cn't find any more precise information, exepte the one that it uses Ai - but it's rather vaste as an answer.

Could anyone share wit me more detail on it?

thanks a lot

Yola

JolaKudelaMrsDalloway4.jpg

[Synthetik]

What is Studio Artist?

Studio Artist transcends a lot of existing graphics software product categories. It's a digital paint program. It's an image processing and photo manipulation program. It's a video processing program. It's a paint animation program. It's a procedural art generation program. You can create 2D imagery, but you can also create time based animation or video processing effects. By straddling existing product boundaries it isn't restricted by the artificial marketing categories that have been created by restricting software products to be just a photo manipulation or digital paint or video processing program. Studio Artist is all of these things and more.

Studio Artist is a Graphics Synthesizer. This means it's a tool for synthesizing artistic visual imagery as well as a tool for building an infinite variety of custom visual effects.

Studio Artist takes working metaphors from music synthesis and applies them to digital painting, photo manipulation, and video processing. Studio Artist also incorporates advanced visual modeling based on human perception of imagery in the brain and applies that visual intelligence to control automatic painting as well as image and video processing effects. Studio Artist provides intelligent assistance that helps you create artistic imagery much faster and with more fun than other programs.

You can do everything in Studio Artist manually by hand if you wish. Or you can select a preset, press a button and let Studio Artist do all the work. You can also work interactively while Studio Artist intelligently works behind the scenes to dynamically assist your manual hand work.

What i love about Studio Artist is that it helps you create artistic imagery. So you can quickly generate an infinite variety of different artistic images or processing effects. You're still in the loop as an artist, making decisions about what you like and don't like. But at the same time you don't have to do all the work. Unless you really want to, and even then intelligent assistance is working behind the scenes to make your work more organic, visually complex and compelling.

When i sit down to use other digital painting or visual effects programs i tend to get frustrated very quickly. I also tend to get very little accomplished, even after putting a lot of work into a project. Because nothing happens in those other programs that isn't directly instigated and driven by the user. So creating a single painting or artistic image or visual effect is a very tedious and time consuming task. One that is limited by the physical motor skills you have available to you. One that is limited by your internal energy and concentration to preserver until the job is finished. What should be a fun and creative process quickly becomes a chore. A marathon of internal energy is required to actually create something.

After using one of these other digital art program, returning to Studio Artist is like a breath of fresh air. I can immediately start generating an infinite variety of different artistic images very quickly.

My work could be derived off of source images, source video, or it could be complete visual abstraction generated from no prior imagery. Work derived from source imagery could be realistic, or very abstracted output that is nothing like a conventional clone effect and has no direct relationship to the content of the original source imagery.

I can try out different pre-built intelligent presets to stimulate my visual creativity. When i find something i like i can then edit it to customize or further enhance the visual effect. When i create something new i like i can then save it as a new custom preset for use at a later date. I can do detailed editing, potentially adjusting hundreds of different parameters that affect the look and feel of a digital paint tool. Or i can press a button and have Studio Artist generate new sets of presets automatically for me to cull through looking for keepers.

I can generate sets of procedural abstractions or modular image processing effects at the click of the mouse, working with directed evolution over time to build amazing artistic imagery i could never have created or even conceived of if i had to manually build them from scratch. At the same time i have the ability to get under the hood and fine tune modular image processing or procedural art effects via detailed editing if i wish to refine or fine tune a particular effect or image. With over 600 modular image processing modules that can be combined together in an unlimited number of different ways i have access to an endless supply of different visual effects.

I can generate free running gallery shows that generate new artwork and visual effects unattended. Gallery show can be used to build dynamic customized art displays, visual performance pieces, or as a way to automatically generate new presets effects or sets of art images that can be culled through later for keepers.

I can work with live visual synthesis in real time. Building recursive visual effects that feed back on themselves. Processing live video captures. Changing between different preset effects on the fly without interrupting my visual performance. Live editing effects and watching them adjust in real time. Streaming my live visual performance out to a video file to capture a visual performance.

Studio Artist is all this and more.

[yola]

thank you for your answer.
I'm specialy interested in this aspect:
"advanced visual modeling based on human perception of imagery in the brain and applies that visual intelligence to control automatic painting as well as image and video processing effects."
because the guys who are going to interview me specialise in art driven by AI
could you please tell me more about how AI is involved in it.
What exacly does it mean?
what's the process?
As Iunderstand it - the algorithm picks luminance values in the video image and applies a graphic vector to it. is that right? now - how could I, as a user, modify this vector and control it further ?

"modeling based on human perception in the brain" - could you elaborate please ?

[Synthetik]

Studio Artist is built on underlying visual modeling based on studies of the human (and primate) visual system up to the IT cortex area in the modular visual system of the brain. Studio Artist looks at a source image like an artist would look at a model, and then tries to paint or draw it based on those underlying visual attributes. 

Studio Artist also tries to take work metaphors from music synthesis and apply them to computer graphics.  The system is designed to try and be collaborative in nature, computer and artist working together to create a piece of art. So human visual modeling and visual attribute modulation is incorporated within the overall graphics synthesis system.

Studio Artist is not a convolutional neural net. Studio Artist is not an implementation of Gatys, et al neural style transformation.  However, the author of Studio Artist apparently did write the first paper on image style transformation based on a convolutional neural network in the early 90s.

[yola]

would you know where to find the paper that he wrote?

[Synthetik]

You should talk to Siggraph Lance Williams if you are really interested in tracking down a very old and obscure publication.  He's the one who brought up this angle on that very old research done at Apple Computer ATG in the early 90s.  It doesn't really have anything to do with Studio Artist, it's just an interesting curiosity.

There is also another slightly less obscure paper in IST Electronic Imaging Proceedings in the Human Vision conference at some point that discusses the underlying neural modeling used in Studio Artist at the time that paper was published.

If you are unfamiliar with the human visual cortex, here's a somewhat gentle introduction to it.

So the visual cortex is modular, and stacked in hierarchy of modules.  So V1, V2, V3, V4, IT cortex. If you look at what is going on in these hierarchical representations in the cortex, and you look at convolutional neural net visualization studies (like done by Rob Fergus and collaborators), you can see some loose correlations.  They are both computing feature space representations based off of the input image.   That get more elaborate as you move up the processing hierarchy.

Approaches like the 'Gatys, et al neural style' work, use the feature space representation of a neural net trained for object recognition in images as a basis set for generating texture, and then later splitting the feature space representation into texture and style for neural style transfer.  They use a gradient based procedure to take random noise and optimize it into an image that corresponds to activating a particular feature space vector representation.

Studio Artist is different in that it uses it's internal human visual modeling visual attribute representation to modulate dynamic graphics and paint processes. So Studio Artist looks at an image (or a frame of video), analyzes it into visual attributes, and then uses those visual attributes to influence and control an active painting process.

But all of this lives inside of a much larger larger working framework that includes the human artist in the loop.  Built on working metaphors from music synthesis hardware and software.  That was designed to try and stimulate and encourage creativity and exploration.  That can create diversity and variation.  That also includes a lot of additional internal heuristic intelligence derived from how artists work.

You need to look at and understand the entire system to really get what is going inside of Studio Artist.

[Synthetik]

Cool film.  Thanks for letting us know about it.

I love how the particular Studio Artist effect you are working with fits and enhances the mood of the piece.

[erik goetze]

Having just watched a lot of shorts in several film festivals recently, I was hungry for more--thanks for sharing your work. I enjoyed it, the animation technique seemed well suited for conveying memories.

[yola]

thank you Erik!