I decided to post my question here because some of you may better understand what I am asking about. I have done some searching for photomosaic software but what I found did not allow for much interactivity during creation. I've not made one, ever, but was wondering if there is a brush that allows one to stroke the canvas that then "paints" the canvas with images (or text/words) created from a library (or from a selection of words aka a story or document).
Additionally, I wanted to know if this might be the basis of a photomosaic if one has a "source" image designated as the starting point?
Following are my questions regarding the movie brush after viewing a video tutorial and reading about it:
I am just trying to get my mind around what and how a movie brush creates a Mosaic from images in a folder.
First off, is there any kind of analysis on the images themselves to try to match up or coincide with colors, contrast, features of the original source image you are trying to create a mosaic of?
Next, just what is the sequence or perhaps the thought process behind creating a movie brush? I tried following the "Building Your Own Custom Movie Brush for Photo Mosaic Effects" but got lost.
I understand that you create or have a folder of images to be used as the "paint"? I assume they should be jpgs. After that I get lost. You create a new PAS by using the menu item Action>Process with Paint action sequence> image to movie. Which then leads to selecting the folder and canvas size. Just what does that menu selection do?
After that, I see a list of steps. What does the "SetCanvas-Source image" mean and how does one find out what that means. It seems as though it is kind of command as though it is a software laguage command but I have no idea what or where to find this syntax or command list.
Again, I am just trying to wade through the complexity and richness that must obviously be SA, it is just that I don't know where to start.
Perhaps, I should describe what I would like to do and some of you can direct me as to how I can start to either find out about it or where I can start to read about the process?
I have some horse images, many of them. Among the images there are a few of them that I like. I was thinking that I could choose one of them and create a photomosaic using the folder of other horse images. How would I set out to do something like that?
Also, just how or what program does do the analysis as to which image (or sequence of images) out of many, would be chosen to best reproduce features and colors of the image one is trying to reproduce as a photomosaic? I hope I'm using the terms correctly, I've never tried to make one simply because I've never known where to start. I've only seen completed works of others who have created one image of for example, the portrait of a person and that image upon close inspection, is actually and cleverly made up of much smaller images that were seemingly hand picked to reproduce the eye, or features such as shadows as opposed to a random selection or sequence of images. I realize that what I am asking for requires quite a bit of "intelligence" or image processing, I just don't know if the images are painstakingly assembled one teeny image at a time or whether there is some kind of image processing algorithm that does this analysis and sub-image selection for you.
Thanks and hope my question is clear and understood.
Jorge
Replies
Happy to try to answer your questions. We plan to put together some beginner style video tutorials on this topic, so bear with us while we work on putting those together. Your feedback as a 'beginner' is also good for us to hear, because we all know studio artist so well that some concepts (like a PASeq that sets the canvas to the source image) just seem obvious, and obviously they aren't (as you have just pointed out) if you are just getting started.
...
I want to point you to a particular tutorial post on the user forum that addresses questions regarding creating photo mosaics. The original post is about making alpha masked mosaic images, but the discussion post after that initial beginning is an entry point into a ton of existing online tutorials and tips that can answer many of your questions.
...
So what is a movie brush? the movie is really just a way to provide a container for a collection of images. You can use a movie brush in many different ways (a common theme for every aspect of studio artist). From the perspective of making a photo mosaic image, the movie is what holds the collection of smaller images you are going to use to make a larger image out of.
...
You could use the movie in other ways. It might be different sized brush images, or different orientations of brush images, and you could be using interactive pen pressure or tilt orientation to modulation between the frames of the movie brush as you interactively paint. This introduces a concept you mentioned, how are the frames in the movie brush being accessed during the process of making a photo mosaic image?
The paint synthesizer consists of a number of different control panels. Each control panel controls some particular aspect of the overall paint synthesizer process of generating a painted image in Studio Artist. Some control panels deal with automatic path generation ( for paint presets that auto paint). Others deal with paint color. Others deal with how colored paint is applied to a virtual brush, and then applied to the canvas. The Brush Source control panel deals with how the virtual brush is defined. A computational brush is a procedural mathematical shape (typically round, but could be very different). An image brush is based on a single image (could be black and white or color) A movie brush is based on indexing through the individual frames in a quicktime movie file.
...
If you already have a quicktime movie file, you can just load it as a new movie brush source by using the File : Paint Synthesizer : New Movie Brush menu command. Doing so will load the movie as the movie brush, and the individual controls in the Brush Source control panel will adjust to be appropriate for a movie brush.
So the screen shot above shows the brush Source control panel after i just loaded a movie file called 'haeckle-256' as my movie brush. You can see that the frame size of the movie is 256x256 pixels, and there are 121 frames in the movie file.
MB in the control naming is short for movie brush.
MB Indexing refers to the type of indexing. It can be 1D or 2D (which is shot for 1 dimensional or 2 dimensional). For now, lets just focus on 1 dimensional indexing.
The MB 1D Frame Mod control determines how the frame indexing will be generated as painting progresses. Remember, studio artist works by building individual dabs of paint on a virtual brush, and then applying them along a vector path (which could be manually or automatically generated). So each time a dab of paint is generated, the movie brush is frame index modulated, based on this MB 1D Frame Mod control setting. Right now i have it set to Cycle Fwd. So it is just cycling forward through the frames, 1 at a time. When it gets to the end, it will start at the beginning again.
The MB Attribute Index control specifies what is really being indexed through. This can get a little complicated to wrap your head around, and typically you would just leave it set on Sequential. So then cycling forward through the frames means running through the frames in order as they are stored in the movie file. There are other MB Attribute Index settings that virtually reorder the indexing of the movie frames, so that cycling forward through them would cycle forward based on their individual luminance or orientation characteristics.
MB Load Status refers to whether the entire movie file is loaded into ram (preload), or is accessed from disk (disk draw). Preload is better for really small movie files used for interactive painting (since it gives the fastest drawing speed). Disk Draw is better if you have large frame sizes in the movie brush, or have a lot of frames, because preload uses up a lot of memory in theses kinds of situations. But the drawing is a little slower. For photo mosaic images, you often find Disk Draw to be a better solution.
Ad you can see from the screen shot above, there are a lot of different ways to index through the movie brush frames. One useful option for photo mosaic purposes is the RGB Mapping one. What this does is that every time the movie index gets recomputed at a particular location for painting a dab of paint, studio artist figures out which frame image movie best represents that spatial location in the currently loaded source image.
...
So you asked about how the program analyzes which image in the movie file to paint in a given location of the canvas to build up a photo mosaic of the source image. We have already talked about 2 different ways this could happen. the first was by virtually re-indexing the movie frames based on visual attributes like luminance or orientation. The second is a Frame Modulation option like the RGB Mapping one presented above, based on matching up color.
There is a whole other way to do the 'analysis-mapping'. It is called recolorization. I actually use this a lot. You can use additional image processing algorithms (available in the paint synthesizer) to recolorize a movie frame image on the fly to best represent the sub area of the source image it is trying to draw. Here's a studio artist effects blog post that discusses this in great detail.
What re-colorization gives you is the ability to use any frame indexing option (including sequential cycle forward, or random) and still generate a color accurate photo mosaic. And even if you are using RGB frame indexing, you still might want to turn it on, so that you get better local color reproduction in your photo mosaic.
...
let's back up a step in the discussion, because something you initially alluded to was that you have a folder of jpg images, and you need to first convert them into a movie file to use them as a movie brush. So the tip on "Building Your Own Custom Movie Brush for Photo Mosaic Effects" is trying to run you through that process. This one called Making a Movie Brush to Create a Photo Mosaic might be easier to follow.
What both of them do is run you through the mechanics of using studio artist to generate a movie file as output from a folder of images. Studio Artist is a very flexible movie processing and animation program, in addition to being an amazing 2D digital paint program. So, we're using the more advanced movie processing features to convert a folder of images into a movie.
Studio Artist uses a paint action sequence to do movie processing. PASeq is short for paint action sequence. You can think of it as a script of recordable actions. Anything you do in the program that is an interactive action (like manually painting a paint stroke), or an automatic action (like pressing the action button for auto-painting, or for running an image operation effect) is recorded as a single action step in a PASeq. So you can use the recording properties of paint action sequences (PASeq) to build up all kinds of amazing multi-step effects. that can then be played back and applied to single images, to folders of images, or to movie files.
But all we want to do is convert a folder of images into a movie file. So the PASeq for that is very simple. You just record erasing the canvas to the source image. End of story. This takes the source image and puts it in the canvas. And does nothing else to it at all (other than maybe automatically resizing it if the source image is a different size than the canvas).
So, both of the tips mentioned above on making a custom movie brush run you through the steps i just discussed.
As i mentioned before, we'll try to put together a beginners movie tutorial that runs you through this with very simple explanations. once you understand the process, it's really very simple. the hump you need to get over is understanding how studio artist generates a movie file, using a PAseq script to run a series of action steps frame by frame to process a movie. In this case, the source is a folder of frames (the individual images you want to put into a movie file), the output is a movie file.
...
If you own Quicktime Player 7 Pro, you can convert a folder of images into a movie file using that application as well Using a single open image sequence menu command). We should probably just add that to Studio Artist to make this super easy for beginners like you. I will in fact add that to my 'to do' list.
Using a PASeq is so essential and basic to anyone who uses Studio Artist for processing movie files or for generating animation output, that we build our movie brushes that way. But as you have pointed out, it's a conceptual hump you need to get over when you are first learning the program.
...
I've probably already covered too much in too much detail here. So i'll stop, and then re-read your original questions, and add any additional individual answers. Please feel free to ask questions for clarification about anything.
So yes, you could choose one horse image, and then make a photo mosaic of that horse image out of a folder of other different horse images. Or you could even make the photo mosaic out of the same image (via re-colorization). And it is all done automatically in Studio Artist.
...
I've been starting to use the distinction photo mosaic vs photo collage recently. You can do both automatically using Studio Artist, with extreme artistic flexibility. Photo mosaic would be arrangement of the sub images into regular grids. Photo collage being more irregular shapes based on individual movie frames that work to build an image.
Here's an example below of a sub-nested regularly tiled photo mosaic
Here's an example below of what i would call a photo collage effect
Both were generated from movie brushes in Studio Artist. All of the painting for both was generated automaticlly.
...
Here's a link to a series of effects blog posts on photo mosaic effects. These were generated before i started distinguishing photo collage from photo mosaic, so you will see both effects there. There's also a movie brush tag on that studio artist effects blog, which will bring up some of he same articles as well as others.
...
Here's a series of photo mosaic articles on the Studio Artist news site.
...
Here's a series of photo mosaic art posts on my daily art blog.
You asked about painting with words from a story. yes, you can do that as well in Studio Artist.
Here's a link to a series of effects blog posts on working with building images out of text.
I know i've done images that combined text worked painting into photo mosaics, but i'm having trouble finding an example for you right now. You can definitely combine the 2 techniques together if you want to.
I think you are trying to simulate tsevis' (a grafic artist from grece) type of art. I can tell you that he is not using SA alone for his results he is combining programs. But as john just demonstrated you can still have great results with just SA.
Actually Charis Tsevis does create his mosaics using Studio Artist. He's a master of using movie brushes and paint synthesizer tiling effects. He sometimes builds a lot of custom geometric images in a program like illustrator, and then uses those as sub images in a movie brush. But the photo mosaic tiling is generated in Studio Artist.
...
Generating a straight regular movie brush tiling effect is very straighforward. Generating a sub-nested tile (something Charis is a master at creating) does currently involve some additional work. You need to run multiple passes at 50% reduced tiling sizes in succession to build the sub-nested details.
You can either do this using manual painting with a reduced tiling size to touch up edge detail. Or you can use automatically generated texture derived masks to allow for doing it all automatically in a multi-step paint action sequence. This was covered in a series of Studio Artist effects blog posts.
...
Here's a section from the tutorial post i mentioned in my first post in this discussion that points people to different online articles about photo mosaic effects related to this.
Many people are interested in duplicating the look of the wildly popular Studio Artist photo mosaic work generated by artist Charis Tsevis. To duplicate his style of sub-nested adaptive mosaic blocks, they are interested in working with grid tiling and nested sub blocks within the mosaic grid to generate detail at edge and feature areas of the larger image the adaptive mosaic is trying to reproduce. There are approaches you can use to build sub-nested tiled grids automatically to build detail in a tiled photo mosaic. Charis is a member of the Studio Artist User Forum if you wish to communicate with him directly, or follow the links on his user page to more of his amazing Studio Artist generated artwork.
...
The next version of Studio Artist allows for full nested tiling as a single action button auto-paint step, just using the paint synthesizer and nothing else. We specifically added this feature so that people interested in duplicating Charis's approach could achieve it with a single auto paint step.
But it's easy to take the PASeq we show how to build in the effect posts mentioned above and adapt that to building custom mosaics in his style automatically today. And manual painting in detail at 50% reduced grid sizes is pretty easy.
...
Sub-nested tiles are just one of a multitude of different approaches you can take to build photo mosaic effects. Personally i think people should focus less on trying to exactly duplicate Charis's work, and more on creating their own unique custom signature photo mosaic style or aesthetic. There's so much possibility for customization in Studio Artist, take advantage of it to create your own unique approach to this style of effect.
Wow! John, that was amazing explanation. Thanks for taking the time to think through and explain it. I have not tried any of it nor have I clicked on any of the links but your explanation really helped me to better understand more of what is going on.
Yes, I realize that most of you have become so accustomed to doing things in SA that you don't realize in your tutorials or posts when you've skipped from one topic to a completely different one. Knowing that, I try to be as detailed as I can in my question asking when I don't see a logical connection between steps or screen shots or when a term is thrown in that was not previously defined or introduced, thus throwing newbies off the scent.
I am not clear on the 50% reduced and "nested sub blocks" but I expect that terminology one would naturally become familiar once they jump into it. Right now, I am guessing that by using a reduced grid size, one would be afforded a higher degree of detail while still retaining an orderly grid and I assume that although one may be able to vary the size of the grid by any degree, a reduction by 50, 25 or 12.5% and so on (Sorry, I have forgotten so much math that I can't even remember what 1/2X where X=1,2,3...would be. Don't judge me!), would yield the neatest "fit".
Ok, I will try to follow the links in order but will probably only get to the first one since I should be asleep.
Thanks again,
Jorge
Here's a series of effect blog posts on tiling.
Sub-nested tiling (what i referred to as nested sub blocks) means that smaller tiles match up to the original tiling spacing as they are laid out. So 4 128x128 pixel tiles would exactly fit inside of 1 256x256 tile.
...
One current limitation with size modulation of a movie brush in the paint synthesizer is that the parameters you edit in the paint synth control panels are integers. This isn't true for all studio artist operation modes, image ops or MSG let you edit floating point variables.
The side effect of being restricted to integer size modulation is that if you want to 'sub-nest' tiles, you are limited to 100%, 50%, and 25%. Since you can't physically enter 12.5%.
The way you get around this is to use 2 different sized movie brushes. So a 256x256 movie brush would let you sub-nest tile at 256, 128, and 64 pixel sizings. You could then use a 64x64 pixel movie brush to sub tile 64, 32, and 16 pixel sizings. These 2 movie brushes i'm referring to contain the same images just sized differently.
This integer editing restriction for paint synthesizer parameters will be resolved in a future release.
...
Brush Size in Studio Artist is a function of 2 different control panels. the Brush Source control panel is where the maximum size brush is defined. So when you load a movie brush, if it's based on a 256x256 frame size movie, then that size is shown in the Brush Source control panel. If you were working with a computational brush, then you could set that max brush size with adjustable controls in that control panel.
But the Brush Modulation control panel contains many different editable controls that allow you to determine how the original max size brush source is modulated. You can specify how the size of the brush is modulated, now the orientation of the brush is modulated, and also potentially modulate the appearance of the brush by multiplying it with the background texture module in the paint synthesizer.
The background texture typically is a procedural texture generator, but could also be an image texture, or a movie texture, or a live video feed texture.
...
Sub-nested tiling is just one of a million different approaches to building a photo mosaic.
Here's a link to a post that discusses hexagonal tiling for a mosaic effect.
Here's a link to a mosaic tiling effect that looks more like a natural hand tiled mosaic build out of stone pieces. It's based on emulating a pantograph to build the mosaic cells.
Here's a link to an article that details a whole different approach to building a non-regular mosaic. So it lays out a compendium of potential non-regular tiling techniques.
...
I think you may be starting to get a feel for how flexible the studio artist paint synthesizer is capable of being. So the last article i pointed to above works to generate tiling effects in a very different way than just laying down brush nibs in a regular tiling pattern.
Studio Artist has a feature called 'path start regionization'. It's used in conjunction with a pen mode called 'region fill as brush' pen mode. More strange terminology being thrown out to confuse the beginner i know.
The way to think about region fill as brush pen mode is that you use the pen to define a region shape (like drawing with a lasso tool). That shape is then used to define a dynamic brush shape on the fly. So it's a whole different approach to painting than the normal apply a series of individual paint nibs along a vector path style of painting. The path actually defines the shape of one nib of paint in this particular pen mode.
Path Start Regionization is a very sophisticated automatic visual analyzation of the source image. The source image is intelligently analyzed, and broken up into a series of individual regions. The paint synthesizer can be configured to then automatically paint in these intelligent regions in a variety of different ways.
When using these 2 features together to build photo mosaic (or photo collage) effects, a frame from the movie brush is dynamically resized on the fly to fit the automatically generated region. The generated region could be rectangular, but most of the more interesting variations are irregular shapes that are smart about representing the details of the source image.
So the image above is an example of what i mean by all of this. The photo mosaic is an irregular tiling. It's adaptive, in the sense that it is laid out to try and intelligently represent the details of the source image. And the tiles all match up without any overlap, so it's not random positioning at all. And the individual blocks in the tiling were filled by using the 'region fill as brush' pen mode, as opposed to the conventional paint synthesizer approach of directly painting with a nib of paint derived from the brush source.
The movie brush used for the photo mosaic image above is a movie of a series of images of brick walls. Both the movie and the paint presets that use it are available for download here. The original photo post that lead to the discussion that lead to me posting the presets can be seen here.
...
I hope this discussion is starting to give you a feel for why studio artist is the most sophisticated and versatile tool for creating photo mosaic and photo collage imagery on the planet.
Learn alot from this John ty.