Audio Modulation of SA Graphics?

Here's how I got into this topic.  I used to play digital keyboards in my spare time, eg the Kurzweil PC3K6.  But when I got serious about digital art, and especially Studio Artist, all that time got pre-empted--but I miss my music.  So now I'm wondering whether a synthesizer could provide some kind of audio modulation for graphic effects in SA--so I could play, and yet also be doing serious digital art at the same time! :-)

 I checked back in our SA Forum, and found that movie audio pass-through was introduced in SA V3, in 2007.  Subsequently, quite a few people have suggested various kinds of audio modulation for SA, and generally synthetik was supportive, looking into it and so on. But now, 11 years and two major new SA versions later, there's still no sign of any audio modulation support.  Why is this?

 A detailed overview of the state of play would be much appreciated.  Eg were none of the audio modulation suggestions feasible, of sufficient value to implement, or of interest only to a few?  Or is audio modulation still a desirable future addition--and if so, in what ways?   Thanks.

You need to be a member of Studio Artist to add comments!

Join Studio Artist

Email me when people reply –

Replies

  • John Dalton, could you please say at least something on this issue?  I'm sure many others would also like to know where we are with audio modulation.

  • I'm always happy to hear people's suggestions for enhanced or new studio artist features. So fire away if you have some specific feature requests regarding audio modulation features you'd like to see. I'd also like to hear how that would fit into your existing or future workflow.

    Personally, i think osc or midi modulation would be potentially more useful. But i'm more than welcome to listen to people's (specific) opinions about how they would personally use direct audio modulation.

    Keep in mind that there are multiple hundreds (probably more) of features that have been suggested here over the years. Some end up in future releases, others do not. There are always tradeoffs that need to be made to get a product out the door. Many things are also beyond our direct control (like software apis changing and/or vanishing). that's why we have always made a point of not discussing future release specific features until they are actually released.

    And of course we always need to gauge whether a feature is going to be used by 5 people, or a large number of our users. Or whether a small group of users will get excited about something, but a larger group will just get confused and dismayed by it because of associated complexity. So there are always a lot of different tradeoffs that need to be made associated with any potential new feature.

    You can easily add a quicktime movie file that contains an audio track with visual hit point markers on it, and manually keyframe visual changes. So there are work arounds available today for people interested in synching visuals to specific audio hit points.

    Again, i'd really like to hear what people would specifically want to do with this. Are you interested in real time manipulation of studio artist. or modulation associated with non-real time movie rendering? Very different ways of working with the program. What specifically would you want to modulation? With what aspect of audio? How would this be better than manual keyframing to bar/bet markers? Etc...

    • Thanks John, that's helpful. I'll add my main suggestion as a new item in the Feature Requests forum, to keep things better organized.

    • Given that almost 100% of my work focuses on (creating) a dialog between music and images, there’s one feature we used to have and unfortunately lost, and which I find essential, it was/is the ability to set “in" and “out" points in the source video when dealing with a PASeq.

      That loss has forced me to get into workarounds that are not intuitive and has greatly reduced my use of SA to it becoming like an add-on to FCP X, while it used to be the other way around.

      The kind of intuitive articulation of the video stream I was able to get into in SA is no longer possible and has now been shifted mostly to FCP X (and the GLMixer).

  • This discussion got me thinking. It might be entertaining to think about this in reverse. Rather than audio modulating studio artist, using the way a painting is generated to derive musical output. I don't mean a spectrogram, you can do that already in a number of existing applications. I mean translating the physical painting process into a midi stream output.

    • But somewhere in the archives you said that interesting painting effects typically would translate only to unsatisfactory sounds?

      • I think in that archive i might have been talking about using the painted image as a spectrogram.

        There are other approaches people have looked at more recently that use neural nets to generate sound from images that are different than using an image as a spectrogram.

        What i was referring her was using the dynamics of how the paint strokes are applied to generate sound. Again, very different than a spectrogram approach.

        • Thanks, interesting!

    • Right on.
      Like UPIC or Metasynth... But without any commitment to deal with audio. 

  • Hello, everyone!

    I am a brand new user of Studio Artist 5 and I think this issue is what I am trying to solve.  I came to this program to make abstract video content for music I have recorded.  I hope I can reasonably do this with the program, but am struggling with the daunting learning curve.  If I can be pointed in the right direction, that would be great.

    Two issues so far - 1) using dual paint mode and loop action to record and output a movie works, but the resultant movie file runs much faster than I see when creating it in the program. How can I regulate speed, both while creating the loop action and in output?  2) After turning off the loop action, I was using the dark objects preset and clicking to interactively create new overlays (very cool) while listening to the music, but the timing was slightly delayed.  My idea is to later combine sound and visuals in Moviemaker.  In both these circumstances, how might I sync the timing, either speeding up or slowing down the visuals to match the BPM or when I click interactively?  Maybe my HP Spectre 360 laptop does not have the graphics capability that I need? There seem to be some jerky motions, maybe my laptop, maybe the desired preset effect, I don't know. Or maybe I need another video processing program to bring these together in tempo?

    I may need to scale back my ambitions with Studio Artist, but either way, I see possibilities if I can surmount the learning curve.  

    Sorry - I'm so new to this...

    Paul

This reply was deleted.

Interdimensional Coincidence Control

Hi everyone, I am glad the site is still here! Here is a new short video I made. All made in Studio Artist, several separate videos with alpha channels, then combined in layers with the music in Blender. A lot of MSG running through brushes, with several of the brush Path Starts being controlled by the MSG Scan Generator in the Generator part of the Path Start in the Paint Synthesizer. Also some MSG running through a brush, then making a video of that with an alpha channel, then making that a…

Read more…
3 Replies · Reply by Thor Johnson Mar 22

Whats going on with this site?

Has anyone else gotten a warning about this site disappearing? An email form just popped up, asking me to contract the owner and leave a message to let them know that they may loose their "network"Did Synthetik forget to pay it's bills, or is something else going on?I think 8 months is more than enough vacation time. Is anyone at Synthetik doing any development work at all? 

Read more…
4 Replies · Reply by Emil G. on Saturday

Having difficulty exporting canvas as image

I'd like to export a canvas as a .tiff/.tif image file to a folder I made on my desktop.I select that from a dropdown menu, I can name the exported file, change the extension, etc, and I press save but nothing happens.It's always worked until now. It seems like a simple task. Any ideas?I'm on Mac OSX 12.6, if that matters, and my system hasn't changed since the last time I was able to export successfully.Thanks  

Read more…
2 Replies · Reply by Tony Bouttell Mar 5