• Register
  • Help
Page 1 of 2 12 LastLast
Results 1 to 10 of 19

Topic: Musician Modelling

  1. #1

    Musician Modelling

    I personally believe that "Musician Modelling" is an important next step in recreating ensembles. Why is it that computer composers can have highly detailed samples, yet still create a "flat" performance? Much of it is due to the fact that the composer doesn't know much (if anything) about how to play the instrument in question. Without that expertise, he or she is truly just "playing the sample", and he's bound to the expression that is imprinted on the data.

    Even if the composer knows how to play an instrument, and knows how to get the best from the samples, the process of playing each part over and over (and over!) for each member of a large ensemble (e.g. orchestra) is extremely time consuming! Moreover, current control schemes are woefully inefficient for capturing a realistic performance. With towards 10 controls shaping a single instrument performance in JABB, and only two hands and (a maybe a foot or two) to record with at one time, each part takes several passes!

    Without specialized virtual controllers like EWIs to "broadband" this performance data across, you have no choice but to make this a step by step process by recording slider or knob or mouse motions. While I'd love to see more specialized controllers emerge, I think there's more of a hardware hurdle due to lack of demand for them. And even if these controllers were available, a composer would then have to master them all before he or she is in the position to "save time" when making a mock-up (let alone cost restriction).

    So my take on the solution is to enhance "human playback" software systems to include the quirks of an individual. Some HP systems right now capture the randomness of performance, and that's great, but I'd like to extend it to "personalities".

    For instance, we have a trombone player named "Tony". Tony likes to really lean on low notes, so his mf on the bottom end probably creeps up towards forte. He's also a little slow to start a cue when he's been resting for a while (lips get a bit "cold"!). For fast passages, though, he gets a bit excitable and rushes them a bit and gets a bit lazy on the articulation so that he's perhaps slurring a bit too much. His slide positioning tends to be a bit flat, too, but he adjusts fairly quickly when his ear picks up on it (on longer notes).

    Other players in the section would have different quirks to their performance in slightly different ways. "Shane" can play louder. "Jim" has a cold so he runs out of breath quicker, long held notes petering out gradually. "Bill" is quick on the ramp to a crescendo. All of these performers together create a more varied, richer timbral and textural response.

    You could use these personalities as presets, tweak them to taste, or randomize from scratch. Additionally, you could introduce the concept of a "section leader". Say that "Irving" is first chair. He has a moderate (50%) amount of sway over the other members of his section. As such, everyone else's performance patterns will be "morphed" to approximate Irving's playing style. In this manner, you can give a bias to the perfomance, while still allowing some weighted deltas on the part of other players.

    Then comes the beautiful part: you become the conductor, with a gestural controller or MIDI data track, and each of the performers reacts to your direction on an individual basis (based on their parameters, influenced by section leaders), so the ensemble lives and breathes not as a machine, but as the real thing.

    I think it would be very nice to be able to play back the same line of notation for each of these personalities, through the exact same sample set, and be able to determine who is playing according to what you hear in the performance and how it responds to your direction. After all, if you gave the same trumpet to Lee Morgan and Freddie Hubbard and told them to play the same line without embellishment, they'd still be discernable via their tonguing curves, vibrato shape and speed, etc. (well, *and* a brass player does exhibit a certain sound based on embouchure, but let's not get too picky! )

    Of course, having access to different instrument sample sets will only enhance this. Let "Lee" play an Olds Ambassador. Let "Freddie" play a Calicchio 3/9L. Whether these sounds are made by samples, additive synthesis, or physical modelling, it will only serve to further distinguish their "personalities".

    And really, much of this Musician Modelling can be done today. We don't have to sit on our heels and wait for some technology breakthrough. It is low-hanging fruit, just waiting to be picked, but it *does* require a bit of cooperation between sample library and host/notation program developers. Will we see it happen?

    The goal would be to allow composers to be composers, not experts in tweaking MIDI CC messages or virtuosos in a handful of virtual instruments. Convincing mockups should be as easy as dropping notes on a staff with efficient, minimal gestural notation. Building your ensemble should be as easy as selecting "musicians" and pairing them with instruments according to the preference of your ear. And as part of that, using a new sample library shouldn't make you have to learn a whole new control scheme, wasting your time to become an expert on the ephemera of yet another piece of software. I'd love to be able to buy a single instrument "off the shelf" just for the love of the sound ("one Buecher tenor sax, please!") instead of having to discount it because I'd have to learn a new control scheme.

    I'd definitely like to be a part of making this happen.

    - m
    Free MFX and other plugins: http://www.TenCrazy.com/
    Markleford's music: http://www.markleford.com/music/

  2. #2
    Senior Member
    Join Date
    Jun 2004
    Brooklyn, NY

    Re: Musician Modelling

    Gee, maybe we could somehow combine GPO with 'The Sims' - then we could actually watch them play as well as listen to them. Of course, they'll also want beautiful houses and the most expensive cars...

  3. #3

    Re: Musician Modelling

    It would be really cool!

    Personally I don't like much to have go through all this midi cc programming stuff. It would be cool I could just choose character to play my music, like you say - I would become the conductor not the programmer

    my cafepress store
    my fine art print store
    my music store

    Please support your fellow musician from Iceland where the currency has fallen 70% against USD and 90% against EURO! Every dollar or euro counts!

  4. #4

    Smile Re: Musician Modelling

    I agree: your is not a crazy dream but a feasable project.

    The amount of programming, the CPU load resulting, the arbitrary decisions related to style, are a huge work, that can terrify every business oriented software house.

    I see for your dream two possible ways to become real:

    - the start of an open source team of experienced but young programmers looking for fun and challenging to show his ability to find new jobs and contracts.

    - a slow step by step evolution of new HP versions: may be we will have Finale 2013 with Human Players enhanced ensamble maker, with 3D virtual reality sensors to conduct the score in real time, generating with your hands undered of controllers...

    In both the case, I think that GVMISA (Garritan Virtual Musical Instrument Shop Advanced) will be used to give an instrument to the Human Players.
    It will run also on MAC OSY for Pentium7 in your Home Media Wall (a wall of your house providing computer, multimedia and communication tools, and house automation all in one)

  5. #5

    Thumbs up Re: Musician Modelling

    Re-reading my previous message, I feel the need of a clear up: I'm NOT ONLY JOKING!

    It's really a meaningful description of a very good and possible project.

    The fear is the amount of time/work necessary to do it, and some new or more extended application of existing technologies.

  6. #6

    Re: Musician Modelling

    "Wes" always shows up late, plays flat, and mooches some of the pizza brought by "Tony".

    ...or is that TOO realistic?

  7. #7

    Re: Musician Modelling

    Musician moddeling is already well on it's way I think. Some instruments lend itself easier for it.

    Band in a Box has done a fair bit to moddel musicians on Piano and guitar. It is mainly done by analysing phrasing, harmonies and emphasis from existing recordings.

    Classical instruments like GPO's library other than solo instruments would be almost impossible, as it is more about orchestration and choice of instruments. I think many has tried to computerise orchestrations, including BIAB, but I have not yet found anything convincing enough.

  8. #8
    Senior Member squoze's Avatar
    Join Date
    Aug 2004

    Re: Musician Modelling

    similar discussion in this thread

  9. #9

    Re: Musician Modelling

    Quote Originally Posted by Markleford
    I personally believe that "Musician Modelling" is an important next step in recreating ensembles. Why is it that computer composers can have highly detailed samples, yet still create a "flat" performance? Much of it is due to the fact that the composer doesn't know much (if anything) about how to play the instrument in question. Without that expertise, he or she is truly just "playing the sample", and he's bound to the expression that is imprinted on the data.- m
    I think you are spot on in both your analysis of the problems, and at least one avenue towards a solution... but, I am not sure that is all of it.

    GOS, GPO, and some of the really large/high-end libraries have made tremendous strides towards realism, and in the hands of someone who really knows how to use the tools, the results can be VERY realistic.

    But there is still that element of perfection, or rather the lack of randomness that can show through, and I don't know it if is solved by musician modelling alone.

    If the musician modelling takes into account the differences between different instruments as well as different players, and randomizes other attributes then perhaps it is the solution.

    And I mean no disrespect, but this search for more realism goes back to (at least) the early 1970's... it is the holy grail for synthesis. Way back in the dark ages it was unrealizable, simply because of the available horsepower. Subtractive Synthesis was tool of choice and it has limitations! You could do FM, in a VERY limited and not too reproducible form on the ARP 2600 and some others, and if you built a large enough modular synth you could do some primitive additive synthesis as well.

    Then there was C-Sound, and all the other synthesis tools that ran on rather large computers, and all in batch mode. They were cool, and they led to some wonderful research, but they were not at all easy to use.

    One of the more important lessons I remember was that it was next to impossible to synthesize a single violin, for example, but we could synthesize an ensemble that had a passing resemblence for the real thing, at least with respect to everything else that was going on then<G>!

    My first sampler was the Mirage, I don't remember how much memory it had, but it wasn't a lot, and they managed to jam an awful lot of usefulness into the box in spite of that. And it was a huge leap forward. Huge. I also remember hearing a demo of the Kurzweil 250 at an AES show back then, and being completely spell bound!

    And of course that was just the beginning... and at first most of us were satisfied with the samples we had. But after a while we began to recognize that they weren't a replacement for live players. They simply were not real enough. So we learned tricks!

    One of my favorites was combining samples, which were static because of the technology limitations, with synthesis, which wasn't all that realistic, but it was dynamic. For example, the Yamaha TX-81Z had the ability to play in monophonic mode using all eight voices in a round-robin schedule. The first note would play patch #1, the second note would play patch #2, etc. If you were to use a cello sample in the Mirage, and eight slightly different cello patches in the TX-81Z, the result was really quite animated. Maybe too animated<G>, but it beat the heck out of static samples.

    But I ramble...

    I think this is a very important area for research, and I know of at least three companies that have already made huge investments in that research, not to mention dozens of universities!!!

    There are two obstacles presently - the first will go away, it is the resources available for implementation, and that increases daily. The second will be a bit more challenging. How do we recognize, let alone quantify those elements that make a musical performance interesting? And how do we allow for the fact that you and I may have different opinions on just what makes a performance interesting??

    Still, I really enjoyed your post, and your descriptions. It is still more food for thought on a topic that is near and dear to probably everyone in this group!!!

    Take care, and thanks for the great post!


  10. #10
    Join Date
    Oct 2005
    London / Uzes (France)

    Re: Musician Modelling

    This has been done for CG characters in the animation world - the program's called "Massive". Each "agent" has a brain which is pre programmed with certain parameters (character, reaction to outside influences, in the case of a virtual musician a set of rules as to how the musician plays, what styles he/she favours etc), and at runtime the agents interact with each other to create the scene (or in this case the song ).

    To get a true *ensemble* feel, any virtual musician would have to interact with other musicians (bass locking with drums etc etc), so it would be a massive (no pun intended) AI/neural network system.

    You could eventually go the whole hog and have a virtual conductor for a virtual orchestra. The conductor would be set with simple rules as to how they interpret music, and then just hit play and see what happens.

    A system based on AI would also have the benfit (or drawback, depending on your point of view) that no 2 performances would be the same...

    Very interesting thread.

Go Back to forum


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts