• Register
  • Help
Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Topic: Samplers:Is sample-accurate rendering needed ? Your take?

  1. #1

    Samplers:Is sample-accurate rendering needed ? Your take?

    the other day I had a discussion with Christian (main dev of LinuxSampler) regarding what's the best acceptable tradeoff between percormance and accuracy when it comes to sample playback.

    The reason why I'm asking is because we want to maximize performance (aka polyphony, lower CPU usage etc) avoiding complicated sample playback algorithms which will not bring tangible advantages in practice.

    We would like to hear your opinions as very demanding sampler users (both in terms of audio quality and system performance), opinions which will be valuable to influence the direction of the development of LinuxSampler.

    Now to the question: Is sample-accurate rendering needed ?

    Let me elaborate a bit:
    Basically as we know most virtual instrument plugin standards like VST and others allow for sample accurate audio rendering.

    This means for example that if you have a midi sequence in your sequencer and use a VST plugin to render the audio
    then you will not be limited by the resolution of physical MIDI (which takes about 1.1msec to transmit a note-on/off message) but you can get down to the resolution of a single sample which at 44.1kHz equals to about 22usecs = 0.022msec.

    The problem of handling sample accurate rendering in software samplers (and software instruments in general) is that since audio playback in PC enviroments is implemented by sending a buffer of N samples to the soundcard, the most efficient way to play a sample is to try to avoid complex routines and algorithms within the innermost audio rendering routines and sample accurate rendering requires some additional program logic which can decrease the overall performance in the playback.

    Let's take the example of playing a sample with a simple exponential envelope.

    Assume a buffer of 64 samples. (1.5msec worth of data at 44.1kHz).

    If we support sample accurate rendering and want to start the playback of the sample at buffer offset 19

    we need insert a branch instruction (if .. then) and calculate when we will sample offset 19 to start rendering. (this must not be within the inner loop so the the performance hit will not be so big).

    But enveloping (and filters and any synthesis paramenter in general) do need some special checks which can sometimes be time consuming this decreasing the performance.

    For example when doing sample accurate enveloping you have multiple ways to implement it.

    One could be calculating the offset where the envelope starts, set the parameters and then calculate new envelope values within the innermost audio rendering loop.

    Or you could (LinuxSampler uses this approach currently), pre-render the envelope information for the whole audio buffer (64 samples in the above case) and then just have the inner audio loop fetching those values from a table.
    Unfortunately this causes a performance hit since you are moving lots of values back and forth in memory (even if they are cached in the CPU's cache it will still eat some CPU).

    So our question was wheter sample accurate rendering is really necessary in practice or if it's just overkill.

    For example if you take normal MIDI devices using regular MIDI cables. As we know MIDI works using a serial connection with 31kbit. Each midi message is made of one or more bytes (for example a MIDI NOTE ON is made of 3 bytes). Transmitting 3 bytes over the MIDI cable takes about 1.1msec.
    So for example if you play a 7 note chord on a MIDI master keyboard connected to a MIDI device, the latest note will start sounding about 7msecs later.

    But it still seems acceptable even for live play since all electronic instruments still use MIDI as a commuication protocol. (let's forget about firewire,m-lan etc for now since they are not widespread yet).

    Of course with serial MIDI communication you can hit the wall pretty fast, especially when playing a full (16 parts) arrangement through a single MIDI cable. Timing gets sloppy, and you have to do tricks like moving non rythmic parts off the bar/beat marks to avoid the drums, basslines etc sounding like played by a bad musicians.

    Now enter the PC:
    When a sequencer like Cubase sends MIDI events to VST plugins it does not have to go through the slow external MIDI protocol. It uses the computer's RAM which is blazing fast and the limit of the number of note events per second is only given by the speed of the computer and as we see we can easily achieve sample accurate rendering.

    But does our ear hear it ?
    For live playback even a minimum of chords smearing (the above example of 7msec delay) can hardly be heard and MIDI devices are used live all the time.
    The PC can process many simultaneous MIDI notes without delays. For example assume the 7 note chord example but this time using Cubase and a VST sampler.
    If you look at the audio notes will probably all start at the same time because the VST plugin renders N samples (N = current audio buffer size in the sequencer).

    So what if the VST sampler instead of doing sample accurate rendering quantized the MIDI events a bit.
    For example even if you use a relatively big audio buffer of 2048 samples(for offline rendering or to achieve greater performance).
    The VST sampler could quantize MIDI events to an offset that is a multiple of 64. Which would mean that you would loose sample accurate audio rendering and all midi events would be quantized with a granularity of 1.5msec which would be as good as MIDI devices using the normal serial MIDI protocol playing a single note.

    The advantage of PC based sampling would be that even if you quantized events if the sequencer sends a 10 note chord where all notes start at the same time, in the resulting audio the notes would still start all at the same time because RAM based MIDI transmission is instantaneous.

    So after all this boring explanations I pose my question again:

    Is sample accurate playback really needed in most cases (plase make an example where you cannot live without it) or is quantized sample playback with reasonably low quantization times (like the above 1.5msec, alias 64 samples) just fine ?

    The advantage of quantized sample playback is that you can use simpler algorithms which means bigger performance.

    I think its probably the reason why Gigastudio is more performant that other software samplers.
    Gigastudio cannot work as a VST plugin and therefore can make the above assumtions of quantizing the rendering MIDI events at the start of buffer boundaries.

    Did anyone make some tests with common VST/AU softsamplers to see if they are really sample accurate, if this sample accurate mode can be turn off and if yes what performance gains you had ?

    We would like to hear your opinions and experiences on the sample accurate / quantized sample playback rendering matter.

    You could try to do the following, assuming you use a software sampler which provides sample accurate playback: make a sequence in your sequencer and render 2 audio files. One with the sample accurate events and the other where you quantized notes where you use 1.5msec quantization values. Post the files here without telling what one is the sample accurate audio file and which one is the quantized one.

    Then let people here on the forum try to figure out which is the one is the quantized one.

    Thanks for listening and for your valuable inputs.

    PS: for the curious. LinuxSampler is not an useless piece of opensource code but it actually makes sound
    Below is a videoclip taken at musikmesse of the PMI Boesendorfer played on a linux-based keyboard, too bad the 76key keyboard cannot bring the Boesi to its full expressivenes.


    PS2: porting the sampler to other operating systems is in the works and I think a native VST/AU is not too far away. Plus, many of the GIG v3 format features are already supported too (except convolution).



  2. #2

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    For those wanting to watch the video - safe way is to choose "save as" after rightclicking it, directly showing did not work here.

    I don't know if it's better to have variable transmit times instead of fixed ones. What would happen if we go on the upper limit on let's say like 40 note on events in a bar with various controller informations on almost all of them (this is possible in a big orchestration) - let's say we used Finale or library midi file input, all extremely quantized. How would that affect the performance? I can see there is serious performance improvement in live situations, but I'm unsure about complete orchestrations that are also in a way "live" input for the sampler.

    Thanks for your explanations and good luck on finishing your project, it looks very promising!


  3. #3

    Re: Samplers:Is sample-accurate rendering needed ? Your take?


    Now to the question: Is sample-accurate rendering needed ?


    I only can speak about my personal point of view regarding this:

    Of course, someone who programs string quartett mockups can live without sample-precision. The unavoidable jitter in MIDI transmissions (values of 3-4 ms for a single note are not unusual in a multitasking system like a PC) is not a problem of life/death for him.
    Things change dramaticaly when one deals with rhythm-oriented programming. In this case, yes, 3-4 ms imprecision is lethal. Just to give you the simplest example in the world: for years now, in the struggle to program a realistic ride cymbal, I put together two samples: a ride and the adjacent cup (from the same cymbal). By varying the velocity of the cup in respect to the ride, I achieve a more " alive" feeling.
    Now...you can imagine that I have two sounds playing together, but I want them to be heard as one. No flams. One cymbal. Without sample-precision, this can't be achieved, period.
    This was a very simple example. Modern samplers like DFHS or BFD use 12 or 14 sounds for A SINGLE STROKE (the main sound, the bouncings into adjacent drums, the bouncing into overheads and ambience mics etc). Imagine which flams would appear without sample-accuracy :-)
    Hope it offers a valid point of view.

  4. #4

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    multiple mic positions playing back at the same time can create the need for very accurate playback

    also there are other library designs and sample methods that can/will show up that will require very accurate rendering. Whether you want to support that in your playback format may depend more on your choices in this aspect of performance tho.

    Why not have an option for accurate rendering and one for "quantized" stuff for realtime implimentation for people who are jsut "noodling". I could see alot of people living with the less than accurate rendering, and some even being fine with the render quantizing.

    having it all would be the best of course
    Operation Mindcrime 2, in stores now.
    or go here...
    The Digital Bitphonic Orchestra
    -Ashif "Ash" Hakik

  5. #5

    Re: Samplers:Is sample-accurate rendering needed ? Your take?


    Are we talking only about sample playback, or rendering in general?

    If we are talking about rendering in a dynamic environment, within virtual softwares, then that's a whole can in itself.

    If rendering purely in a playback situation within vsti management, then you need to consider the overall usage of multiple buffers for mulitple streams of data, a system most cards adopt. Fixed points and quantise methods work up to a point, but when faced with multiple realtime scenarios, some form of floating point processing is required, and for that you will need detailed algorithms. Quantising also introduces problems when faced with further dynamic processing and then dithering. If you do not maintain fully divisble integers, then you will need higher alais factors, and that in itself is a hindrance when using dynamic processing and dithering.

    Another area that will be affected by this is the area of sample management. Sound designers have to have sample accurate integers for their cycles, otherwise accurate looping will be a problem and dc offsetting will only adjust the axis point and not correct the cycle integrity. The rendering of the final output, that then needs to be submitted to the contractors, will not be accurate. Quantising in this situation does not help, as systems that use this method do not provide cycle accurate sample loops.

    The final obstacle I can foresee is that of re-rendering. The more inaccurate the rendering algorithms, the least amount of times the process can be applied. If quantising is used, then rendering the sample again and again will cause all sorts of anomalies and inaccuracies. Before anyone jumps down my throat about this, think about the application of realtime effects and dynamics within a playback scenario.

    I was thinking that with today's cards having mulitple buffers, today's softwares having floating point processing, and today's ram and cache management, that the detailed algorithms would not pose such a noticeable difference in playback situations.

    Maybe it is more down to the way the host software and hardware manage mulitple vstis and vsts that causes latency and anomalies associated with sample accurate playback?

  6. #6

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    thanks for your responses.

    I fully share your concerns, but since my time quantization proposal quantizes only the trigger times of notes but does not impose limits on the number of notes that can be triggered at the same time it would not introduce note smearing etc.

    Let me make the following example:

    You have a combi sound which fires up several samples at the same time (like the drumsounds you described).

    Assume you use Cubase and a sample-accurate VST sampler and assume we use the current sample playback position as a time base. (44.1kHz sample rate)

    For example time = 88200 means we are at position 00:2.000000 in the song (2 seconds from the start) ( 6 zeros after the dot means precision of down to 1usec (microsecond), just to show the difference between quantized and non quantized behaviour)

    Now let's say the we want to fire up the drum sound that consists of 4 samples played at the same time
    at sample position 88210 which translated into absolute time
    equals to
    2.000226 secs from start

    The sample accurate VST sampler would trigger all the 4 notes at the exact position 88210.

    Now assume the same piece is played using a time quantization of 64 samples.
    This means that if you take the 88210 value and calculate the next multiple of 64 you get sample position 88256
    which equals to
    2.001270 secs from the start

    In the above case it leads to a quantization error of 181usec which is a really low time (as said transmitting a single midi note over serial midi cable takes 1.1msec).

    But this quantization error is the same for all 4 notes since the VST plugin can process as many simultaneous note-ons as you need.
    So basically in the above drum sound example all notes will start at the same time, with the only difference that they will sound 181usec (0.18msec) later.

    KingIdiot: your concerns are of the same category as the ones from MozillaUser. So multiple mic positions playing back at the same time will still work perfectly.

    Samplecraze: perhaps you misunderstood me, I talked only about quantizing the note trigger times a bit, the sample data will still played back in full resolution (16,24,32bit float ingpoint, whatever).

    Ok, any other examples where sampel accuracy is needed where my proposal fails to satisfy requirements ?

    Please keep discussing


  7. #7
    Senior Member Bruce A. Richardson's Avatar
    Join Date
    Sep 1999
    Dallas, Texas

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    My thoughts:

    I agree with King Idiot that a switchable mode might be ideal. I think most phasing issues would be moot, since, as you say, the quanitization would probably just push back the samples equally and all would be well.

    The second issue it would solve is some 16-year old coming onto a forum like this and screaming that your sampler is not sample accurate, and creating a tempest in a teapot over it.

  8. #8

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    Quote Originally Posted by sbenno

    Let me make the following example:

    The sample accurate VST sampler would trigger all the 4 notes at the exact position 88210.

    Now assume the same piece is played using a time quantization of 64 samples.
    This means that if you take the 88210 value and calculate the next multiple of 64 you get sample position 88256

    But this quantization error is the same for all 4 notes since the VST plugin can process as many simultaneous note-ons as you need.
    Ok, now I seem to get what you are up to.
    No, such small imprecision wouldn´t disturb me too much. But... 'cause there is a "but", you see :
    1. I wonder why should you do that? Do you have any gain, in terms of processing time or something? Au contraire, you spend a little more time - I guess - in zeroing some LSBits in the time information. Am I wrong?

    2. You know how it is: one uses different things for personal purposes; one of those things is "having a VSTi on another rig, and rendering - or importing - the wave from there. Now, when importing, is not unusual to displace the imported wave a few samples to left or right, to compensate for latency or other issues. When I´m sample-accurate, I know for sure: if the first note is lined up with all other tracks, the whole imported track is lined up as well. Will this be valid for the "quantized-as-you-say" track as well? Please consider that your "fixed" quantize has nothing to do with the tempo, and not all songs are at "fixed 120bpm" tempo :-) Tempo might change, but your quantizing remains fixed, you see. Suddenly, because of the tempo change, one note or a group of notes fall into another quantizing slice. What was "nicely laid-back" originally, suddenly is "unbearable late". Please believe me, these things are very, very sensitive.

    3. I´m a guitar player. I can play equally bad an American Strat or a Koreean Squier, which is a copy. No one could tell the difference, let´s pretend. I can ruin them just the same ;-) But....after ruining an American Strat, I don't know...I sleep a little better afterwards. I guess I sleep a little better when I know I´m sample-precise, and not quantized in some way I can neither control, nor whatever.

    Of course, I don´t want to overcriticize your intentions. I just try to explain my point of view, which can be as wrong as anyone else's.

  9. #9

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    Bruce, I agree about the switchable mode, we will see what we can do. (Currently we are sample accurate but not fully satisfied with the performance, perhaps we will switch to a more optimized algorithm).

    1) The gain you have in quantizing the events to the next audio buffer (or fraction of it) is that within that buffer timeframe you don't need to perform checks if some kind of events occurred (new notes, modulations, envelope,filters state changes). And in some case this can save some CPU time. The actual quantizing of time information is very light on CPU (1-2 machine instructions), something like: quantized_time = (time LOGICAL_AND 63).

    2) You are right about latency compensation but keep in mind we talk about very small values here. 64 samples = 1.5msec and the average error will be around 0.7msec. So assume you want to compensate 20msec latency in your sequencer. The quanrtizing VST sampler would still compensate the latency correctly with the exception that notes would start with a mean error of 0.7msec around the ideal value. And I hardly believe it can be heard.
    Think about this: Assume you start and stop a sound each 1.5msec. The result will be 1/1.5msec / 2 = around 300 cycles/sec = a 300Hz tone.
    Can you hear a single cycle of a 300Hz tone ? AFAIK the answer is no. Feel free to prove me wrong.

    You wrote: What was "nicely laid-back" originally, suddenly is "unbearable late".

    Regarding the tempo change:
    the quantizing time (1.5msec as in the examples above) will remain constant, and is totally independent from the tempo.
    If you increase or decrease the tempo, the sequencer will calculate the sample time (the timebase in VST is the current sample) when to render the note.
    And the quantizing algorithm will take this note and "round" it to the next buffer boundary.
    So the average note trigger time eorror will still still be about 0.7msec in average which is a damn small time.

    Don't worry MozillaUser, (constructive) criticism is exactly what I wanted to hear in response of my postings, which will help to understand what most critical users demand for high quality sample playback. And as we know if you satisfy the most demanding user you will satisfy any user


  10. #10

    Re: Samplers:Is sample-accurate rendering needed ? Your take?

    Hi Benno,
    At this time, I could give you the short or the long answer. It's up to you to choose if you want to hear the long one - in which case we should continue this discution by private email, since the reader's time is to be respected, right?
    In short: More I read, the more I consolidate the impression that this thread is not about some VSTi sampler, but about a HOST. Am I right? In fact, your debating question was to be "Look peeps, we here want to write - or port - a host. Forget about the sample-accuracy (coz my question applies not only to samplers, but to synths as well) - it was the wrong approach; I take it from the top: in fact, we have problems with the quantizing. We feel that the whole long and glorious road from C64s and Sinclair Spectrums ("Our Sequencer quantizes at 32 ppq! which is almost as good as realtime!") through Ataris 1040 ("Our sequencer quantizes up to 256 ppq! which is almost as good as realtime!"), through whatever Falcons ("Our Sequencer etc etc") up to present G4s or P4s or whatever, which are quantizing at thousands, or tens of thousands of ppq's - this long and glorious road is simply too much. We feel that nobody needs that degree of definition - but we can be wrong of course, that´s why we want to hear from you at which point the evolution was becoming sick, out of purpose, just to impress the buyer's eye with some high figures"

    If this is the case, then I only wish you the best luck in the world. You want to create a HOST which quantizes at a more modest precision than your competitor's, and you even hope to be successful in selling it (oops, "selling"...we talk about Linux?....err...)? Best of luck from my part! :-)
    Or, au contraire, you want to create/port a host which can quantize at those high figures - so the buyer's eye is impressed just as much; it can request that precision from its plugins; but the plugins are to be independently-thinking: "This stupid host asks me to do nonsense. I will quietly quantize it down, I will simply throw away the last 6 LSBs from the timestamp (quote: "AND 63h") , and hush-hush, I won´t tell nobody about what I do, so no user (read "no 18-year-old whinner", oink, oink) would be intrigued. Lots listen, few understand - or hear; so why not taking advantage of their naivety?"
    Even if this is the more accurate case, I still wish you the best of luck, my friend. I have seen worse in my life; there was a time when Emagic sold an audio daw for Atari Falcon, where nobody could even RECORD a SINGLE track (at that precise moment, Steinberg's Falcon audio daw could successfuly deal with 16 tracks); yet they sold it for good money, bro. Yes, my money too. Who's to judge such things? Certainly not me :-)

    I desperately hope that there's no need to say that I have nothing against you personally, or against your approach, even when I expect to be flamed in the next 30 seconds. 29....28....27....

    P.S. if I can hear a single-cicle 300 Hz wave? Err...I can hear even ONE FRONT, Benno. You can hear one front too. Everyone can hear a front, and tell if it's sudden or slow. And in the case in discution: everyone can hear TWO fronts, no matter how close they are.

    and P.S.2 - although I said "long discution by private email", sorry to continue here:
    Benno, man...ok, you quantize the thing down, no matter where - in the host already, or in the plug. This matter of starting point...this is so small in comparison to what comes afterwards...you quantize the timestamp or not, you add the offset to the buffer start, and you're DONE with it, forever. What comes after is like the worldwide hurricane caused by the flip of a butterfly's wings in Pekin ( which was the dealing with the starting point). I know I don´t need to explain to you what comes after etablishing the start point. You will have to go thru all that hurricane, sooner or later. So you gain only half a millisecond of delay, just to catch your breath. The processing peak will follow anyway, sooner or later. So why quantizing?????? Do you really hope that that half a millisecond of "not doing it yet" vould make any difference in performance??

    Late edit: it´s half an hour now, and nobody flames me. There's definitely something wrong with this forum. Well, <sigh> :-)

Go Back to forum


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts