• Register
  • Help
Results 1 to 7 of 7

Topic: Disk Cache size important for streaming samples?

  1. #1

    Disk Cache size important for streaming samples?

    I read a post on the VSL boards (It was on accident, I swear!! ) stating that disk cache sizes are not important for streaming samples, for the file size that needs to be stored in cache is extremely small for samples compared to other file types. Thus having a larger cache lowers overall performance, for it takes longer to dump and reload the information in a large cache (e.g. 2MB vs 8MB). Is this indeed true? It seems to make logical sense to me (though I might not have explained it well here )

    The reason I ask is that I'm in the market for a FW800/400 drive to use mainly as a way to interface between my old ibook (FW400) and new G5 (FW800). I do not plan on using this drive exclusively for samples on the G5, though it would be nice to use it (with limitation) to stream samples in a portable setting on the ibook. Of course, the 8MB cache drives cost more (and make me buy more GB of space) then the 2MB drives. Will opting for a smaller, 160GB 8MB cache FW drive hurt me in any way?


  2. #2

    Re: Disk Cache size important for streaming samples?


    I don't if this will help you but I know kontakt has an option that can disable disk cache. It is usefull when you select a large voice buffer size.


  3. #3

    Re: Disk Cache size important for streaming samples?

    Hi JT3_Jon what those people said is completely bull[nss][nss][nss][nss]

    Having a larger cache on the phyisical disk can NEVER hurt.

    I'll give you a few technical infos:

    disk input/output works as following:

    - rotating physical disk platters with disk head that moves around to read/write from/to the disk

    - RAM which resides on the disk itself which we call "hardware disk cache", these days usually 2-8MByte

    operating system which acts as an intermediary between the application itself and the hardware (the disk controller attached to the disk).

    - a dynamic amount of the PC RAM that the operating system uses to cache disk data that is frequently accessed, we call this "OS disk cache"

    - the application itself which issues read, write commands to the disk (through the operating system).

    Now assume our sampling app wants to read some data from disk.
    It tells the OS (operating system) "give me that chunk of data in file XYZ at position P".

    First the OS will look in its OS disk cache and ask "was this cunk of data recently read and stored in the cache" ?
    If yes then a disk access is completely avoided because the data is fetched from this cache thus making the read much much faster.
    Of course the size of OS disk cache is limited so it will help only in case of data blocks that are accessed frequently. If you read gigabytes worth of data the cache will not help really much since it will get overwritten often.

    Now assume that the datablock requested by the application does not reside in the OS disk cache.
    In that case the OS must tell the disk controller it wants to read the data from the physical disk.
    The read request is passed from the disk controller to the physical disk (which has some kind of controller on it too).

    Before performing the actual disk access (moving the disk head to the right position, waiting that the rotating platter causes the requested data sectors passing under the head), it checks in it's own (relatively small) "hardware disk cache" if the data was recently read.
    If yes then a disk access is avoided and even if the OS thought it had read the data from the physical disk it was read from the RAM cache which installed in the disk.
    The OS does not care about this, to it the operation just took less time.

    summing up: does it matter if a disk has a larger hardware cache ?
    It depends from application to application, in some cases you can feel the speedup quite a bit (if you read/write lots of small files), in some cases the benefits are probably a bit lower, when reading large chunks of data like large samples.
    But one is certain: a larger hardware disk cache never hurts.
    And usually larger disks come with larger hardware caches too so you won't have much choice anyway (to choose very large disk with a very small hardware cache).

    JT3_Jon, I hope that with the above explanation I eliminated your concerns
    just post the link to my response to the VSL forum where people made those misleading statements, they are free to prove me wrong

    the disk cache disabling option in Kontakt refers to the
    "OS disk cache".

    And indeed Windows has a suboptimal disk caching algoritm
    which often gets in the way when you stream large files from disk.
    In my benchmarks I made both cached and non cached disk reads tests and if you use the OS disk cache the performance sucks when dealing with very large files.
    For example my Laptop achieves around 16-17MByte/sec using non cached disk reads (disabling the OS disk cache) and only
    around 7MB/sec when the OS disk cache is not disabled.
    This is related to the fact that Windows as soon as your files don't fit in the OS disk cache anymore it begins to let the disk seek around like mad, throwing the read throughput in the toilet.

    This is a shame ! And once again proves that Windows delivers really bad performance and application writers need to resort to all sort of dirty tricks to squeeze out the most of the OS. (like implementing their own internal disk caching etc)

    Just to give you some comparision data (on Linux, and OS X too because both are UNIX based) the OS disk cache works much much better and cached reads are in average faster than non cached reads.

    For example on when playing the PMI Steinway piano on a 512MB RAM machine with LinuxSampler when I play a classical midi piece the first time the disk is quite busy since it needs to stream lots of simultaeous voices from disk, but when I play
    the midi file a second time, the disk is barely accessed because the OS cached the last used parts of the file in RAM so you often need to go over 50-60 voices of polyphony to see the disk light flashing. This all without LinuxSampler needing to do its own caching in the application itself.

    This is what I call an efficient OS


  4. #4

    Re: Disk Cache size important for streaming samples?

    UPDATE: I've read the post on the VSL forum (from cm), it's totally misleading and I'm sorry to say it he got the understanding of the functionof disk caching totally wrong.
    OS disk caching is performed on a per page (usually 4096bytes) so his argument of that the whole file must fit in cache does no make sense, besides that he seems unaware that there are two kinds of caches, the hardware and the OS disk cache. Regarding slow RAID controllers that's due to the inefficiency of the controllers themselves not because of caching.
    Having written several streaming engines, the first one 13 years ago (on the Commodore Amiga when the PC was barely able to emit a boring beep) I think I know what I'm talking about

    About the journaling issue cm is right: using non journaled filesystems, it might increase performance a bit.


  5. #5

    Smile Re: Disk Cache size important for streaming samples?


    I know you've responded to my other silly hard drive posts and seem to really know your stuff, and thus wanted to check with you (and other users) if what was said on the VSL boards was indeed correct. I truly appreciate your help! I know knowledge on these topics will come in time (and use) but I dont want to have to make too many mistakes setting up my system before I get one that works. Us college students just dont have the money to afford it!

    Thanks again sbenno! Would you mind it if I PM'ed you before I make any drive purchases for my G5 sample setup, just to make sure everything seems to be ok? (no hard feelings if your simply too busy)


  6. #6

    Re: Disk Cache size important for streaming samples?

    For GigaStudio, running on Win98, you should set the Max and MinDiskCachSize parameters really LOW, in the order of 16 Mb or 32 Mb. This has been discussed and proved in the last 3 years on these forums AD NAUSEUM. You will have more Ram available for loading prefetch buffers and thus more instruments. So, this theoretical approach does not cover all practical situations.

  7. #7

    Re: Disk Cache size important for streaming samples?

    thanks JT_Jon3 , you are welcome

    No problem about sending me PM, but I think since this is a community forum and other people might ask the same or similar questions it's probably better if we leave the discussion public so that anyone can profit from it, plus since search engines index forums the information becomes relatively easy to locate.
    I think anyone of us can confirm how much we learned by reading forums or casually stumbling over mailing list archives or forums posts that search engines returns.

    Your posts were not silly, just normal questions. You cannot expect that everyone in the world is an expert in hardware, hard disks, operating systems etc. People here are usually musicians wanting to make music using computers, they see it a tool, like most of us don't care what's going on under the hood of our cars.

    BTW: on those VSL posts it seems that one guy said he had performance problems with an external firewire disk and that internal SATA performed better, which just confirms my thesis
    Of course if you have a good firewire disk (controller + fast HD) performance would be ok, but if I had to buy a new G5 for audio I'd buy it with 2 internal SATA drives (7200rpm and up), no surprises attached, and usually cheaper than external disks.
    (since I don't own a Mac I don't know how many SATA connectors it has, I guess either 2 or 4).

    Peter: yes, I think under windows the OS disk cache gets conflicts a bit with the memory allocation by applications.

    On unix-like OSes (I think OS X too) it usually works the following way: the OS disk cache tries to snap up all the free memory to cache blocks of files that were read recently BUT if there is demand of large amounts of memory by user applications (like a sampler app) then the memory from the cache is freed transparently. This is an ideal situation because if you have lots of spare memory the OS disk cache caches the segments of samples that were recently played which helps to lighten the load on the disk. On the contrary if you use up all the available memory the OS disk cache is reduced to a minimum and the system still works well except that the load on the disk is higher.


Go Back to forum


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts