[svlug] Performance and memory

[Karsten M. Self]

on Fri, Apr 29, 2005 at 11:45:06PM -0700, Jeffrey Siegal (jbs at quiotix.com) wrote:
> 
> On Apr 29, 2005, at 18:34, Karsten M. Self wrote:
> 
> >> Another issue with swap is the speed, type, and layout of your
> >> harddrives.  If your swapping to the same IDE drive it's trying to 
> >> read
> >> the incoming program/data from, you'll bottleneck as IDE only does 1
> >> transaction per controller per interval.
> >
> > Yep.  RAID will buy you _massive_ benefits here (the controller handles
> > the i/o, the CPU doesn't block), for good values of RAID controller.
> > Dittos SCSI.
> 
> Yes and no.  A better storage subsystem will improve your swapping
> performance, but that's rearranging the proverbial deck chairs.  In
> reality you are *far* better off in almost all cases just getting
> enough memory that you aren't swapping so much.
> 
> Faster disk subsystems (RAID, SCSI, etc.) are a good solution when you
> need persistent storage (mostly databases).  If you need more memory,
> get more memory.

No argument.

But the impact of disk, particularly slow or contended disk, is pretty
highly underappreciated.

Two case studies:

 1. A consultancy I worked for in the mid 1990s had, by 2003, the same
    HP 9000K<mumble> midrange HPUX server bought new in 1997.  And some
    other Unix kit.  And a bunch of x86 legacy MS Windows systems.
    Heavy lifting app was SAS.  x86 ran circles around the RISC kit for
    CPU intensive stuff.  
    
    But for raw I/O, it was the 6 year old HP that had the best
    performance.  Nothing beats fast disk, good channels, and IIRC, a
    sane RAID setup.

 2. About the same time, I was running SAS, also on legacy MS Windows,
    on a then-current Dell box, 2.4 GHz, 384 MiB or so, 7200 RPM ATA
    disk.  A fire-sale 3u server was aquired, and for shits'n'grins, SAS
    loaded.  500 MHz, 1 GiB RAM, dual SCSI disks, striped for the
    experiment.

    Straight read/write ops were 2x faster than the "faster" Dell.

    A more complex analysis routine involving multiple subqueries and
    table creations and reads, was 10x faster on the SCSI striped
    system.


My own system here (based on laptop 4200 RPM disk, tuned w/ hdparm to
a blazing <coff> 6.13 MiB/s, gets really heavily contended in some
processing and cron.daily tasks.  I watch system load climg at these
times.  1 GiB RAM really doesn't do much, nor does a 1.7 GHz processor,
not when a single-spindle, one-task-at-a-time disk gets in the way.


The real key is:  Know your system and its bottlenecks.  Apply remedies
accordingly.
  


Peace.

-- 
Karsten M. Self <kmself at ix.netcom.com>        http://kmself.home.netcom.com/
 What Part of "Gestalt" don't you understand?
    Free Software Primer -- concepts you need to understand
    http://twiki.iwethey.org/Main/FreeSoftwarePrimer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.svlug.org/archives/svlug/attachments/20050430/a008feee/attachment.bin