[svlug] Linux + MySQL w/ very large data sets

[Drew Bertola]

On Wed, Jun 12, 2002 at 03:53:15PM -0700, Jeremy Zawodny wrote:
> On Wed, Jun 12, 2002 at 03:00:22PM -0700, Drew Bertola wrote:
> >
> > Can anyone tell me what the best architecture / OS support is
> > suggested for handling data with possible table sizes in the 100s of
> > GB range?  This would be under MySQL (though I'd consider other DBs)
> > and, other than monthly updates, only SELECT transactions need be
> > considered.
> 
> As long as the OS in question can handle files that big, you shouldn't
> have much trouble.  If your query volume isn't substantial, you'll be
> looking at a ton of disk space (obviously) and as much ram as is
> feasible for the box.  Giving MySQL a large key buffer will be
> essential.
> 
> CPU isn't much of an issue--this is an I/O bound problem.  So having
> some form of RAID and fast disks will certainly help.

The queries will be frequent and somewhat complex, so I'm thinking a
quad Xeon.  I agree that disk I/O is the biggest issue.  I guess I'll
need about 1TB RAID5 to start.  Any suggestions on the RAID hardware?

Would it be better to employ a 64-bit machine?

Performance + scalability is more important than cost in this case.

-- 
+---------------------- T h e C o o p . n e t ----------------------+
| Drew Bertola         Hosting - Colocation      Tel:  408-738-8337 |
| TheCoop.net      Programming - Administration  Mob:  408-887-0426 |
| drew at thecoop.net    Open Source Specialists    http://thecoop.net |
+-------------------------------------------------------------------+