I've got good news... and I've got ...

Post by **Rollie** » Wed Jan 31, 2007 2:36 pm

More good news!

The good news is that the update process is now back up and running. The more good news is that whereas the last db was processing data files around 30 per hour, this one is processing around 180 per hour. Which is a very good thing because it's behind by about 6000 data submissions =x

NiennaMiriel · Post by **NiennaMiriel** » Wed Jan 31, 2007 2:37 pm

AWESOME!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Good things come to those who wait!

Elmaless · Post by **Elmaless** » Wed Jan 31, 2007 3:10 pm

Rollie wrote:More good news!

The good news is that the update process is now back up and running. The more good news is that whereas the last db was processing data files around 30 per hour, this one is processing around 180 per hour. Which is a very good thing because it's behind by about 6000 data submissions =x

Geesh, 6000? That's incredible XD guess a site can be too popular

BWDemor · Post by **BWDemor** » Wed Jan 31, 2007 5:27 pm

Sweet and very good news. Guess that means we need to submit more and see if we can bring the server down?

J/K

BW

oiseaux · Post by **oiseaux** » Wed Jan 31, 2007 5:53 pm

BWDemor wrote:Sweet and very good news. Guess that means we need to submit more and see if we can bring the server down?

J/K

BW

haha Nice one.

Rollie that is great news. I am glad that things are working out with the database.

Zolimox · Post by **Zolimox** » Wed Jan 31, 2007 6:12 pm

From a purely morbid technological curiosity, may i ask what you did to the DB to increase performance by 6x?

(Feel free to get technical)

Post by **Rollie** » Wed Jan 31, 2007 6:16 pm

Well, basically I just moved the database to a much beefier machine.

Went from a dual AMD Athlon 2600 MP with 2GB of RAM and dual 80GB IDE HD to a dual Opteron 265 with 8GB of RAM and dual 160GB SATA HDs

The biggest improvement is probably the hard drives, but the increased ram is also very helpful.

This still isn't the optimal setup and over the next few months there will likely be another move when I get fully set up on a raid 5 machine with full replication.

Zolimox · Post by **Zolimox** » Wed Jan 31, 2007 6:20 pm

Awesome, sounds like you had a fairly linear increase in performance.

Dremvek · Post by **Dremvek** » Wed Jan 31, 2007 11:23 pm

I'm sure you're aware of this, but RAID 5 is horrible performance for writes, though great performance for reads. If the submissions are the slow point, you might want to consider raid 10 or some other raid to elminate disks as bottleneck.

WyriHaximus · Post by **WyriHaximus** » Thu Feb 01, 2007 2:06 am

Rollie wrote:Well, basically I just moved the database to a much beefier machine.

Went from a dual AMD Athlon 2600 MP with 2GB of RAM and dual 80GB IDE HD to a dual Opteron 265 with 8GB of RAM and dual 160GB SATA HDs

The biggest improvement is probably the hard drives, but the increased ram is also very helpful.

This still isn't the optimal setup and over the next few months there will likely be another move when I get fully set up on a raid 5 machine with full replication.

Wow thats a big improvement. Especialy the memory should speed things up

!

xpolockx · Post by **xpolockx** » Thu Feb 01, 2007 2:44 am

I'm excited about this!

WyriHaximus · Post by **WyriHaximus** » Thu Feb 01, 2007 3:59 am

Yeah me to somehow I allways start thinking of creating heavy data systems for tracking and gathering data from wow to a website when I see such specs

!

Zimeron · Post by **Zimeron** » Thu Feb 01, 2007 8:56 am

on that subject, what ever happened to the thing that showed the last file processed upload time? So you could "estimate" how far behind you were :-p

DM. · Post by **DM.** » Thu Feb 01, 2007 8:58 am

http://www.warcraftrealms.com/dbstats.php

Post by **Rollie** » Thu Feb 01, 2007 9:34 am

I estimate we'll be caught up by the end of the weekend, if not sooner.

Balgair · Post by **Balgair** » Thu Feb 01, 2007 9:45 am

Excellent! Looking forward to being up to date again

BWDemor · Post by **BWDemor** » Thu Feb 01, 2007 1:16 pm

Nice specs on the machine and also looking forward to the catch up. Then the question I have, since you are processing faster, would you be thinking of increaseing the file size? or maybe make a small unmentioned buffer incase people go over the file size?

Think i went over on my last Submit since the process time has passed it and it was never processed.

Thanks
BW

Babs · Post by **Babs** » Thu Feb 01, 2007 3:28 pm

oeh yes, its cleary catching up now =)

Alanthus · Post by **Alanthus** » Thu Feb 01, 2007 5:28 pm

RAM would make the biggest difference, at least up to 4GB depending on platform, provided tuning parameters for the database are set to utilize the RAM without messing with the OS space. HDD speed would be second, especially bigger HDD cache, a RAID5 array should help provided the logic is handled by the card and not offloaded to the cpu, if the RAID card has a decent cache that helps a lot too. (and yeah, it all boils down to handling as much as possible in RAM and dumping it as sequentially as possible on the HDD, new hybrid HDD's should really boost performance for a lot of stuff)

Nice to see the increase in performance, I'll likely not contribute much now that I'm employed but I'll still lurk and play on weekends at least

Post by **Rollie** » Thu Feb 01, 2007 6:46 pm

Work is overrated!

Getting paid is not =x

WarcraftRealms.com

I've got good news... and I've got ...

I've got good news... and I've got ...

Re: I've got good news... and I've got ...