Who needs to find bugs when you can make them.. sigh

Post by **bringoutyourdead** » Mon Aug 15, 2016 11:47 pm

the upload processor during the servers 23rd hour for the day apparently failed to trigger.
It may auto reset sometime during the night.. or Metalbeast may need to fix what I may have broken...

I found what should have been the issue with Demon Hunters not getting recorded cleanly... and added them to the process performance section of the Class processing file.

Either Metalbeast will check in the morning and find everything is working as intended.. or he will kickstart the process.

Just means the queue will back up a little bit .. and instead of processing in one or two minutes.. it might take 10 minutes to clear the queue..
not a big deal.

Post by **bringoutyourdead** » Tue Aug 16, 2016 1:29 pm

found my typo.. fixed it and now waiting on automated job processor to fire off the upload processing.

superxdude · Post by **superxdude** » Tue Aug 16, 2016 8:24 pm

last five uploads i have still at 0...assuming something went bonk in the night?

Post by **bringoutyourdead** » Tue Aug 16, 2016 10:01 pm

Yea me.
And in the process, I discovered an system level administration issue, that Metalbeast will need to fix.

He and I had a 10- part email conversation, that stopped when I noted that I no had the needed remote sysop access when the the last server replacement happened.

(or at least I think so, as it is an access method that I keep away from when at all possible.. after all I don't want to be responsible for wiping out not only our site but dozens of others on the same hardware!.. Let Metalbeast do that if it happens, not me!)

On the other hand.. don't worry .. the processing of files will happen.... and the queue will clear far faster then what Rollie could afford.
Keep on uploading your updates, there is plenty of disk space on the server.. no need to hold the files on your local machine.

superxdude · Post by **superxdude** » Wed Aug 17, 2016 12:49 am

Thanks BOYD!

Post by **bringoutyourdead** » Wed Aug 17, 2016 4:32 pm

Processing the queue... but not fixing the problem .. reverted to previous processing file.

The update was suppose to fix localization of Demon Hunter class names for processing....
but this is a issue that actually had a deeper problem.

The initial Class processing file change for Demon Hunter back on 7/16, broke the activity tracking for the class Monk

I have been comparing my updates against a broken file!..

I expect once we find out why Monk activity tracking stopped the rest will fall into place.

Post by **bringoutyourdead** » Wed Aug 17, 2016 4:33 pm

Oh and in less then 10 minutes 95 uploads were processed.
actually probably much less then 10 minutes since I was here in the forums and not looking at the stats.

Post by **bringoutyourdead** » Wed Aug 17, 2016 10:55 pm

Much but not all of the upload processing has been fixed.

the Class activity data is now fully being handled.. so as you will see, your new character find numbers are reduced back to the real numbers.
The big and always rising numbers you have been seeing are bogus.. mostly
in that they were the previous new count + the latest new count.
To figure the real new count of any upload just subtract the previous upload number from the one of interest.

Demon Hunters are still having an issue.. that at this time is hidden from my understanding.. but I expect from my research Metalbeast will go .. oh yea.. that needs updated also.

We have accumulated a bucket full of Demon Hunters.. that are obviously DH.. but not marked so in the main character database.. which is one reason no one can search or see them at this time.

Metalbeast may decide to do a mass update of the specific field... base upon data I have provided him. if so he may decide to lock the database for a period of time... hour or two as more then 1.3 million Demon Hunters need to be marked as such.
Or he may decide to do it in smaller chunks which will have smaller impact, but take longer to finish the whole update process.

Post by **bringoutyourdead** » Thu Aug 18, 2016 2:59 pm

This hours run of the database upload process correctly loaded Demon Hunters to all the databases that need the data.
There was a 'Oh yea.. forgot about that limiter we have on one of the table elements.. need to document that for the next time.'

Older DH records are being fixed on a realm by realm basis.. in order to limit impact on users.. we have 1.3 million Demon Hunter character class IDs to fix.

display pages are in the works.. I have a sample that may or may not go live... Metalbeast has also been working on that.. He wants to update the page so it doesn't look like something from 2005.

Post by **bringoutyourdead** » Thu Aug 18, 2016 5:31 pm

All the newly created Night Elf and Blood Elf characters that showed up at or above level 98 have been tagged correctly as the Demon Hunters that they are.

The US realm Emerald Dreams gained the largest batch at over 47 thousand
a hand full of realms were in the 20 thousands
and too large a number were at Zero, either we haven't run a census on the realm.. or there is such a small population that we never caught anyone playing a DH on the realm

FuxieDK · Post by **FuxieDK** » Thu Aug 18, 2016 9:45 pm

On my server, there was (in my UI, which had data for 30 days) 4000 Belf DHs after 3 days..
The most played class was (ofc) Paladin, which is also dominated by Belfs, by 5000..

You cannot go ANYWHERE in Orgrimmar, without tripping over Belf scum...

And this was after THREE DAYS... And I don't play very often ATM..
Wonder how many DHs Balgair (who's on the same server) have...

Post by **bringoutyourdead** » Thu Aug 18, 2016 10:49 pm

Well due to your heads up, we now know the counts are inflated.

FuxieDK · Post by **FuxieDK** » Fri Aug 19, 2016 3:39 am

Inflated maybe.. But I doubt it's artificially inflated..

Everyone and their grandmother seems to have created a DH (myself included when though I hate hate hate Elves, and swore I wouldn't play one)...

Balgair · Post by **Balgair** » Fri Aug 19, 2016 8:15 am

FuxieDK wrote:On my server, there was (in my UI, which had data for 30 days) 4000 Belf DHs after 3 days..
The most played class was (ofc) Paladin, which is also dominated by Belfs, by 5000..

You cannot go ANYWHERE in Orgrimmar, without tripping over Belf scum...

And this was after THREE DAYS... And I don't play very often ATM..
Wonder how many DHs Balgair (who's on the same server) have...

Haven't played much this last week or so due to watching loads of the Olympics but I've got 2678 demon hunters in my data, 8% of all characters, between them and belf pallies belfs make 39% of all races. Small downside to DH getting given to what was already the most popular race, at least on Alliance humans tended to (narrowly) beat nelfs in popularity so DHs didn't make it even more unbalanced!

I'm actually slightly surprised DH isn't even higher in my data, but then I guess everyone's playing their older characters as well to get invasion loot, so other classes than DH are being seen too. On launch day over 30% online were DH, and when I went over to our super low-pop alliance side it was 50% lol.

Post by **bringoutyourdead** » Fri Aug 19, 2016 10:15 am

The database shows 1.3 million DH.. but thanks to FuxieDX, I now know that is wrong.

Assuming FuxieDX is the worse case example

with 10 entries for the same DH character. The real censused count may be 130K.

I expect the number will be higher then 130K as a number of realms haven't been censused 10 times in the same faction since the 9th.

The duplicate records will need to remain in the database for a little while as we will need to write a procedure to remove the duplicate records not only in the main table but all the tables that feed off of the main character data.

Later today I will verify that the processing fix is holding and that all the new records are clean, as I expect them to be.

Post by **bringoutyourdead** » Fri Aug 19, 2016 9:42 pm

The Good news is that there were no new bad DH leaks into the database.
The bad news... hmm... none here.

the Good news with the database issues calming down... I created an update to the census.php data display with both the Demon Hunters and the level increase to 110.
The Bad new.. well, it isn't really what Metalbeast wants... but considering Real Life issues.. it may be what he has to accept.. waiting on his decision.

while by raw counts US - Emerald Dream had over 47 thousand Demon Hunters..
the new Census page will show (assume Metalbeast says go with it), 13406 Demon Hunters.
And the real count seems to be 308,199 Demon Hunters censused at this time, down from the incorrect 1.3 million.

The good news is that the duplicates will get cleared from the database next week..
oh wait that is the bad news...
At first I was thinking we would keep only the most current record.. but on further thought, I expect we will keep the first and most DH will revert back to level98 until the next census after the clear out.
If you really wanted to track your DH going from level 98, 99, 100... get use to disappointment.

Speaking of disappointment, most if not all of you will see your inflated New character count burst back to the real numbers at some time in the near future. tbd

superxdude · Post by **superxdude** » Sat Aug 27, 2016 7:18 pm

parser bugged out again?

Post by **bringoutyourdead** » Sat Aug 27, 2016 7:43 pm

Nope.. doing the Demon Hunter duplicate clear out..
Expect to re-enable upload processing tomorrow afternoon.. may be sooner.. I am about 1/3 of the way through the very tedious process.

superxdude · Post by **superxdude** » Sat Aug 27, 2016 8:55 pm

gotcha...rock on bro

Balgair · Post by **Balgair** » Mon Aug 29, 2016 1:43 pm

Speaking of duplicates, tracking down an alt that got purged in one of the clearouts located a 2nd copy of it, must be quite old as I was definitely 90 by the end of MoP.

http://www.warcraftrealms.com/charsheet/187270195 - lvl 85 last seen 1969... clearly a bugged one! In fact it doesn't even show the name/realm on the page, but that was a link I got when I searched for "Ruire" and selected the two on EU-Draenor.

http://www.warcraftrealms.com/charsheet/172678960 - the real one

(I know the purges are necessary but it really sucks to lose my years of history, I created that character back in Wrath when I first got together with my other half and we were levelling alts together - well, no way to know THAT from the history any more, it could just as easily be a lvl 100 boost I used a few days back...

)