Site updated for MoP - Censusplus v6.0 download link

1974ER · Post by **1974ER** » Fri Nov 30, 2012 2:18 pm

This process is out of my hands...

Nov 28, 12 00:52:54 0 0 E

also failed...

I am pretty sure Rollie can't babysit the queue, so I'll just have to live with whatever manages to make it through successfully. *deep sigh* All files went out successfully as far as I could tell at the time, so I have zero idea on why some of them are failing.

I'll just start feeding in the rest in a little bit.

1974ER · Post by **1974ER** » Fri Nov 30, 2012 4:45 pm

Strange... now that one registers as success and the next one as faulty... *scratches head* Strange beyond my limited understanding... Oh well, just to let everyone know that I have now uploaded everything I had ready. Which currently translates into about five and half pages of not yet processed files.

1974ER · Post by **1974ER** » Sat Dec 01, 2012 12:08 am

A few hours of sleep later...

Nov 28, 12 00:58:13 0 0 E
Nov 28, 12 00:56:32 1936 9676 Y
Nov 28, 12 00:54:30 0 0 E
Nov 28, 12 00:52:54 2067 9919 Y
Nov 28, 12 00:51:02 0 0 E
Nov 28, 12 00:49:43 884 5027 Y
Nov 28, 12 00:47:27 0 0 E
Nov 28, 12 00:45:49 1398 6696 Y

Like a Swiss cheese and no idea why... *deep sigh*

Wynta · Post by **Wynta** » Sun Dec 02, 2012 3:27 am

hi,
any chance of linking to the latest addon, to run censuses with,at the top of this thread? I know Ive seen it somewhere,but am having trouble finding it (assuming 5.02 is not the current one, since dated 2010?!)
thanks

Balgair · Post by **Balgair** » Sun Dec 02, 2012 6:15 am

Good idea, will do that! http://www.curse.com/addons/wow/census-plus is the latest version

1974ER · Post by **1974ER** » Sun Dec 02, 2012 6:41 am

Many hours later... a few successes, then more Swiss cheese...

Nov 28, 12 01:26:03 0 0 E
Nov 28, 12 01:24:43 1011 5725 Y
Nov 28, 12 01:23:20 0 0 E
Nov 28, 12 01:21:52 1357 6832 Y
Nov 28, 12 01:20:28 0 0 E

*sigh*

Wynta · Post by **Wynta** » Sun Dec 02, 2012 1:02 pm

Brillig, thanks

Balgair wrote:Good idea, will do that! http://www.curse.com/addons/wow/census-plus is the latest version

1974ER · Post by **1974ER** » Sun Dec 02, 2012 5:26 pm

Even more hours later... the queue has slown down to a permanent crawl and the Swiss cheese just keeps piling up...

Nov 28, 12 01:32:00 0 0 E
Nov 28, 12 01:30:36 874 4834 Y
Nov 28, 12 01:28:55 0 0 E
Nov 28, 12 01:27:40 845 4661 Y

esiemiat · Post by **esiemiat** » Sun Dec 02, 2012 9:32 pm

Just curious, how many more from that batch of uploads are left?

Padanfain · Post by **Padanfain** » Sun Dec 02, 2012 11:35 pm

1974ER's files to be processed.... 27 uploaded on the 28th then 2 days later
52 files, 2 files on dec 1st and 1 on the 2nd. Give it about 2 1/2 to 3 weeks
to clear up and things will be back to 'normal'. Sucks that he has so many
errors not only in server lag time but the loss of census credit to his
account.

1974ER · Post by **1974ER** » Mon Dec 03, 2012 12:28 am

I most seriously hope it won't take that long... Two months of enforced break was apparently too much for the queue engine, too. :/ Strange thing about the errors is that they seem to mostly alternate with successes. It's unfortunate that Rollie is so busy, it would be nice to have some sort of input from him. It has been a very hard year for him, so I hope things start going better as Christmas is approaching fast.

Also, I would be happy with much smaller figures if it meant all files process correctly. All the failures that slip through mean some data is permanently lost. And judging by the numbers in the successes, in some cases the data loss is rather significant.

Seems the queue is struggling again...

Nov 28, 12 01:59:57 0 0 E

*sigh*

Padanfain · Post by **Padanfain** » Mon Dec 03, 2012 12:49 am

The first 43 files, taking in failure rate and time per failure, has taken about
6 days and you have over 80 queued so, if that continues, 11 or more days,
2 weeks tops plus our files

, is not out of range. I hope for a smaller
failure rate not only for time but your count. As for Rollie, absolutely family
comes first and I wish him well and better days to come. I'm glad he could
keep such a creation as this site running as a hobby/interest considering life
as it happens.

Balgair · Post by **Balgair** » Mon Dec 03, 2012 3:31 am

I've mailed Rollie to let him know about your files ER, maybe if he has a spare minute he can take a look and see what's wrong with them. Can't imagine what's wrong myself, since I'm sure you're doing nothing different to usual!

1974ER · Post by **1974ER** » Mon Dec 03, 2012 6:41 am

Padanfain, as far as I am aware, due to certain technical quirks most other people who stockpiled had already cleared their piles by the time I began... So what remains of November... is mostly me.

Thank you, Balgair. As I noted earlier, I have no idea why the files go awry... and the success, fail, success, fail, success, fail pattern that is evident multiple times suggests to me that the problem might actually be in the queue, not in the files themselves. It's also possible that I simply put in too many files in too short time period... but there have been other such times when it has worked... and the pattern is not fully systematic, which renders it even weirder.

I am also assuming that if there were no failures, the queue would move significantly faster, because it seems to jump forward when multiple Y statuses are achieved and then halts to more or less complete stop upon hitting E.

Post by **bringoutyourdead** » Mon Dec 03, 2012 12:30 pm

I expect there isn't anything wrong with the files.. instead I expect there is a timing issue in Rollie's processing that blip in the data being read in and stored.. where either the PHP is getting or feeding faulty data or the database engine is getting confused and saying that it has seen data it hasn't really or isn't seeing the data it has been given.

When the process errors out there is time for the system to settle down.. and the next file processes fine. It is possible that rapid change of realm keys with the random cross-realm entries might be giving fits to the buffering (cache problems, memory fragmentation etc.) in the database. This is something that can be fixed by tweaking the configuration parameters but you can't do the tweaking until you see that limits you are hitting during processing.

If you remember Rollie wanted to upgrade his database hardware we don't know if he was able to do that before he opened the flood gates.

Which is another way of saying that only Rollie can know what is causing the problem.. and he won't know until he can take the time to analyze the database logs.

Balgair · Post by **Balgair** » Tue Dec 04, 2012 2:40 am

Rollie was looking at it earlier, don't know if he reached a final conclusion yet but he did mention that the file he was looking at had over 56k characters in it and took over an hour to process, so maybe size could be an issue there....

1974ER · Post by **1974ER** » Tue Dec 04, 2012 10:49 am

Admittably size could be an issue... but it's unlikely to be the only reason, as files in excess of 80000 characters were pretty much routine for me before site was updated. Also, with a quick glance... the previous time I had a file failure prior to it... was on May 15th... more looks... next failure... Sep 10th, 2011!

Overall, both the queue and my files have been pretty reliable, so I have a very hard time trying to understand why they are now encountering multiple errors in the space of just few days when preceding over a year contains only two, extremely far apart from each other, too.

Balgair · Post by **Balgair** » Tue Dec 04, 2012 11:00 am

Yeah, mine tend to be something similar, at least I've pruned chunks of 40k before now and never had trouble. Wondering if the database is struggling in another way, such that it can no longer handle such big files. If so, that's a headache, but maybe there's some other explanation, we'll see when Rollie investigates further I guess.

1974ER · Post by **1974ER** » Tue Dec 04, 2012 4:02 pm

Unfortunately, it seems the situation is getting worse as I am now seeing multiple failures in row...

Nov 28, 12 02:12:37 0 0 E
Nov 28, 12 02:11:14 0 0 E
Nov 28, 12 02:05:16 0 0 E
Nov 28, 12 02:03:55 2871 9726 Y
Nov 28, 12 02:02:42 0 0 E
Nov 28, 12 02:01:15 1956 8662 Y
Nov 28, 12 01:59:57 0 0 E

Padanfain · Post by **Padanfain** » Tue Dec 04, 2012 9:36 pm

Woot! This is lookin better 1974ER

Nov 28, 12 02:27:05 2631 8195 Y
Nov 28, 12 02:24:41 576 2035 Y
Nov 28, 12 02:19:45 1053 4294 Y
Nov 28, 12 02:16:40 1953 4991 Y
Nov 28, 12 02:15:41 409 2084 Y
Nov 28, 12 02:14:35 1929 6379 Y
Nov 28, 12 02:13:32 1964 6528 Y