Site updated for MoP - Censusplus v6.0 download link

Balgair · Post by **Balgair** » Tue Jan 08, 2013 11:26 am

Getting worse for me too, hoping Rollie's changes help and he can get them in fairly quickly...

Dec 16, 12 20:18:44 0 0 E
Dec 15, 12 17:01:40 0 0 E
Dec 15, 12 09:46:31 434 1850 Y
Dec 14, 12 22:01:52 0 0 E
Dec 14, 12 17:04:57 343 1404 Y

Padanfain · Post by **Padanfain** » Tue Jan 08, 2013 11:30 am

Me three. That time of day submission theory is looking pretty good Balgair.

Dec 17, 12 21:19:41 0 0 N
Dec 17, 12 01:05:56 0 0 E
Dec 17, 12 00:17:40 0 0 E
Dec 16, 12 22:40:19 0 0 E
Dec 16, 12 21:30:32 0 0 E
Dec 16, 12 20:04:04 0 0 E
Dec 16, 12 01:53:49 644 1838 Y
Dec 16, 12 01:04:36 0 0 E
Dec 15, 12 23:27:28 0 0 E
Dec 15, 12 22:13:48 0 0 E
Dec 15, 12 20:50:40 0 0 E
Dec 15, 12 01:40:50 603 1760 Y
Dec 15, 12 00:55:27 0 0 E
Dec 14, 12 23:18:46 0 0 E
Dec 14, 12 22:18:24 0 0 E
Dec 14, 12 21:02:57 0 0 E
Dec 14, 12 01:26:29 537 1583 Y
Dec 14, 12 00:42:27 0 0 E
Dec 13, 12 23:18:38 0 0 E
Dec 13, 12 22:20:11 0 0 E
Dec 13, 12 20:57:10 0 0 E
Dec 13, 12 00:23:25 0 0 E
Dec 12, 12 23:27:26 0 0 E
Dec 12, 12 21:32:09 0 0 E
Dec 12, 12 20:14:03 0 0 E
Dec 12, 12 18:56:45 553 2213 Y
Dec 12, 12 17:14:13 921 2278 Y

Post by **bringoutyourdead** » Tue Jan 08, 2013 2:03 pm

Fuxie are your E submissions also of the same approximate size of characters updated?
and are you pruning your files?
I'm wondering if your files have a lot of older data vs. new update data?

If your files are purged/deleted and you have only or mostly new update data.. all I can say is "hmmn.. I haven't a clue what could be the processing issue."
And that bugs me.. I tend to spend much of my professional time, diagnosing IT problems and getting them fixed.

The one thing I do know is that so far as of the 12/15 data.. I'm batting .000% with 6 failures. Each submission I'm doing is with /census serverprune processed before I start doing my census runs on 40 to 50 factions each submission.

The fact that I'm not showing nearly 100,000 update isn't what bothers me.. as much as the fact that with all these E submissions from all of us, the most wanted and server submission pages here on the site are of no value.. no one can know which realm/factions need census runs more then others. sigh

oh well it is better then my other project (I run Wow in background, switching to it when I here the Census finished bell), that project has a program that randomly crashes on memory clean up when the program tells windows(win7 & win8) it no longer needs blocks of allocated memory... the problem seems to be a complied dll file which no one seems to have the source to any more.. it was lost during a hardware crash.. who needs backups .. sigh
So I get to try to figure out everything that dll was suppose to do and recreate it again from scratch.
Sort of like putting a picture puzzle together.. where there isn't a picture... and you haven't a clue if you have all the pieces.

piratejenny · Post by **piratejenny** » Tue Jan 08, 2013 11:33 pm

Aw, me and my big mouth--now I have errors too

Dec 18, 12 01:55:26 0 0 N
Dec 16, 12 20:06:15 0 0 E
Dec 16, 12 01:52:22 111 482 Y
Dec 15, 12 21:02:31 0 0 E
Dec 14, 12 02:29:36 35 207 Y

I mostly haven't been pruning my files, since I know they're not that big (though I have pruned a few times lately).

FuxieDK · Post by **FuxieDK** » Wed Jan 09, 2013 12:09 am

@ bring....

Honestly, I can't remember, what I did, prune-wise, 3 weeks ago, but most likely, I used standard 30 days prune.
My files are NEVER over 4MB in size.. 3,5ish is the biggest files I have uploaded in MoP.

After the first failed file, a few days ago, I have started a prune-7 routine..

Djambo · Post by **Djambo** » Wed Jan 09, 2013 4:34 am

FuxieDK wrote:After the first failed file, a few days ago, I have started a prune-7 routine..

To be honest: That won't help you. I'm doing the 7 day prune for years now and two of my files error'd.

Balgair · Post by **Balgair** » Wed Jan 09, 2013 8:16 am

Yep, I've been doing prune 5 the last few weeks with all the problems (only even played on 2 servers so won't be that high anyway), hasn't helped me much.

I suppose on the plus side, the backlog is progressing a lot faster than it was... I just think it's doing that by not processing many of the files

FuxieDK · Post by **FuxieDK** » Wed Jan 09, 2013 11:21 am

Djambo wrote:
FuxieDK wrote:After the first failed file, a few days ago, I have started a prune-7 routine..
To be honest: That won't help you. I'm doing the 7 day prune for years now and two of my files error'd.

It's morelikely to be number of entries, rather than days.
However, you might do more censusing (and thus more entries) than me, and thus, it might work for me, but not you..

Strictly guessing, ofc...

Post by **bringoutyourdead** » Thu Jan 10, 2013 12:39 pm

Since I'm not interested in keeping a local history on the factions I census what I'm doing is - /census serverprune - which will prune ALL data except for the current realm you are on.

One of the side effects of that is that every time I do a census run all characters are new characters.. so what I see and the end of the run doesn't come close to what warcraftrealms.com sees.

So far from what has cleanly processed.. I see about 22000 characters... and WR sees 7000 - 9000 updates and about 1/2 of that as new.

1974ER · Post by **1974ER** » Fri Jan 11, 2013 11:36 am

It's starting to look like that there will not be a rollback to reprocess failed files and lost data will remain that way forever. No obvious news from Rollie and DB queue remains roughly 22 days behind schedule.

Fett · Post by **Fett** » Fri Jan 11, 2013 12:05 pm

If there's not already a song about how lost data is gone forever, there ought to be.

I had two fails in a row from late on December 19. If our observed pattern of fails coming in bunches holds true, the rest of you should be seeing fails around the same time as well. How about it? Is the pattern holding true?

Balgair · Post by **Balgair** » Fri Jan 11, 2013 12:28 pm

Dec 19, 12 18:41:19 0 0 E

Yep

Padanfain · Post by **Padanfain** » Fri Jan 11, 2013 7:18 pm

I think the time range is expanding.

Dec 19, 12 23:57:30 0 0 E
Dec 19, 12 22:09:15 0 0 E
Dec 19, 12 21:02:25 0 0 E
Dec 19, 12 19:49:10 0 0 E
Dec 19, 12 16:41:15 0 0 E
Dec 19, 12 15:44:57 0 0 E
Dec 19, 12 14:18:13 0 0 E
Dec 19, 12 13:37:21 0 0 E
Dec 19, 12 12:51:21 0 0 E
Dec 19, 12 00:21:13 0 0 E

Padanfain · Post by **Padanfain** » Fri Jan 11, 2013 9:17 pm

BTW five of the failures on the 19th were taken on a second computer with a
fresh install of WoW, the only addon installed and updated is CensusPlus via
curse client and a purge after each upload on both computers. I'm getting
the feeling the database just hates me.

PJEckenheim · Post by **PJEckenheim** » Sun Jan 13, 2013 2:32 pm

Padanfain wrote:BTW five of the failures on the 19th were taken on a second computer with a fresh install of WoW, the only addon installed and updated is CensusPlus via curse client and a purge after each upload on both computers. I'm getting the feeling the database just hates me.

Judging by the number of errors from pretty much everyone, I'm beginning to think it's not a problem with the uploads, but with the database itself.

I'm not into coding, but would it be possible that the database can't cope with the number of entries? After all, "Total character count: 98,373,879" and "Total guild count 6,580,801" does mean a really big lot of datasets...

Padanfain · Post by **Padanfain** » Tue Jan 15, 2013 5:48 pm

Dec 25, 12 23:57:52 1787 4425 Y
Dec 25, 12 22:23:20 3519 7105 Y
Dec 25, 12 20:10:30 1281 3168 Y
Dec 25, 12 19:08:30 932 3018 Y
Dec 25, 12 18:13:16 607 2147 Y
Dec 25, 12 17:57:00 1343 4110 Y
Dec 25, 12 17:40:16 1154 2792 Y
Dec 25, 12 17:24:49 3302 6466 Y
Dec 25, 12 17:09:18 1331 3091 Y
Dec 25, 12 16:53:43 1007 3085 Y
Dec 25, 12 16:37:34 1467 3884 Y
Dec 25, 12 16:20:39 718 2353 Y
Dec 24, 12 02:02:46 1910 4276 Y
Dec 24, 12 01:18:56 1768 4446 Y

Looking promising along with the queue gaining 2 days toward being
caught up. Did Rollie make some changes or ??? Hope this trend
continues.

SannaSK · Post by **SannaSK** » Tue Jan 15, 2013 6:15 pm

Not that I know anything for sure, and also not to be a total naysayer, but those dates were when the db was already known by the userbase to be having issues, plus it's the Christmas timeframe, so my first thought was that fewer files were getting submitted on those days, thus the DB might still be processing at its same slow speed, and only has less to chew through.

Although I'd much prefer to subscribe to the theory of 'the problems are fixed' X D.

This thread has inspired me to do a complete purge of data after every upload, because even when things are fine serverside, why make it chew through the same data twice?

Padanfain · Post by **Padanfain** » Wed Jan 16, 2013 12:01 am

Good points SannaSK, I prefer to subscribe to the theory too.

I purged after every submission for most of December and in hindsight I
wish I hadn't. Each file that errors is just gone, poof, recycled electrons
before they were counted. Pruning to, say, two days or maybe three would
have contributed something of those soon to be recycled electrons when
that one file out of five or ten was a success.

PJEckenheim · Post by **PJEckenheim** » Fri Jan 18, 2013 12:46 pm

Jan 02, 13 20:10:19 0 0 N
Dec 27, 12 06:32:46 803 2534 Y
Dec 26, 12 13:34:41 4603 8626 Y
Dec 23, 12 20:33:10 0 0 E
Dec 23, 12 19:50:43 0 0 E
Dec 23, 12 13:24:47 0 0 E
Dec 23, 12 05:33:49 1676 3742 Y
Dec 22, 12 19:45:54 0 0 E

... we'll see if the bugs are gone. I'm afraid I don't subscribe to the "purge of data after every upload" method. I'd suggest a /prune to the date that the person uploading had the last confirmed successful upload...

Balgair · Post by **Balgair** » Tue Jan 22, 2013 2:43 pm

Dec 31, 12 10:30:49 218 1368 Y
Dec 30, 12 17:16:30 850 3416 Y
Dec 29, 12 08:23:58 524 2225 Y
Dec 28, 12 16:58:21 0 0 E
Dec 26, 12 16:53:41 3172 9275 Y
Dec 21, 12 08:08:55 4696 8451 Y
Dec 20, 12 16:50:30 1193 4743 Y

Certainly improving for me, not perfect but definitely doing better than it was. I mailed Rollie a few days back asking if he'd done anything to cause the recent improvements, but no response as yet so I guess he's not available right now, it may just be lesser load with fewer uploads over the holiday period, we'll see.