Despite the various troubles...

General Discussion Area
1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Despite the various troubles...

Post by 1974ER »

...it is again time for some big figures:

Yesterday, my new ones count exceeded yet another breakpoint and now stands at 3 503 974 entries.

Earlier today, my all time updates count exceeded... 20 million and now stands at 20 003 746 entries.

User avatar
FuxieDK
Census Taker
Posts: 659
Joined: Thu May 22, 2008 11:36 am
Location: Copenhagen, DK

Post by FuxieDK »

You never cease to amaze me :P
Doing census mainly on Draenor; Raluf - Nimsay - Lusmo - Quixx - Sosyan - Garthog - Trubin - Zalistra - Zesmi and Djaang

Balgair
Araiceil
Posts: 1716
Joined: Fri Sep 30, 2005 11:47 am
Location: UK

Post by Balgair »

Omg congratulations! :) That is just crazy, well done! I might see, um, 9 million soon, if I actually do some censusing (given we're usually 2-3 weeks behind I kiinda stopped since it's a bit hard to tell what needs doing :P)

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

Almost unbelievable, I can't keep that consistent in working my way through realm updates. I keep getting distracted by playing the game. :lol:

What is almost as unbelievable is that with 880k updates recorded for March the database has dropped to less then 11 days behind.. the best we have seen in a long time. Who knows with luck we might see current (inside of 7 days) data before Summer.

Now if I could only get the pet battle issue handled... Blizzard allows my trial account to buy the training but not actually try pet battle so I have no way to test. sigh

User avatar
FuxieDK
Census Taker
Posts: 659
Joined: Thu May 22, 2008 11:36 am
Location: Copenhagen, DK

Post by FuxieDK »

bringoutyourdead wrote:Now if I could only get the pet battle issue handled... Blizzard allows my trial account to buy the training but not actually try pet battle so I have no way to test. sigh
Is it the Mini-map button? If so, look at my ticket --> http://wow.curseforge.com/addons/census ... when-mini/ for some details :lol:

I'd be happy to test for you, if you can't test yourself..
Doing census mainly on Draenor; Raluf - Nimsay - Lusmo - Quixx - Sosyan - Garthog - Trubin - Zalistra - Zesmi and Djaang

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Well, I hope I will not jinx things... but actually, the queue was under 9,5 days for a little while. Now it seems to be taking a breather and is a little bit over that.

On the shortening note... One reason for that might be that due to problems with the site, I missed out on 19th, 20th, 21st and again on 23rd, 24th and 25th as far as submissions go. I did gather data, but I am actually backlogged myself, though only about by 35 hours.

Another, far more serious reason is that we have really taken a hit on submission makers. I am not sure where you saw that 880k figure... but according to my quick count, the current top 10 for March accounts for roughly 495k of that. That is.... TEN people have done well over 55% of overall result.

Balgair
Araiceil
Posts: 1716
Joined: Fri Sep 30, 2005 11:47 am
Location: UK

Post by Balgair »

I'm not surprised submissions are down tbh, there's been problems with the site so long now that I can understand a lot of people just giving up, or at least waiting for things to improve (I've given up on extra censusing myself and am just sticking with the servers I'm playing, as it's impossible to get any info on which servers are currently in need - ok, it's probably all of them!).

The times where the site was down obviously won't have helped either, since people trying to upload won't have succeeded (and if using UniUploader may not even have realised, since you have to be checking the server response tab to actually see it didn't respond...).

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

Epic - I know where you come up with your count... but you need to remember there is one other class of uploader that far, and I really mean far, exceeds your updated data.

Log out of the site and then check the March numbers... that uncredited set of users Anonymous has 2.5x your upload for the month and has more then 10 times your total numbers to date.

currently averaging more then 1/2 million a month.. and last year at this time was averaging more then 1 million a month.

I wonder how many people are involved in the uncredited submissions?

Fuxie - Yes it is that never to be sufficiently damned mini-map button which isn't really a mini-map button. The coding that Rollie evolved to make that :twisted:thing:twisted: caused as much as 50% of the retesting that Balgair had forced upon her by my inability to understand what it was really doing.. as it interacted with the various parts of the addon.
As it is really a cosmetic problem and not a functional issue, I loathe to take it on at this time as I have other projects underway.
But someday I will get back to it.. and other user interface changes I have considered.. and I'll keep in mind your offer as a test subject :twisted: !!

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Bringoutyourdead... I, however don't know, where you conjured that 880k figure you cited earlier... because top 10 + anonymous is already just a hair under one million entries. Did you perhaps mean 880k credited + 490k from anonymous for March so far? *slightly confused*

I also have to point out that despite my title, I have often not been the very best of a single month. This February being a prime example... and another point:

Padanfain: 395,208
Anonymous: 433,000

One single person almost beat the entire group of anonymous.

Back in November 2008, when I started...

I was 2nd: 309,415
Anonymous: 3,959,509

That's about 13 times as much as I achieved.

Right now March values look like this:

Mine: 169,226
Padanfain: 79,888
Anonymous: 490,701

That is almost exactly x 3 mine... or a bit more than x 6 Padanfain. The anonymous users have been hit even harder than others. Which is exactly what worries me. We are losing those vital people who throw up a few hundred or thousand updates once or twice a week.

Additionally, you might consider things in reverse... in slightly under 53 months, my update count is roughly equalent to 9% of what all anonymous users have done in total since site opened. You see, despite that feat... I am not hard to replace, but replacing or regaining dozens or possibly even hundreds of anonymous... now THAT's hard.

I was actually thinking about writing more, but it's extremely late here and I should already be asleep, so I'll just end with this note for now:

Last processed file was uploaded on 2013-03-26 15:41:05.
Current server time is 2013-04-04 17:27:08

About 9 days, 2 hours and looking good. :)

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

1974ER wrote:Bringoutyourdead... I, however don't know, where you conjured that 880k figure you cited earlier... because top 10 + anonymous is already just a hair under one million entries. Did you perhaps mean 880k credited + 490k from anonymous for March so far? *slightly confused*
Yes that is exactly what I had meant.. on my trusty "solar powered" calculator I added up the top 10 and Anonymous and got the 880k
1974ER wrote:... but replacing or regaining dozens or possibly even hundreds of anonymous... now THAT's hard.
To that I totally agree, sigh the less then fully envolved management of the site is cutting into our numbers of submitters.
If I could, one of the things I would do is post an acknowledgement page on a monthly basis.. all it would have is the names of everyone that made at least a single submission for the previous month along with a message of thanks.
I would also make it easier for people to do credited uploads, including method for having uploaders to know if they are nearing the raw upload limit and enabling selecting pruning of the data file as part of the upload process is wanted/desired.

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Well, couple of days have passed and now less than 23 hours worth of March remain to be processed. The figures now look like this:

Mine: 207,877 (up about 38k)
Anonymous: 562,942 (up about 72k)

Which sadly means that the anonymous submissions have gone down even more. The ratio doesn't even reach 1:2 during the time from my previous post to this moment.

Last processed file was uploaded on 2013-03-31 01:12:24.
Current server time is 2013-04-07 09:37:08

The news isn't all bad, though. :) The backlog has now shrunk to about 7 days, 9 hours. As for me, I have finally gotten rid of my personal backlog. Overall, if things go really well, we might start seeing data within 7 days some time tomorrow as there can't be very many files left for March. I have 4 and several people I checked have none, 1 or 2.

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

Sigh the E's are back.
While I don't know for sure.. I expect 1974ER got caught in the month end processing as the database fills in the table entries for full month based data.

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Whatever the reason is, the queue is choking on my files. Seems it has managed only 5 entries for entire April so far, blowing mine while succeeding with the 4 others... and now it's stuck on my second one. Over twenty hours of real time equals less than 1,5 hours of queue progress... URGH. :(

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

I looks like Rollie disabled some of the data processing... some time after 6am on the 2nd.

Submitters pages stopped showing updates for the monthly numbers as of that time.

Also even though the processing lag time has now dropped to 5.5 days the Census DB stats page hasn't started updating the "characters 'seen || created' in last 7 days" lines.

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Total character count 101,411,931
Characters seen in the last 7 days 88,068
Characters created in the last 7 days 18,212
Total guild count 6,678,654
Number of Alliance realms updated in the past 24 hours 487
Number of Horde realms updated in the past 24 hours 487

Last processed file was uploaded on 2013-04-05 15:49:53.
Current server time is 2013-04-10 23:52:44

It MIGHT have been an additional temporary glitch. It didn't work for a while for me either, but now we are seeing some within 7 days data, finally. :D

I had two Es on 1st, two more on 4th and might become one more for 5th. That's not sure yet, because the data above suggests my file is still under process... as it was uploaded at 16:02:23.

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

*very deep sigh* The E's have made yet another comeback, queue is stuck on MY files (again) and we are about to lose the under 7 days of backlog status, which we were able to enjoy for several days. :(

Apr 17, 13 09:50:57 0 0 E
Apr 17, 13 09:04:36 0 0 E
Apr 17, 13 05:56:27 895 3832 Y
Apr 17, 13 00:58:34 182 1039 Y
Apr 16, 13 17:22:28 1682 7031 Y
Apr 16, 13 11:28:36 0 0 E
Apr 16, 13 04:59:31 0 0 E

Characters seen in the last 7 days 4,499
Characters created in the last 7 days 648
Total guild count 6,691,652
Number of Alliance realms updated in the past 24 hours 485
Number of Horde realms updated in the past 24 hours 487

Last processed file was uploaded on 2013-04-17 09:24:31.
Current server time is 2013-04-24 03:36:27

Blaxx

Post by Blaxx »

Hi guys, thx for all your hard work. You rock! Just a quick question:
I have uploaded some data via the web interface for EU-Area 52 and it seems to be not updated.

Currently Census+ finds rough about 2.100 Chars on Alliance side and about 900 on Horde, taken snapshots in the evening, when many folks play.

http://www.warcraftrealms.com/census.ph ... rtypeid=-1 ist shows way too less players :/.

Do i have simply to wait till it gets updated? I find Census+ very interesting and pointing to "real live player numbers." We think that our Server gets more and more underpopulated and so we think abour a server change. But for example http://wow.realmpop.com/ shows ridiculous high numbers, which are silly. E.g. on our server are never ever > 40.000 players lol.

So Census+ seems to figure out the real server population pretty precise, but just the DB / Website seems to be not working right now?

Balgair
Araiceil
Posts: 1716
Joined: Fri Sep 30, 2005 11:47 am
Location: UK

Post by Balgair »

Hi Blaxx, welcome! The database is currently running around a week behind on processing submissions; it seems we've got more data than the hardware can really cope with so it's been struggling for a while. So if your uploads were within the last week, then I'd expect the server info on the website to update and show numbers more in line with what you're expecting soon, it just may take a few days! :)

You can see where the database is up to at http://www.warcraftrealms.com/dbstats.php - right now it seems 6 days and 10 hours behind, which should give a fair idea of how long it'll take to reach your uploads.

Blaxx

Post by Blaxx »

Hi Balgair :) Thx a lot for the welcome and the update. Oh yes, i see, the db has to cope with Total character count 101,855,464 - wow what a bummer :)

Cheers and a fantastic weekend guys

1974ER
Epic Censi
Posts: 762
Joined: Fri Nov 07, 2008 3:30 am

Post by 1974ER »

Hello Blaxx and welcome from me as well.

Balgair already filled you in on the database backlog issue, but I want to comment your note on realmpop. Realmpop show excessive figures, because they count existing characters, not ones that are active. In short, absolutely every single realm has more characters on realmpop, because the system there doesn't care if the character has been inactive 2 years, as long as it exists. Most likely, there are about 40 000 existing characters, but only a fraction of them are ever played.

If you persist with us through the troubles and keep on submitting, you will soon start seeing more accurate values for your own faction(s). :) Under the current circumstances, we need all the help we can get. Due to the troubles, the number of people using CensusPlus has gone down and many realms have really poor coverage (in many cases, no regular coverage at all).

Warcraftrealms strives to represent the current (within last 30 days) activity. We are currently hampered by the backlog, but the values here are generally speaking closer to the truth (or in case of poorly covered realms, way too low). Also, unlike other sites, we have a huge amount of historical data.

Enough for single post...

This is Epic Censi, wishing a good weekend to all readers. :)

Post Reply