World of Warcraft Characters as seen by this site

[bringoutyourdead]

Just some random (or not so random) comments on data as seen here.
The database stats says we currently have over 69 million different characters recorded.

Doing some queries I noticed:

21,417,046 have not been seen since 2010-10-11 (day before Cataclysm release)

another 15,520,230 dropped away by 2012-09-24 (the day before Mists was released)

the loss of characters between these two dates was offset by the creation of 12,687,077 new characters currently above level 20.

Though out Mists of Pandaria another 12,844,102 stopped being seen, while 12,891,551 new characters currently above level 20 were added.

[bringoutyourdead]

There are currently 1,623,124 characters who are listed at current level of between 21 and 89 and not at an x5 or x0 level.
The earliest recorded of this group is almost one year ago.

The code that has been pruning characters will be found and killed.. or at least maimed.. ok.. at least stunned.

The code that has been pruning character history will also be found and mutilated to the point were it is replaced by a kinder more rational or at least thought out data management scheme.

Any free to play characters that do not get converted to a normal account (i.e. capped at level 20) will be removed.
The assumption is that any character that stopped leveling at 20 is not a normal account character.
The time frame for these characters will most likely be based on Blizzards name reclaim policy.
Currently 718 thousand level 20's that haven't been seen since 2010-10-11 are slowly being purged as I tweak the code.
I expect to take about 40 days to clear out these zombie characters at which time the next phase will be implemented.
Either more level 20's if Blizzard announces a new reclaim character date, or starting up the date/level criteria.

My goal is to have the database purging old inactive characters at some reasonable fraction of the rate of new character creation counts each day.

Characters that fall into the time frame of Blizzard character name reuse policy will be on the bubble for removal if needed.

The longer it has been since a character has been seen the more likely the character will put into the removal queue.
For characters that are on the bubble date, lower level characters will have higher priority for removal.

The past character history will also be treated the same.. in fact the removal of the zombie characters will reduce the count of history records.
In the process of cleaning out some discovered problem characters stored with faulty data I was able to removed almost a million worthless history records.
I would like to get the nearly 300 million records reduced to 250 million.. but we will see what is needed after the character maintenance is stable.

At some point the database will grow or shrink based upon how well Blizzard keeps current, old and new players interested in the game.

[bringoutyourdead]

The process to remove the Starter account characters is enabled.
At the current rate the above noted level 20 characters will be purged in about 24 more days.
After that phase two will start and will purge level 20 characters who have not been seen since patch 5.4.8 (the last patch before Warlords) and who are not listed as in a guild.
If guild listed then they can't be starter account characters.
Phase two will remove another 564 thousands characters.
This phase will take around 30 to 40 days to process.

There will probably be a phase three on level 20 characters who are not in guild probably cutting off with Legion pre-patch date... or maybe not, to be determined.

These phases have and will control the history growth and allow me to insure that we do not need to purge active character history data.

We were all very unhappy when that happened.

History data.
Some time later this year, probably after phase two, history data will be purged but other then seeing the total number of records dropping, no one will see any change in their characters history.

I have determined that as of this date, about 21% of all history records are zombies.
I call them zombie records because while the history is there, the basic character data no longer exists.
This was mostly caused by the data corruption that occurred when Cyrillic names slipped in past the character set limits.
Although I expect some of these zombies have been around for a very long time.
Using the same slow (low impact process) that is clearing out unwanted characters, the history records will probably take a number of months to clear out.

The outcome of this will be that the data will be cleaner and have less garbage taking up space.
Metalbeast will not feel the need to do a Broad-Axe melee attack on the tables, and we will all be happier about the character and character history recording.
Some time next year when all of this is done a short site outage will happen to reconfigure the history data tables.
There are eight history archive tables and a new data table.
The outage will be to reallocate data from the new table into the archives as appropriate.