Upload processing...

[Padanfain]

...looks like something broke on about the 25th. Looking at a bunch of user updates tabs and everyone I could see has E's since the 25th.

[piratejenny]

I made a successful upload last night (a bit later than your post); if somebody fixed something, it worked

[bringoutyourdead]

Metalbeast did some underlying software upgrades.. including the version of PHP that is used to handle most of the site pages.

[FuxieDK]

...looks like something broke on about the 25th. Looking at a bunch of user updates tabs and everyone I could see has E's since the 25th.

are you able to salvage the uploads?

I also had an E on 20th...

[bringoutyourdead]

I am looking at the logs to see what might have gone wrong..
more later after I learn more.

[bringoutyourdead]

hmm.. the problem is not something really broken.
I have been able to get both web upload and UniUpload files to process.
And I have seen that both fail.
A first level look at the processing log shows the failed files to not pass the initial Lua parsing (to verify that the file is a clean CensusPlus uploaded file.
Unfortunately the next step is never invoked and things appear to just hang until the timer limit is hit.
This could be (and probably is ) related to the PHP upgrade, but it might also be a systems level configuration issue as other code at that level was also replaced.

We may need to wait until Monday before Metalbeast can check what went wrong.

What I think is happening is that PHP processes are hitting default cut off timers when the upload files are too big for the default settings.

[bringoutyourdead]

Further investigation discovers this

Uploads of about 1.5MB and smaller process fine.

It appears that two of the control parameters for the PHP processor may have been reset back to default levels.

Correcting these parameters and making them active would require a restart of the PHP processor.. and may impact other domains on the server.. so I have informed Metalbeast of what I have found.

I expect the problem to be fixed shortly after he reads my email sometime Monday morning.. probably shortly after he starts his business day.

After he has informed me of the system level fix I will go into the control table for uploaded files and reset the failed uploads as unprocessed and they will process again.
By Monday Evening (PST) we should be back to normal with all files correctly processed.

[Padanfain]

Shiny TY

[bringoutyourdead]

Metalbeast discovered that his newest version of server control software had broken out some of the PHP control variables into a separate setup function... so they had been missed on setup.. love new software version changes.

I reset about 150 failed uploads back to unprocessed and kicked the processor into action.. of those, 12 failed again.. ( I knew some would because they failed initially with unknown or corrupted LUA data.)

I am looking at those 12 to see if I can spot the problem as these failures stopped the current active processing cycle.
This means it might be 12 hours before all the problem files were removed from the to be processed list (if I did nothing), not a real big problem... but something that Metalbeast might want to look into himself.

NOTE:
The reprocessing was in the order received so you should have gotten credit for updates as normal.
Except if someone including yourself, did a small file update on the same server/faction they might have slipped in and gotten credit for a small number of updates..
Just the way it goes.

[bringoutyourdead]

Looks like we have a new old problem
files larger then 8MB or so are failing to process..
I am not sure exactly what the problem is.. and have passed it on to Metalbeast.

Either we need to change some parameters to handle the larger files or we need to change the process for verifying that the files are clean CensusPlus upload files.

FuxieDK and Kanegasi have the most failed uploads.. having hit the size limit.. that shouldn't have happened.

After Metalbeast determines the fix.. I will again reset those 12 uploads and process them again.. maybe tomorrow... or maybe much later if we have to rewrite code.

[FuxieDK]

I upload, solely, with UniUpload, where size shouldn't matter..

The uploads from 25th-27th are now flagged as Y, but my latest upload is E...

[bringoutyourdead]

The problem wasn't the actual upload process where UniUpload gets around the 8MB Apache web server limit.
The problem was (mostly) the maximum memory limit for running a php process.
Metalbeast increased the limit again last night just before he left work.
I am going to check why the last few E results still exist.. it just may be I missed marking them as unprocessed.

[bringoutyourdead]

All the E status results for the last 10 days that are still E status, are valid failures.
The all failed the clean lua parsing.

Either the files were corrupted in creation or in transmission.

If you continue to have E status on uploads with those files.. delete the data file and restart your data collection.