Upload Not Being Credited

Fett · Post by **Fett** » Tue Jul 30, 2019 8:36 pm

Has anyone else had an issue with maybe 10-20% of the files you upload while you are logged in are not being credited to your account?
The data is still added to the database, presumably anonymously.
I often upload data in batches, so when it happens, I'll get the "We have detected that you submitted data within the past hour. Your previous file has been overwritten with with this one" message, but as nearly as I can tell, it doesn't actually overwrite anything.
It's not a big deal, but I think it's a new thing. I think it started between a couple of weeks and a month ago. Is it something I'm doing, or is everyone seeing the same issue?

1974ER · Post by **1974ER** » Thu Aug 01, 2019 11:09 am

@Fett: It does not happen that often to me and since I use manual uploads, the symptoms are a bit different, but due to that I occasionally get 0, 0, Y style results as the file has uploaded and processed, but not listed originally on my own listing.

At first I thought I had made some sort of error myself and somehow forgotten to upload, but then it happened again... and repeated itself today, just a little while ago...

Post by **bringoutyourdead** » Mon Aug 05, 2019 6:16 pm

Remember when you see 0,0,Y
That means your upload was valid, it was processed.. just that someone else had recently uploaded for the same server/faction and there were no new updates to be found in your upload.

Post by **bringoutyourdead** » Mon Aug 05, 2019 6:47 pm

Fett my recommendation is to pace out your uploads.. you might be running into an issue where the system hasn't finished all processing before your next upload hits. I am probably saying this badly.

Slow the uploads down.. say one every few minutes .. instead of your latest blast of 5 downloads within 4 minutes.
There just may be times where the database hasn't finished all background processing when the next upload hits.

As to the question of your data being credited someone other then yourself... that does not happen.
I scanned the upload database from what I could see all uploads coming from your then current IP were getting credited to your user id.

Post by **bringoutyourdead** » Mon Aug 05, 2019 7:12 pm

The last thing I am looking at on this were files that failed lua parsing checks.

Often times these have character or more likely guild names that create a problem in parsing the data as valid lua constructs.
Sometimes these issues self correct with later census runs.. but when they don't the only fix is to delete your copy of data and start fresh.

And then there is for example the last failed file from 1974ER.. which had the character data but had no data for the [TimesPlus] section of the database which gets populated as one of the last things before a census run finishes.
Apparently the login session was broken before the processing was finished and the client saved the data that existed at that time.

1974ER · Post by **1974ER** » Tue Aug 06, 2019 2:17 am

@Bringoutyourdead: The thing is... From my point of view... I am the one who submits the data, but the upload does not get listed in my history, then, when I try again, it uploads and lists, but after it processes, it shows up as 0, 0, Y, because the first upload actually went through and processed, but did not show up in my list of uploads.

Due to time difference this may be difficult to track, but it is now 11:13 my local time and the error occured again about 7 minutes ago. If you can, try to look up whether I actually sent up the same file twice within about 4 minutes. My upload history only shows one...

Aug 06, 19 03:06:43 0 0 Y

It processed while I was typing this...

Post by **bringoutyourdead** » Tue Aug 06, 2019 3:15 pm

What you see is all that is recorded in the logs.. so the upload for some reason is not getting fully accepted.. even if it has been.. sigh that is a metalbeast problem.. and you know what that means.

Fett · Post by **Fett** » Wed Aug 07, 2019 12:48 am

1974ER wrote:@Fett: It does not happen that often to me and since I use manual uploads, the symptoms are a bit different, but due to that I occasionally get 0, 0, Y style results as the file has uploaded and processed, but not listed originally on my own listing.

At first I thought I had made some sort of error myself and somehow forgotten to upload, but then it happened again... and repeated itself today, just a little while ago...

I do upload them manually. What I mean by uploading in batches is that I save files until I have several to upload at once.
Since you're seeing it too, I am curious about commonalities. I am using Firefox, are you using the same or a different browser?
(I'm not sure that even could affect it, but it was something that occurred to me.)

When you said it wasn't happening that often to you, it got me to wondering if I was accidentally exaggerating, so I started tracking data. My first eight uploads after that, 25% of them did it, but that was an awfully small sample. By the time I got to 30 uploads, It was down to 16%.

bringoutyourdead wrote:Fett my recommendation is to pace out your uploads.. you might be running into an issue where the system hasn't finished all processing before your next upload hits. I am probably saying this badly.

Slow the uploads down.. say one every few minutes .. instead of your latest blast of 5 downloads within 4 minutes.
There just may be times where the database hasn't finished all background processing when the next upload hits.

I'll try that. I have seen it happen when I haven't uploaded anything for over an hour, so I don't think that will eliminate it, but if I can even reduce the frequency, it's worth taking the extra time.

1974ER wrote:@Bringoutyourdead: The thing is... From my point of view... I am the one who submits the data, but the upload does not get listed in my history, then, when I try again, it uploads and lists, but after it processes, it shows up as 0, 0, Y, because the first upload actually went through and processed, but did not show up in my list of uploads.

This is what I'm seeing too. When I get the error message, if I don't reupload, I don't get the 0,0,Y, it just doesn't create any entry in my upload log. So I usually reupload knowing that it will be a 0,0,Y, but that entry is still a placeholder so I know how many files I've uploaded.

Here's the kicker. I just uploaded five files a bit ago, before reading that I should space them out. Two of them gave me that message. I reuploaded both of them. One of them came back with a 1865,4079,Y instead. This is the first time I have noticed the reupload returning anything but a 0,0,Y. So now I'm more confused than ever.

bringoutyourdead wrote:What you see is all that is recorded in the logs.. so the upload for some reason is not getting fully accepted.. even if it has been.. sigh that is a metalbeast problem.. and you know what that means.

No worries. My main concern was that it was something that I was doing. 1974ER seeing it too means that it at least isn't just me. If we can figure out something we're both doing that triggers it, great. If not, it's still no big deal, just a curiosity.