Why am I getting this problem all of a sudden?

Moderator: Forum Moderator

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Why am I getting this problem all of a sudden?

Post by Zimoon » Wed Jun 09, 2010 1:07 pm

Me neither, everything seems so odd. And the area in SWGAide that handles these thing has not been changed in quite some time.

So, the new version with better logging is available. But then there is that guards thingie.

/Zimoon

User avatar
Monty Burns
Master Crafter
Posts: 549
Joined: Sat Mar 08, 2008 9:26 am
Location: New Zealand

Re: Why am I getting this problem all of a sudden?

Post by Monty Burns » Wed Jun 09, 2010 1:27 pm

Well I seem to have managed to update the Chemicals section so I guess things are back to normal.

So I really dont know but I would like to figure it out before it happens again.
:)

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Why am I getting this problem all of a sudden?

Post by Zimoon » Wed Jun 09, 2010 2:58 pm

Me too, nothing is a frustrating as a heisenbug, the worst thing.

Being able to reproduce or trace down an issue is not only a step in he right direction bot very often the only step in any direction, and it impossible it is a standstill.

/Zimoon

User avatar
Monty Burns
Master Crafter
Posts: 549
Joined: Sat Mar 08, 2008 9:26 am
Location: New Zealand

Re: Why am I getting this problem all of a sudden?

Post by Monty Burns » Wed Jun 09, 2010 9:15 pm

What are the chances this was a router issue on my end?

I can not understand why all of a sudden everything is working again on 2 computers when I only made changes on one of the computers (I left the other alone as a control so to speak) and the only connecting link on my end is the modem/router.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Why am I getting this problem all of a sudden?

Post by Zimoon » Wed Jun 09, 2010 9:34 pm

- unless you have a router that does file caching I cannot see this happen.

- unless Windows does file caching I cannot see this happen; and if it did more people would report it.

- unless you thought you did see something but reality was not that...

- if it was not for that odd time-stamp in the status file, I would say user-misconception, but I am not convinced so I pass the chance.

- now I just regret that I did not ask you to check the file permissions on that status file, but on the other hand I cannot for my life imagine that a file-access clash on the status file (the ISDroid-logic reading it during the check that lead to the dialog while the download wrote the file as its last task) could cause anything, what would happen is an exception that the file is locked and cannot be accessed bu the other thread. And that would read in a log file.

- so I am as puzzled as you are :|

/Zimoon

User avatar
Monty Burns
Master Crafter
Posts: 549
Joined: Sat Mar 08, 2008 9:26 am
Location: New Zealand

Re: Why am I getting this problem all of a sudden?

Post by Monty Burns » Thu Jun 10, 2010 12:00 am

Ok it is still happening but it seems to correct itself after the first few seconds, I just did a mineral update and it got to about 10 seconds before it kicked in and read read the emails so could this be an internet problem?

Essentially I am approaching the problem from the angle that it has to be a connection issue as it has only occurred on my 2 PC's (no one else has reported it), it can not be a SWGAide or PC problem as both PC's are different in all respects except router and the problem rectified itself at the same time on both PC's even though I reinstalled it on one.

The only point which both systems have is from the router to SWGCraft therefore the problem must lay there.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Why am I getting this problem all of a sudden?

Post by Zimoon » Thu Jun 10, 2010 2:35 pm

When you say "problem" here, exactly what do you mean by that?

Let me explain how the ISDroid thing works and SWGAide's internal updating to see if that shed some light over the entire issue. And I do it from a startup of SWGAide.

When you launch SWGAide and if the option to auto-update the main galaxy (Sunrunner in your case) is enabled, then SWGAide first checks the content of that small status file against the content of the same file at SWGCraft. If they are not equal SWGAide downloads the updated resource file from SWGCraft. There is no fuss or complex stuff with this.

Both of these operations of course take some time, determined by the distance between user and SWGCraft, but also the quality of and the contention at the Internet connection. New Zealand versus GB is of course not negligible but should not be a major issue. Obtaining the smaller status file should be well within half a second at any distance, and the larger resource file is still just about 25 kB so even with an old 28.8 modem it would take less than 10 seconds.

SWGAide unpacking the zipped file plus parsing the XML file takes anything from milliseconds to perhaps a few seconds, determined by the computer and if some Java classes have already been loaded once, or not. Only if there is a parallel action happening it could take a fraction longer, if that action is blocking the resource manager; the manager must be blocked so that one thread in SWGAide does not change something that another thread reverts a split second later.

OK, this far SWGAide is started and you see the list at Current Resources. In fact, this list is populated already before the download is finished, that is if you are really quick and browse to it before the download is finished. If there was something new, some seconds later you'd see the news show up topmost and at the same time you'd see a message at the status bar and perhaps hear an alert.

From now on SWGAide checks that little status file versus SWGCraft every 3 minutes, and if there is anything new it is downloaded and SWGAide scans the stuff for guards/monitors/idling-harvesters and notifies the GUI that it should update itself. Actually, because SWGCraft compiles the export files for all galaxies, also those without anything changed, the download takes place every 30 minutes whatsoever. A waste of Internet bandwidth and computer cycles, but considering the hardly noticeable size of these files ... 8)

After a successful download, but only at success, SWGAide updates the local status file to read the same value as at SWGCraft. But rather than one file, as at SWGCraft, there are one file per galaxy which supports the case that users sometimes but not regularly browse other galaxies in SWGAide. Only the main galaxy is auto-updated, any other galaxy is updated on demand and their status files can be days or even years old.

Now, what about ISDroids? SWGAide handles the reports in this manner. The mail reports are loaded individually. For the four different lists the reports are compared against what SWGCraft knows about the galaxy. This way it is possible to conclude which resources are depleted (known at SWGCraft but no trace of them in the reports), which resource are known at SWGCraft but not reported for all planets (the reports tell more planets than SWGCraft knows about), which resources are known but without stats (you guess how), and which resources are new (unknown at SWGCraft but told in the reports).

To ensure best possible result, before SWGAide starts comparing reports versus what SWGCraft knows, it checks that little status file versus SWGCraft once again. If there is a new resource file available it downloads it and wants to use that rather than the previous which is about 30 minutes old.

Only if you launched SWGAide and instantly went into the ISDroid folder the previous download is from the recent time you run SWGAide. But then SWGAide is already downloading, perhaps not yet finished but the initial download is going on, and the ISDroid panel shows that "counting-down" text at the status bar. Once the download is finished the panel can go on and begin its routine and populate the lists.

If SWGAide had been running for a while two thing can happen when you go to the ISDroid panel:
1) The status file equals the status at SWGCraft, the ISDroid panel populates right away.
2) The status file comparing denotes that there is an updated file available, and it is initiated and you see that "counting-down" at the status bar, which in your case somehow ended in a time-out after 60 seconds.

This second case may happen only if the most recent time the status file was automatically checked there was no update available, but you happen to go to the ISDroid panel when one is available. Yeah, this can happen, during the 3 minutes window and you go to the ISDroid panel just after xx:15 or xx:45 when SWGCraft has compiled the new files.





What I looked most careful at was if there is a chance that the download and the count-down routine in some way blocked each other. Or if the count-down routine perhaps used a value cached in memory rather than an updated value once the download was finished. I doubt it. I have seen the count-down several times myself, but only for something like 5 to 10 seconds.

What makes this whole issue peculiar and weird is that the status file from your screen shot clearly shows a date that was 77+ hours back in time and SWGAide had been running meanwhile but failed to update it or download anything, but yet the files for Sunrunner had timestamps that displayed a fresh date/time (from the folder screen shot). And none of the files should even be touched by SWGAide unless their content had changed. The status file is only modified if a successful download is finished, otherwise SWGAide just bails out but logs an error. And you say that this was present at two different computers at the same time. Hence your conclusion makes sense, neither SWGAide nor the computers are the culprit but something outside these. If that is your router, the Internet provider, or what ... I cannot tell. If the screen shot of the status file had displayed a time difference of hours or less, then I'd blame you :lol:

Yes, I know Internet providers that have been testing file caching to cut down inter-continental transmission costs. That was about caching files that has equal URL and file size, however, it turned out that the costs for caching was way worse than let the transmission through. Only satellite, rare radio lines, or hot-spot routing under certain circumstances showed a net gain. I doubt New Zealand ISPs play around with this stuff these days. And if they do they have a flaw in the caching, it is not enough to compare URL and size, also a complex enough hash code (finger print) of the content must be identical to make sure everything works :P

Does your router/modem do caching? I have no idea, you could check its specifications. Over at my location there are no modem/routers for home/small-offices that does it.



I am sorry that I cannot get closer to an explanation.

What I can do is to remove the check for the ultimate freshness, which would remove the count-down, and only warn if the most recent download is more than 40 minutes or so. I mean, I cannot see a risk in using a download that is about 30 minutes old rather than one that is less than 5 minutes. Can you? But at least we get rid of this annoyance and a possible error pothole, what do you say? Or anybody else?

/Zimoon
Last edited by Zimoon on Fri Jun 11, 2010 2:03 pm, edited 1 time in total.
Reason: Edited: it is rather 3 minutes between status checks than 6

User avatar
Monty Burns
Master Crafter
Posts: 549
Joined: Sat Mar 08, 2008 9:26 am
Location: New Zealand

Re: Why am I getting this problem all of a sudden?

Post by Monty Burns » Thu Jun 10, 2010 7:36 pm

Zimoon wrote:What I can do is to remove the check for the ultimate freshness, which would remove the count-down, and only warn if the most recent download is more than 40 minutes or so. I mean, I cannot see a risk in using a download that is about 30 minutes old rather than one that is less than 5 minutes. Can you? But at least we get rid of this annoyance and a possible error pothole, what do you say? Or anybody else?

/Zimoon
Would it be difficult to add this as a selectable option since the vast majority appear to have no issues with the current arrangement it seems pointless to go backwards on a single case?

As far as whether New Zealand ISPs are playing with caching or not all I can say is that I wouldn't discount the possibility, after all the New Zealand business mantra is "If it is worth doing, it is worth doing half arsed" and we do that well.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Why am I getting this problem all of a sudden?

Post by Zimoon » Thu Jun 10, 2010 10:08 pm

Monty Burns wrote:Would it be difficult to add this as a selectable option since the vast majority appear to have no issues with the current arrangement it seems pointless to go backwards on a single case?

As far as whether New Zealand ISPs are playing with caching or not all I can say is that I wouldn't discount the possibility, after all the New Zealand business mantra is "If it is worth doing, it is worth doing half arsed" and we do that well.
LOL, I imagined at least some country would have that as its mantra, but I couldn't imagine it is NZ :P

I will disable it completely at my test bed and see what happens until it is time for next release.

At least you now know one way to kind of reset and kick-start SWGAide to do another download if it seems "dead".

/Zimoon

Post Reply

Who is online

Users browsing this forum: No registered users and 18 guests