Data Export

The purpose of this board is for any announcements by the developers. Only developers can start topics on this board.

Moderator: Forum Moderator

User avatar
Sobuno
Developer
Posts: 2589
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Data Export

Post by Sobuno » Wed Jun 20, 2007 2:23 pm

The Data Export feature should now be working. It should have the exact same structure as the SWGCraft.com one, but if it doesn't please let us know.

Here are the rules that must be adhered to if you make use of this feature:

1. You must place the following text and link in a prominent place on your web site or application:

Resource data provided by SWGCraft.co.uk

Anyone found not displaying this text and link may be banned from the site.

2. You are not allowed to charge a user for access to this data, either real money or in-game credits. Using the data to support a premium service is fine as long as the user is only paying for access to data other than the resource data provided in the export files e.g. waypoints. If you are using it to support a premium service then the data provided in the export files must be viewable by all users on your site, not just the premium members.

3. Please use this static data rather than screen scraping the data on the site. Screen scraping puts extra load on our database, this feature doesn't.

4. I cannot guarantee that this feature will be available at all times (server downtime etc.).

5. You must code your web site or application to only download the XML or CSV file when there is a new one available. There is no advantage to be gained from downloading every time you need to access the data and it will keep our bandwidth usage figures down. You can use the status files (see below) to find out if a new version of the file is available.

6. To preserve bandwidth the XML and CSV files are supplied in gzip format (~600KB instead of ~9MB). Your web site or application must be able to decode gzip files. See the links at the bottom of this page for help with this.
Those are the rules from SWGCraft.com, as we believe they make a nice ruleset :)

Now, on to the files:
The links to the full resource files are:

http://www.swgcraft.co.uk/dev/sendfile. ... ces.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ces.csv.gz

If you are only interested in the resource information for a certain server then seperate XML files are available:

http://www.swgcraft.co.uk/dev/sendfile. ... azi.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... fin.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ria.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tra.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... era.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tis.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... pse.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tar.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... rry.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ath.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ity.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... pid.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... uri.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... oor.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... wca.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tus.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ant.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... lla.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ire.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... der.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ner.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... nas.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... est.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ter.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... cyn.xml.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ome.xml.gz

And seperate CSV files are available:

http://www.swgcraft.co.uk/dev/sendfile. ... azi.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... fin.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ria.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tra.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... era.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tis.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... pse.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tar.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... rry.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ath.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ity.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... pid.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... uri.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... oor.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... wca.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... tus.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ant.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... lla.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ire.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... der.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ner.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... nas.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... est.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ter.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... cyn.csv.gz
http://www.swgcraft.co.uk/dev/sendfile. ... ome.csv.gz

There is also a status file available, that allows you to get the date when the files were last generated (in UNIX timestamp format, as are all the dates in the files):

http://www.swgcraft.co.uk/dev/public/status.xml
http://www.swgcraft.co.uk/dev/public/status.txt

XML and CSV versions of the SWGCraft.co.uk server, planet and resource tree data are also available:

Resource Tree XML
Resource Tree CSV
Server XML
Server CSV
Planet XML
Planet CSV

Also, a resource class mapping is available that shows the link between our internal ID and the old SWGCraft.com ID:
Resource Class Mapping
-The SWGCraft.co.uk Development Team
Last edited by Sobuno on Thu Nov 15, 2007 5:10 am, edited 1 time in total.

Tharek Tolemac
Intermediate Crafter
Posts: 139
Joined: Sun Jun 03, 2007 8:48 am

Can anybody help?

Post by Tharek Tolemac » Tue Nov 13, 2007 10:18 am

Hello!
I am also using MyRes.

Actually, I changed all the settings to the new paths, which is acutally like this:
http://www.swgcraft.co.uk/dev/sendfile. ... cetree.csv
http://www.swgcraft.co.uk/dev/sendfile. ... ity.csv.gz
http://www.swgcraft.co.uk/dev/public/status.txt

But still, it doesnt work. I think it is because of the resource tree. I read somewhere the new one doesnt work. Now, the question is:

Can somebody give me an old resource tree from swgcraft and can you tell me, how I get it inside Myres?

That would help me a great deal.

Regards,
Tharek Tolemac

PS: I think there isnt something like a "Support" Forum in here. Thought several people suffer trouble with MyRes actually, I suggest somebody opens a Guide how to configure MyRes correctly and maybe other things.


EDIT: Seems I made it work somehow. Pure luck, I guess. :) Still, it would be an idea to establish a support forum here with guide for such problems.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Tue Nov 13, 2007 12:03 pm

MyRes can use two different resource trees, the one from old SWGCraft and the resourcetrexe2.XML. The latter can be downloaded in its most recent version from
EDITED: Files now found at http://simongronlund.googlepages.com/swgresourcefiles
Unfortunately MyRes seems to have something hard coded because "consider cap" stops working.

Just drop it in the MyRes folder and then choose Updates >> Update Resource-tree >> XML file.

Good luck

/Zimoon
Last edited by Zimoon on Sat Feb 02, 2008 1:11 pm, edited 1 time in total.
Reason: new URL

Tharek Tolemac
Intermediate Crafter
Posts: 139
Joined: Sun Jun 03, 2007 8:48 am

Post by Tharek Tolemac » Tue Nov 13, 2007 12:38 pm

Hmm... it seems MyRes cant save the path for current resources. After each restart of MyRes, it is linked to the swgcraft server again. :(

EDIT: Lol, now I brought in the XML resourcetree, set the paths on the project links and it happens... nothing. It has 0 resources and says "Data is uptodate"

Any ideas?

Strange. Now, it caught the data from project obviously. Crashed after that and changed the path to swgcraft again after restart.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Tue Nov 13, 2007 3:10 pm

Works at my side, must be a problem at your side then?

I have now tried a few things but cannot get it not to work. Is you MyRes installation OK? Or could a reinstallation help?



Joke:
Two Microsoft programmers tried the shooting range but after a set of shots one of them yelled while inspecting the target, "No hits buddy!"

The other guy, still standing at the aim and fire platform looked into the barrel, puzzled as he saw no faults. He decided to debug the thing so he put his thumb over the barrel outlet and fired. "Ouch", he said, but yelled to the other guy, "works at my side, must be something with the target!"

lehorton
Novice Crafter
Posts: 5
Joined: Fri Aug 10, 2007 1:17 pm

Post by lehorton » Tue Nov 13, 2007 5:33 pm

[quote="Tharek Tolemac"]Hmm... it seems MyRes cant save the path for current resources. After each restart of MyRes, it is linked to the swgcraft server again. :(

I have that same problem. Can get it working with a lot of fooling around but it takes some manual configuration each time you start the program up.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Tue Nov 13, 2007 6:03 pm

Again: I think you guys do not have a flawless installation. You should uninstall it, Control Panel >> Add/Remove Applications >> find MyRes. And then install it again.




For those not too faint of heart, open up Windows RegEdit (Start Button >> Run >> type regedit).

Search for MyRes and edit all those places need editing. But if the installation is flawed you may not find any trace of it in the registry.

/Z

lehorton
Novice Crafter
Posts: 5
Joined: Fri Aug 10, 2007 1:17 pm

Post by lehorton » Thu Nov 15, 2007 1:51 am

The links for there resource files have an extra /dev in the path.

User avatar
Sobuno
Developer
Posts: 2589
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Post by Sobuno » Thu Nov 15, 2007 5:10 am

lehorton wrote:The links for there resource files have an extra /dev in the path.
Thanks, fixed :)

-Sobuno

Holden
Apprentice Crafter
Posts: 61
Joined: Fri Oct 19, 2007 4:44 am
Location: Chicago, IL, USA

Re: Data Export

Post by Holden » Sun Dec 30, 2007 6:03 am

Did something change with the forum upgrade?

re: status.txt
The page http://www.swgcraft.co.uk/dev/public/status.txt does not exist

re: currentresources_starsider.csv.gz
File not found

(And BBCode is off?)

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Data Export

Post by Zimoon » Sun Dec 30, 2007 11:05 am

Holden wrote:Did something change with the forum upgrade?

re: status.txt
The page http://www.swgcraft.co.uk/dev/public/status.txt does not exist

re: currentresources_starsider.csv.gz
File not found

(And BBCode is off?)
Verified, the two comments on missing files comments are valid.
I have BBCode though, at least colors and links.

/Zimoon

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Re: Data Export

Post by Zimoon » Sun Dec 30, 2007 11:13 am

Holden wrote:(And BBCode is off?)
Have you checked that the option has not been disabled in your User Control Panel >> Board preferences >> Edit posting defaults?

Just a thought.

/Z

User avatar
Sobuno
Developer
Posts: 2589
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Re: Data Export

Post by Sobuno » Sun Dec 30, 2007 12:22 pm

I knew I forgot something!

The file's should be there now, they're from the night though, give them ~30 minutes to update though

Holden
Apprentice Crafter
Posts: 61
Joined: Fri Oct 19, 2007 4:44 am
Location: Chicago, IL, USA

Re: Data Export

Post by Holden » Sun Dec 30, 2007 12:57 pm

[quote="Zimoon"]Have you checked that the option has not been disabled in your User Control Panel >> Board preferences >> Edit posting defaults?[/quote]

I checked that before I said anything, it was defaulting to on. I set it to off, saved the settings, set it back to on, saved the settings again, and still no luck (even though it still indicates that it's on). Even tried flushing cookies.

Perhaps it's because I'm a lowly novice crafter (*snicker*) while you're a great and powerful moderator?

[quote="Sobuno"]The file's should be there now, they're from the night though, give them ~30 minutes to update though[/quote]

Confirmed that they're there now, and status.txt indicates that it's just a few minutes old. Looks like we're back in business, thanks! :)

User avatar
Sobuno
Developer
Posts: 2589
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Re: Data Export

Post by Sobuno » Sun Dec 30, 2007 1:02 pm

Holden wrote:
Zimoon wrote:Have you checked that the option has not been disabled in your User Control Panel >> Board preferences >> Edit posting defaults?
I checked that before I said anything, it was defaulting to on. I set it to off, saved the settings, set it back to on, saved the settings again, and still no luck (even though it still indicates that it's on). Even tried flushing cookies.

Perhaps it's because I'm a lowly novice crafter (*snicker*) while you're a great and powerful moderator?
Sobuno wrote:The file's should be there now, they're from the night though, give them ~30 minutes to update though
Confirmed that they're there now, and status.txt indicates that it's just a few minutes old. Looks like we're back in business, thanks! :)
Just below the textfield when you're posting a new message is a field named "Disable BBCode". Did you accidently check that one? (Try going back and edit your previous post for example :))

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests