Subscribe to this thread
Home - General / All posts - Multiple File Conversions
Rakau75 post(s)
#09-May-19 10:57

I have many thousands of files going back about 20 years. They have been created in Fugawi system as TRK track files

Each has been saved as a single track enclosing an area, with the name of the file as the date, weight and area ( see sample)

There is no data inside any of these files.

Client now requires to be able to evaluate harvest trends over time.

Using GPS Utility, i can convert a single file, create new columns in manifold and manually insert data.

My question: Is there a way to batch extract the name of each file and place that data inside a converted drawing ( something for Manifold to open and read, such as GPX or shp?) Open to any ideas here.

We do need to retain the track boundaries, but could load data onto a centroid point for some form of thematic visualization

Thanks in advance

Attachments:
FileList.PNG

adamw


8,584 post(s)
#09-May-19 12:08

I'd dump the filenames into a text file from the command line: DIR /B /S *.TRK > list.txt

(/B removes headers, /S recurses into subfolders, *.TRK tells which files to list.)

Then I'd import the resulting file as a table and used transforms / SQL / scripts to extract X / Y into separate fields.

Then convert X / Y into points. In Manifold 8, that could be done by linking drawing from a coordinate-coded table, in Manifold 9, the process is slightly different but similar.

Rakau75 post(s)
#09-May-19 18:29

Thanks. Will have a crack at this. All new territory for me

Rakau75 post(s)
#09-May-19 19:11

Have extracted the list OK - went well thanks.

Manifold 9 will not open the *.trk files (This is a Fugawi format) or am I missing something. Tried Import > FileName

Have included a file sample

Could these be converted through some script ?

Thanks

Attachments:
07.9.04 1609kg 268m2.TRK

adamw


8,584 post(s)
#10-May-19 17:09

I am a little confused - you said previously that there was no data inside any of these files, no? I thought all info was in the filename (although I now see that it contains date / weight / area, but no location data)?

We cannot import these files neither in Manifold 8 nor in Manifold 9, we do not support the format. Is there a description of this format available somewhere? I might be seeing the tip of an iceberg here, but so far it looks simple enough to be read using a script (in 9 at least).

Alternatively, can the GPS utility that you are using to convert a single file (what is it?) be run from the command line and can it maybe produce something that Manifold 8 or 9 read? If so, let's just convert all these files in the command line using a simple batch file, then batch import everything into Manifold, then merge the results in Manifold using a script.

Rakau75 post(s)
#10-May-19 18:46

Thanks Adam.

Sorry for confusion. The original files were created in a mapping program called Fugawi. (https://www.fugawi.com/apps)

It no longer supports the software.

Client has used various GPS units to capture a harvest area over the past 20 years by taking various points off the craft and completing the area covered when close enough to the start position, then naming the file with date/weight/area ref/ etc. Some files have more info such as size etc.

Through lack of GPS/Mapping skills/knowledge they thought using all detail as a name uniquely identified the harvest catch, but have never been able to do any analysis of their work

We can convert the original *.trk file using an application called GPS Utility (http://www.gpsu.co.uk). From that, I can transfer to almost anything. I have used shp & gpx as experiment options.

Their web site has a batch file converter (released 2013) however I am unable to get that to work. My programming skills are very basic

The converted files are are able to be opened no trouble in Man 8 or 9, however once open all we have is the outline of an area in the form of lines (Not an area)

To try summarising, we have a file that can be converted that shows the geometry of several lines only, named after all the attributes that should be recorded within the file.

My request is in the help of being able to batch convert all these files into a readable file, that I can then link the extracted data from the file name ( as already solved yesterday)

Thanks

adamw


8,584 post(s)
#11-May-19 11:14

Thanks, this helps.

I played with this a bit and had some success with the batch converter / batch import into 9.

I installed GPS Utility (5.34, latest version available) and then GPS Utility Batch File Converter (1.3, latest version available). I then launched GPS Utility and it told me that I have to run it once as the administrator to register the interface for the batch converter. The message box had three buttons: OK (fine, remind me later) - Ignore (stop reminding) - Help, I pressed OK, relaunched the utility as the administrator, then closed it, launched it again as a normal user to see if the message box would now disappear, and it did disappear. I then launched batch converter and it reported 'Connected to: GPS Utility (5.34' at the top.

I set the controls in the batch utility to: Source folder = <my folder with several TRK files> (clicked Browse button and selected the folder, this populated the file list on the right), Source file type = TRK, Change grid to = LAT LON DM (was autoselected, I kept the selection), Change datum to = WGS 84 (was autoselected, I kept the selection), Target folder = Same as source folder (checked the box), Target type = SHP. The Status reported that I have 5 files to convert. I clicked Convert All and the converter went to convert them one by one seemingly launching the utility for each file.

I then launched Manifold 9, created a new MAP file, invoked File - Import, navigated to the folder with all files, set file type to SHP, then selected everything and clicked Import. This imported each SHP as an individual drawing + table, the tables had the same field layout.

Then I created a new map, dropped all drawings into that map, opened it, invoked Edit - Merge - Merge Drawings, and merged everything into a single drawing. Then set the coordinate system of the drawing to lat/lon (the conversion process produced SHP without accompanying PRJ files, so they got imported without a coordinate system, but I left the coordinate system unassigned during the merge as well, so no data was moved during the merge, and I was able to set the coordinate system for everything in the final drawing).

Rakau75 post(s)
#13-May-19 04:14

Thanks for all your help Adam

Have learned a valuable lesson with this..

I have a very small SSD as my C:\ drive. As such I limit any applications that I run from the C drive, and install most on my G: drive.

The batch converter was looking in the C: drive for the connection.

Have sorted it all out, and finally able to make headway

Again thanks for all your help.

Neill

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.