Subscribe to this thread
Home - General / All posts - Drawing from Query Question
dchall8
465 post(s)
#12-Apr-18 16:01

I can create the drawing from a query; however, the location of the new drawing is wrong.

I have a drawing of contour lines made from DEMs. The DEM and drawing are in NAD83 / UTM zone 14N (ESPG:26914) and are centered at roughly 98 west and 29 north. From the contour line drawing I selected the lines below the lake level and created a new drawing from the query. In the process I selected the same projection for the new drawing. The new drawing looks good but is centered at roughly 103 west and 0 south. I did a rough measurement of the size and the scale of the query drawing seems good. It just sits in the middle of the ocean west of the Galapagos Islands. What am I doing wrong? Also when I try to zoom to the new drawing, it does not go there. Is that because it is a drawing from query and not from a fixed table?

adamw

8,037 post(s)
#17-Apr-18 10:55

I would guess the coordinate systems aren't the same - one coordinate system has local scales / offsets and the other does not. You can check it by inspecting coordinate systems using dialogs in 9.0.166.4 (they are now selecting current coordinate system values when you open them, the values you want to check are in the Metrics readout at the bottom). If there is a difference, you can edit the coordinate system of the misregistered drawing to use the correct values. Alternatively, you can copy and paste the correct coordinate system from one drawing to another (locate the table / query for the correctly registered drawing, open properties, locate the coordinate system for the geom field, right-click and copy the value to the clipboard, close the dialog, locate the table / query for the incorrectly registered drawing, open properties, locate the coordinate system for the geom field, right-click and paste the value from the clipboard, click OK).

Also when I try to zoom to the new drawing, it does not go there. Is that because it is a drawing from query and not from a fixed table?

Yes. The query does not report the bounding box of the data it returns, because that bounding box is expensive to compute. We will adjust the map window to go ahead and compute the bounding box on its own.

dchall8
465 post(s)
#17-Apr-18 15:57

Copy and paste for coordinate systems.

Who came up with that idea? ...and where do I go to shake his or her hand?

Dimitri


4,941 post(s)
#17-Apr-18 20:10

It is cool and useful, (like copying/pasting Syle values), but keep in mind copy/pasting the JSON for the coordinate system is basically an "assign coordinate system" or "repair coordinate system". It is not a "change coordinate system."

Suppose Drawing A, created from Table A, has the correct coordinate system assigned, some version of Orthographic. Suppose Drawing B, created from Table B, is supposed to be using that exact same version of Orthographic, but instead it somehow got Latitude/Longitude or Pseudo-Mercator assigned, so now it doesn't show up where it should.

In the drawing's table, the FieldCoordSystem.Geom property gives the coordinate system to be used for the Geom field, if that's the name of your geometry field. So, if you want to have Drawing B use the same coordinate system as drawing A, just pop open Table A's properties, copy the value of the FieldCoordSystem.Geom property, and then pop open Table B's properties and paste that into the value of FieldCoordSystem.Geom property in Table B.

Now, suppose Drawing B uses Lat/Lon and it has had Lat/Lon correctly assigned as the coordinate system to be used. Drawing B shows up in the right place, etc. You want to re-project Drawing B from Lat/Lon into the same Orthographic used by Drawing A. To do that you have to use the Change Coordinate System dialog.

If you simply copy the JSON string out of the FieldCoordSystem.Geom property in Table A, and then paste it into the FieldCoordSystem.Geom property in Table B, all that will happen will be that Drawing B will now have assigned the wrong coordinate system, since the actual numbers within the Geom itself aren't changed by that. Doing that is equivalent to launching Repair Coordinate System and saying "whoops... this isn't supposed to be using Lat/Lon, it's really in Orthographic...".

gjsa36 post(s)
#07-Jun-18 11:30

I have wondered about this after duplicating a table with an SQL query and ending up with projection problems.

To speak more generally though. As I understand M9, I should be able to start with a layer and change its initial coordinate system through the entire EPSG catalogue and then set it back to the correct initial coordinate system and the geom records will not have been affected and the layer will display correctly.

Here is an example of where this is not the case:

The tiff attached is projected in EPSG:3577.

When importing tiffs or shapefiles projected in this system, M9 tends to interpret this as a standard coordinate system (Albers Conical Equal Area) which it is not.

You will notice that the tiff imports and displays correctly with the wrong coordinate system. Not surprisingly (because it is displaying correctly), attempting to repair the initial coordinate system to EPSG:3577 results in incorrect coordinates as expressed by lat, lon in the bottom right status bar.

However, repairing the initial coordinate system back to Albers Conical Equal Area does not return the coordinates to the initial imported state.

So there are two issues here:

1. Is M9 allergic to EPSG:3577 on import?

2. Why is the repair initial coordinate system unsuccessful in this example?

Attachments:
vegfuelbase_a.tiff

adamw

8,037 post(s)
#07-Jun-18 12:07

Try 9.0.167.3. It seems to put the image where it should be when overlaid with something like Bing Maps, so unless you are talking about small differences which have to be measured, it seems that the wrong projection in this case was a bug that we managed to fix.

The projection is reported as Albers Conical Equal Area because it is that, but that's not all it is, there are specific parameter values. Key parameters in this case are center lon/lat (132, 0) and two standard lats (-36, -18).

Switching to EPSG:3577 switches to Albers Conical Equal Area with the same parameters, so if I do this, the image stays in place (which seems to be the correct place).

Switching to Albers Conical Equal Area in the Standard tab switches to Albers Conical Equal Area with the default parameters: center lon/lat = (0, 0), two standard lats = (30, 60). This makes the image go to the wrong place, because the parameters are not what they should be. (When you are switching 'back' to Albers Conical Equal Area in the Standard tab, this is not switching back to what the projection was, this is switching to a different variant of the projection.)

gjsa36 post(s)
#07-Jun-18 17:38

Thanks. Comments noted but I am seeing different behaviour with 9.0.167.3.

If you repeat these steps in this order as follows:

1. OK: Import tiff. Displays and locates correctly. Coordinate system noted by M9 is Albers Conical Equal Area.

2. NOT OK: "Change Coordinate system" initial projection to 3577 results in "Operation failed." (see attached screenshot).

3. NOT OK: "Assign initial coordinate system" to 3577 distorts coordinates (completely in wrong place and wrong scale, not a subtle difference)

Perhaps Step 3. is a logical but unintended result. But at this step, there seems to be no going back to the initial import success by reassigning initial coordinate system or changing coordinate system.

Now for another expression of the apparent problem:

1. OK: Import tiff. Displays and locates correctly. Coordinate system noted by M9 is Albers Conical Equal Area.

2. Export image back to tiff

3. Load exported image in QGIS - reports "no coordinate system assuming WGS84"

4. Set initial coordinate system in QGIS to 3577

5. Zoom to image and it is located about 9400 m north of where it should be with some distortion.

Attachments:
Capture.JPG

Dimitri


4,941 post(s)
#08-Jun-18 06:40

I think there are a number of flaws. Let's start with...

1. The first one arises from trying to be helpful:

Displays and locates correctly. Coordinate system noted by M9 is Albers Conical Equal Area.

As can be seen by dragging and dropping the image into a map with "known good" layers and projections, it appears correctly. The projection has, indeed, been correctly read from the GeoTiff and has been assigned. Manifold is telling the truth when it shows the coordinate system in black to indicate it has reason to believe it has been correctly assigned.

One problem is using the short hand expression "Albers Conical Equal Area" to describe the coordinate system. That misleads people who do not understand that the phrase does not refer to one, and only one, coordinate system, but instead refers to a family of coordinate systems where each specific coordinate system is characterized by the settings used. Yes, they are all based on the same Type, that is, a type of "Albers Conical Equal Area", but what specific coordinate system is used varies dramatically based on parameters such as the Center lat and lon, the standard latitudes, the Metric parameters and so on.

To convey that more accurately, one would have to say, "Oh, no, it's not Albers, it is...":

{ "Axes": "XY", "CenterLon": 132, "Eccentricity": 0.08181919104281579, "FirstStdLat": -36, "LocalOffsetX": -1081401.8465999817, "LocalOffsetY": -2576227.7391999997, "LocalScaleX": 250, "LocalScaleY": 250, "MajorAxis": 6378137, "Name": "Custom Coordinate System", "SecondStdLat": -18, "System": "Albers Conical Equal Area", "Transform": "Molodensky-Badekas", "Unit": "Meter", "UnitShort": "m" }

There. That's better. :-) The Component panel and various dialogs would have to get a bit bigger, of course, to squeeze all that in.

Now, I don't know what, exactly, the GeoTiff reports about the projection it uses. I haven't looked inside it. If it reports "Albers" with parameters, the above is what Manifold should, indeed, import and use. In the real world where people don't like to have coordinate systems described as full JSON specifications I think it is an OK compromise to report it as Albers Conical Equal Area, and to educate people that such a projection involves paying attention to specific parameters.

Or, perhaps instead of saying "Albers" it says "Custom Albers Conical Equal Area", prepending the word "Custom" to help remind people that the default case of an Albers centered on a point off Africa is rarely met.

What I think it should not do is that if the GeoTIFF reports an Albers configuration that is an exact match to EPSG:3577 but it does not actually report EPSG, then Manifold should not use the EPSG terminology. That would just confuse people who may have written out the GeoTIFF using some other GIS package where they carefully specified Albers with a particular set of parameters, but where they did not use an EPSG code and don't even know what that might be.

In contrast... if the GeoTIFF reports what it uses as EPSG:3577 then I think that's what Manifold should report. If it doesn't, that's a mistake if Manifold tries to be helpful and uses a standard name to make an otherwise cryptic EPSG code more understandable. It should just report the EPSG code if that is what the GeoTIFF says it uses. The question is... what does the GeoTIFF file itself report about what it contains?

Which reminds me of the old anecdote: "What's the difference between a used car salesman and a GeoTIFF? .... the used car salesman knows when he is lying." That's on-point because one often encounters GeoTIFF files that don't describe what they contain accurately, or which contain tags that contradict each other. The classic example is described in the essay on That YX Thing, where a GeoTIFF declares it is using a EPSG code that requires Y axis first but then in fact the data contains X axis first.

So, it could be in one tag the GeoTIFF says it is using the EPSG code and in other tags it is saying something else. That's unlikely, but if it is happening it sets up ambiguity, so you have to consider the possibility. I don't think that is what is going on but there is one bit of information that raises the issue. Which brings me to...

1a. The datum read by 9 from the file is WGS84. Technically, that is a different datum than GDA94 used by EPSG:3577. For virtually all practical purposes they are identical, being within a meter of each other. I don't know how Manifold uses the latest EPSG database to reckon this, if it just takes WGS84, or if, when reading a specific EPSG code it takes what is often reckoned to be an alternate name for the same thing and uses that. But it is a hint that what is in the GeoTIFF might - and I emphasize "might" - not be a pure and simple EPSG tag. Something to explore.

2. Looks like a simple case of trying to re-project something into itself throws an error. Manifold should guard against that.

3. Likewise, trying to "assign" to itself using a different name throws an error. Manifold should also guard against that, as well as against any possible errors caused from a simplified Albers description that is re-assigned into the same thing but without preserving Metric parameters, etc.

Now for another expression of the apparent problem:

That's hard to say if it is the same problem or a different problem. Exporting the image from within Manifold to a new .tiff and then re-importing the image back into Manifold from that .tiff shows it in a) slightly different location and b) wrong palette exported (if one has colored it with a palette, as I did). That's unacceptable, of course.

That's all routine stuff to be sorted, which it will be.

What's more difficult to sort out is to figure out what was in the original GeoTIFF and how that can best be presented. There should be no bugs, of course, but beyond that one has to deal with contradictions and different styles of saying the same thing, all of which must be consumed and presented in a way that is reasonable.

I say that if the original GeoTIFF cites, unambiguously, a EPSG code that is what should be reported. If the original GeoTIFF says both an EPSG code and other stuff, then the EPSG code (presumed to be the most rigorous) should rule and be cited. If, on the other hand, the original GeoTIFF cites a bunch of tags that describe a customized Albers projection, then I think it would be good to report that as "Custom Albers".

adamw

8,037 post(s)
#08-Jun-18 09:07

The error with 2 - change coordinate system to 3577 - is a known issue, we will take care of it.

I am not sure I understand 3 - assign initial coordinate system to 3577. If I (a) import the image, (b) make a copy of it (select both the image and the table, click Copy in the Project pane toolbar, then click Paste in the same toolbar, this will create the second pair of image and table), then (c) open the copy (have to rebuild intermediate levels for the image by pressing F7 and clicking Update Data) and Repair Initial Coordinate System to EPSG:3577 (click Ctrl-1 to go to Contents - Component, click the button in the coordinate system picker, select Repair Initial Coordinate System - Edit Coordinate System, switch to the EPSG tab, type '3577' in the filter box, select 'GDA94 / Australian Albers (EPSG:3577)' in the list, click OK), both images line up. Do you do something different?

We'll take a look at the export as well.

gjsa36 post(s)
#15-Jun-18 01:24

Thanks kindly everyone for the detailed replies. I will take everything into account and provide a helpful reply as soon as possible.

gjsa36 post(s)
#24-Jun-18 09:38

Ok this isn't a thorough summary by any means of this thread - but what I have found in terms of a successful workflow generally when importing TIFFs from different sources - given that I use M9 for vector processing not image processing:

1. Import image, and as noted above, the coordinates are accurate when imported. It is only when reprojecting the imported image that the result may rarely be different to what was expected.

2. Trace Areas -> the layer is now a vector layer.

3. All conversions of vector layers based on the original image are 100% reliable.

gjsa36 post(s)
#24-Jun-18 11:15

Footnote to 1. above:

A Assigning the initial coordinate system to the technically correct EPSG:3577 results in misalignment (image moves to the northern hemisphere from the southern). Therefore, initial projection must be accepted as the less-specific "Albers Conical Equal Area".

gjsa36 post(s)
#24-Jun-18 11:26

( A late reply to AdamW's comment on 08-Jun-18 09:07:

Have followed/checked your steps exactly using 9.0.167.4 and the copied image doesn't line up as per the footnote above.

After assigning 3577 to the copied image and placing both the copied image (set to 3577) and the original imported image in a map: the copied image is in the northern hemisphere, and real world extent of the image has shrunk.

But having said all that, I have a workflow that is ok. I am posting this detail only to try and clarify approaches with this imported image that can lead to unexpected results. )

adamw

8,037 post(s)
#25-Jun-18 15:18

OK.

I understand that you have a workflow that produces the desired results, but for completeness, I re-checked the workflow that I gave above and I cannot see an error. It might make sense to try re-tracing the steps because we are probably doing slightly different things.

Here is what I do:

Create a new MAP file. Import the TIFF. (This creates a table named 'vegfuelbase_a Tiles', an image named 'vegfuelbase_a' linked to it and a comment.) In the Project pane select both the image and the table. In the Project pane toolbar, click Copy and then Paste. (This creates a table named 'vegfuelbase_a Tiles 2' and an image named 'vegfuelbase_a 2' linked to it.) Open the copy of the image ('vegfuelbase_a 2'). Invoke View - Messages, then click Update Data to build intermediate levels. Go to the Contents - Components pane. (The coordinate system is displayed as 'Albers Conical Equal Area'.) Click the button next to the name of the coordinate system, select Repair Initial Coordinate System - Edit Coordinate System. (The dialog opens in the Custom tab, lists 'Albers Conical Equal Area' as the system, lists parameters, lists metrics at the bottom as '250 m, [ -108..., -257... ]'.) Switch to the EPSG tab. In the filter box, type '3577'. (The list of systems is filtered to just one system.) Select 'GDA94 / Australian Albers (EPSG:3577)' in the list. (The definition of the system lists the system as 'Albers Conical Equal Area' with a name set to 'GDA94 ...', the metrics readout does not change and still shows '250 m, [ -108..., -257... ]'.) Click OK to apply the change (the extent of the change is a new name, the parameters and the metrics - which I assume is what gets lost in your case - are the same). (The coordinate system is now displayed as 'GDA94 ...'.) Drop the original image into the window. Try toggling the layers and they appear to be in the same place.

Sorry for a wall of text. It's just that in this case it's probably some small detail that is different between what we are doing and that's why we are getting different results. If you have some time, try repeating the steps above and if at any point the results are different from what I describe, please let us know.

gjsa36 post(s)
#09-Jul-18 23:12

Ok Adam thanks for the detailed info. I just checked your process and have found the problem at my end.

However, I think it is a bug and probably should be reported if you can confirm it:

  • Your method (which works):
  1. Copy and paste image and table
  2. Invoke View -> Messages, then click Update Data to build intermediate levels.
  3. Contents pane -> Repair Initial Coordinate System
  4. Edit Coordinate System
  5. Search for EPSG 3577 and apply
  • My method - which fails to align the two images:
  1. Copy and paste image and table
  2. Invoke View -> Messages, then click Update Data to build intermediate levels.
  3. Contents Pane -> Repair Initial Coordinate System
  4. Select EPSG 3577 from the favourite drop down list:

My immediate thought is, maybe the parameters for EPSG 3577 in the favorite list somehow were incorrect when added some time ago? But I just deleted EPSG 3577 from my favourites and re-added and the same problem occurs: the original imported image ('Albers Equal Area') and this repaired image are not aligned.

I am using M9 167.5. If you have time could you add EPSG 3577 to your favourite list and see if you encounter the same issue?

tjhb

8,168 post(s)
#10-Jul-18 00:23

It's not a bug.

A favourite coordinate system includes all parameters, including metrics.

If you apply a favourite coordinate system to an image, that will almost always be a mistake, because metrics are not usually shared between images, unless they have the same pixel resolution and the same extent (strictly, the same lower left pixel).

Generally don't assign a favourite coordinate system to an image (unless you are really sure that metrics also match).

Instead, when necessary, use [Repair Initial Coordinate System] > Edit Coordinate System. This preserves known metrics, unless you change them.

Favourite coordinate systems are more suited to use with vector data.

tjhb

8,168 post(s)
#10-Jul-18 00:45

P.s. you are not alone.

In my opinion this is a serious UI design issue, which will waste large amounts of user time unless it is adjusted.

You might like to see this thread, including the comments around my post here. I still agree with my comments and disagree with Dimitri.

Make your view known!

But to repeat, it is a design issue, not a bug.

gjsa36 post(s)
#10-Jul-18 01:22

Agreed and thanks. I have wasted others' time here by discussing a coordinate system issue that was indeed an issue with metrics.

1. Choosing an EPSG from the list does not reset metrics, but offers the ability to edit them.

2. Choosing an EPSG saved as a favourite resets metrics

tjhb

8,168 post(s)
#10-Jul-18 07:16

You raised an important question, no waste of anyone's time.

I predict that this issue will continue to be raised in various forms until the UI design is changed.

Very similar to the long-standing problem exporting vector layers with local offsets and scales from Manifold 8. Now avoided in 9 by better design.

But your summary is perfect and useful.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.