Subscribe to this thread
Home - General / All posts - Cannot import tiff image with correct projection M9
HMS18 post(s)
#22-Jul-19 16:16

I'm experiencing some issues when importing tiff images into M9. In M8 this process always worked and still works fine, but when importing the same image file (tiff) into M9 the projection is always off with all images appearing at the same place.

I tried importing/exporting with different file formats, including ecw (exported directly from M8), but the images always appear with incorrect size and location in M9, as showed in the attached jpg files (M8 and M9).

Any help would be appreciated.

Thanks!

Attachments:
M8.jpg
M9.jpg

HMS18 post(s)
#22-Jul-19 17:46

After a few more tries, the ecw opened successfully in M9. But I'm still without knowing why this happens still with the tif file. My most probable guess is that is something related with the tfw file. I'll examine further on this matter.

Dimitri


5,491 post(s)
#22-Jul-19 18:51

Without knowing more about your "tiff" files it's not possible to do more than guess. It doesn't help to show the results of wrong imports if you do not also provide details, like a list of all the files involved in the import, what the TIFF is supposed to be (a geotiff? not a geotiff?) and exactly how, step by step, you do the import.

But we can guess, so let's do that: You mention a world file, so that indicates at least part of the fileset may not think it is not a geotiff. (TIFF files which know they are geotiffs usually don't bother dragging along a world file.)

In general, 9 does a far better job of reading projection information from different formats than 8. Specifically, 9 does a better job of reading projections from GeoTIFF and also reading accessory files that may accompany TIFF files to specify projections.

That's great, except in situations where accessory files contradict each other. When accessory files contradict each other, the last one in priority wins. So... if only that part of the fileset which 8 can read is correct, while all the extra parts that 9 can read provide bogus information, 8 will harvest the right projection while 9, trusting what the fileset says about itself, will harvest the wrong projection.

If that is the problem in this case, the solution to is easy. In the short term, get rid of any accessory files that may be wrong, so there aren't any extra files that provide wrong information. For example, if you have a geotiff that contains correct coordinate system info, get rid of any world files that provide wrong information. 9 will read the geotiff and won't read bad info from the wrong world file. In the long term, try to ensure that any sidecar files that accompany the TIFF are accurate.

For accessory files Manifold will read to try to find projection information, see the TIFF topic as well as the Projections from Accessory Files section of the Projections topic.

HMS18 post(s)
#22-Jul-19 19:49

Thanks Dimitri for your explanation, even with the scarce information I provided. The tif file refers to official information provided by the Portuguese Military Geographycal Center (https://www.igeoe.pt/). I'm also providing a link for the download of the original raster with all the info provided with the tiff file and the M8 and M9 files: https://we.tl/t-OPAmF6mGVC

Here's the process in M8 and M9 when importing these images.

M8

235_2000_4.tif

File > Import > Image > TIFF Files > 235_2000_4

Edit > Assign Projection > SGHM (Sistema Militar, Datum Lisboa)

Cartograma.shp

File > Import > Drawing > Cartograma (PTM06)

Create map and add both files.

M9

235_2000_4.tif

File > Import > TIFF Files > 235_2000_4.tif

Repair Initial Coordinate System > Datum Lisboa IGP (System similar to Datum Lisboa)

Cartograma.shp

File > Import > Cartograma.shp

Coordinate System > EPSG:3763

The original SGHM (Sistema Militar, Datum Lisboa) was not available originally at M8. I tried to configure Datum Lisboa IGP to match SHGM accordingly with the specifications defined by DGT (the portuguese official authority in cartography: http://www.dgterritorio.pt/cartografia_e_geodesia/geodesia/sistemas_de_referencia/portugal_continental/

I hope this can be enough for any more help.

Dimitri


5,491 post(s)
#23-Jul-19 08:45

The problem is that the TIFF fileset does not contain adequate projection information for the TIFF, so you have to enter that manually. Enter it manually and it works perfectly.

This bit is misleading, by the way, in that it sends debugging off in the wrong direction:

I'm experiencing some issues when importing tiff images into M9. In M8 this process always worked and still works fine, but when importing the same image file (tiff) into M9 the projection is always off with all images appearing at the same place.

As it stands, the above makes it sound like the TIFF imports into 8 correctly, but not into 9. That is incorrect. It appears it would have been better to say "this TIFF doesn't import correctly into either 8 or 9, but I've come up with a procedure to get it georegistered into 8. Here it is step by step [...]. What would be the equivalent procedure in 9?"

Onward...

The TIFF does *not* contain projection information that either Release 8 or Release 9 can use. It does not import into 8 correctly and it doesn't import into 9 correctly. It also doesn't import using GDAL correctly.

The procedure given for a successful import into 8 seems to be not possible as written...

Edit > Assign Projection > SGHM (Sistema Militar, Datum Lisboa)

... as there seems to be no SGHM (Sistema Militar, Datum Lisboa) choice in the Release 8 Assign Projection dialog. So... what did you do in Release 8? If there is a SGHM choice in a Release 8 dialog, how do you get that dialog?

In the Release 8 map you provided (good move), the TIFF image uses a Transverse Mercator projection centered on a given lat/lon with a WGS84 datum with false easting/northing offsets and custom local scale and custom local offset. If you use the same settings in 9, that also works in 9.

In 9, get rid of the useless TFW and import the TIFF and choose Assign Initial Coordinate System, then specify a custom coordinate system using exactly the same parameters as used in 8. However, instead of using the WGS84 datum (referred to as the "Base") use the Lisbon EPSG:4207 datum. That's what the meta data says the coordinate system uses, so specifying a different datum could introduce significant errors.

Do that, and the image lines up perfectly (compare roads, waterways and railways), as compared to a background layer of Bing Streets:

In the image above you can see the military map does not include the A23 highway, the broad purple line toward the right of the map. Perhaps the military map was created before the A23 highway was built. Or, maybe in an austerity campaign they don't want to pay tolls to use the A23, if it is a toll road. (Just kidding :-).

I suggest a careful re-reading of all topics having to do with assigning projections/coordinate systems in 9, so you know how to assign local scale and local offsets, as well as the difference between a datum and a projection.

This, for example, is a mistake as a procedure in 9:

Repair Initial Coordinate System > Datum Lisboa IGP (System similar to Datum Lisboa)

The Lisboa datum is a datum, a base, not a coordinate system. You have to specify a coordinate system.

Attachments:
alignment.png

HMS18 post(s)
#23-Jul-19 13:45

Hi Dimitri, thanks for your time with this matter.

Indeed by forgoting to mention that I had the SHGM Datum availabe in M8 system through an xml with the Portuguese systems (C:\Program Files\Manifold System\Config), inadvertedly, I believe I also mislead you, since it was a simple use of the "Assign Projection" menu. Here's the xml data for SHGM:

<xml>

<preset>

<name>SHGM (Sistema Militar, Datum Lisboa)</name>

<category>National Grids*Portugal</category>

<system>Transverse Mercator</system>

<datum>Datum Lx IGP</datum>

<centerLat>3.966666666666666662965923e+001</centerLat>

<centerLon>-8.13190611111111039122079e+000</centerLon>

<scaleX>1.000000000000000000000000e+000</scaleX>

<scaleY>1.000000000000000000000000e+000</scaleY>

<falseEasting>200000</falseEasting>

<falseNorthing>300000</falseNorthing>

<localOffsetX>0.000000000000000000000000e+000</localOffsetX>

<localOffsetY>0.000000000000000000000000e+000</localOffsetY>

<localScaleX>1.000000000000000000000000e+000</localScaleX>

<localScaleY>1.000000000000000000000000e+000</localScaleY>

<unit>Meter</unit>

</preset>

</xml>

I was successful in reproducing your steps, but after further investigation, I found out that the EPSG code for this system should be 20790 (Datum Lisboa/ Hayford-Gauss with false origin - Coordenadas Militares). After choosing this one, adjusting the values and changing the base for ESPG: 4207 it worked perfectly (Image1.jpg attached). Thanks for that!

The drawing "cartograma" is the official grid that provides the number of this info for a designated area. The M9 mistake you correctly pointed out was indeed based on a false assumption that what I named "Datum Lisboa IGP" would match EPSG 20790. Attached you can find the print screen with the M9 Coordinate System values.

On a side note, this imagery is for sure older than the A23 road. A few years ago I believe there were no tolls in there, but now it is a toll road, for sure. Perhaps that's why the military geographic service didn't updated this one yet ;)

Attachments:
Coordinate System.jpg
Image1.jpg

adamw


8,634 post(s)
#23-Jul-19 12:29

In addition to what Dimitri said, I followed your steps and I think I found a bug in our code as well.

The TIFF file does indeed come without a projection in both 8 and 9, so while I didn't look into the file tags yet to check whether they are missing or are being misread (will do), it seems that the file might not contain projection data. Your steps for 8 include a step for adjusting the projection after the import. The problem I found is that the equivalent steps for 9 (a) suffer from a UI deficiency and (b) run into a bug.

The UI deficiency: when you alter the projection for the image by going to the coordinate system picker in the Components pane, if you use a favorite projection, the local scales / local offsets get overridden. This isn't a new issue, it is appearing and re-appearing in discussions, we agree we should fix it and we will try to fix it.

The bug: a reasonable candidate for the projection from the list of built-in projections seems to be EPSG:20790. However, it specifies the projection center not just via the center longitude parameter, but also via the prime longitude parameter (I am simplifying things, the prime longitude parameter is used for more). The bug is that we seem to be mishandling this and the value of the prime longitude parameter gets ignored. We will fix this.

Thanks for the report.

HMS18 post(s)
#23-Jul-19 13:48

Hi Adam, thanks for your input. I hope what I answered to Dimitri can be of any help in further clarification, but indeed the EPSG: 20790 should be the right projection that matches SHGM.

tjhb

8,805 post(s)
#23-Jul-19 14:00

The UI deficiency: when you alter the projection for the image by going to the coordinate system picker in the Components pane, if you use a favorite projection, the local scales / local offsets get overridden. This isn't a new issue, it is appearing and re-appearing in discussions, we agree we should fix it and we will try to fix it.

That sounds like good news. Do you already have some ideas for how?

For example, (a) would there be a downside to silently keeping non-default metrics (if present) when applying a favourite CS? That would leave the user to knowingly override if necessary with defaults (or a correction) as a separate step. I can't think of a downside to that, but there may be one.

Or (b) would it be safer to throw a confirmation dialog, e.g. "Override existing metrics with defaults?", or some better wording, in cases where non-default metrics are already set and a favourite projection (with default metrics) is applied?

And implicitly (c) should it be possible to store non-default metrics (that is, any metrics) in a favourite CS, or to apply metrics from a favourite? I can't think of a case where it is helpful to do so, but I might be overlooking something.

Lastly, should (a) or (b) be applied only where the chosen CS is a favourite--or also when any new CS is applied?

adamw


8,634 post(s)
#24-Jul-19 08:08

We are going to allow specifying for a favorite coordinate system whether or not applying it should overwrite local scales / offsets. This will be done in the dialog listing favorite coordinate systems. The default favorite coordinate systems will not overwrite local scales / offsets.

We considered several other approaches, here is one example:

Make the application of a favorite coordinate system not overwrite local scales / offsets, BUT show when the coordinate system has non-trivial local scales / offsets in the coordinate system picker (similarly to how we show that the coordinate system uses YX axes - via some text on the right), AND allow quickly resetting local scales / offsets to trivial values by using a new command in the drop down menu (Clear Metrics). We decided against this because this looks much more complex for the user than what we finally chose. (And because this does not allow applying local scales / offsets from the new coordinate system, just clearing them, but that could have been solved by reworking the command to either apply new metrics or maybe apply new metrics automatically and use the command to restore old metrics.)

Answers to (a)(b)(c):

(a) - silently keeping non-default metrics is fine for images, but less fine for drawings. Example scenario: you import a SHP and a corresponding TIFF, TIFF has projection, but SHP does not, so you open the image imported from TIFF, add its coordinate system to favorites, then open the drawing imported from SHP and apply the favorite - this applies local scales / offsets when it should not. Our current chosen approach will allow you to think about whether or not you want the local scales / offsets to apply when you are creating a favorite.

(b) - confirming whether or not to overwrite local scales / offsets was one of the things we discussed, the problem with this is that this feels like the program asking for options for the operation after the operation is done instead of when it is done. You already "completed" the operation mentally by choosing a favorite, your mind is moving on to other things, but the program is grabbing you by the hand and going "wait, wait, wait, so could you clarify that additional part please?". It just flows differently. It is very likely that the user will press *something* to make the dialog go away and then forget what it even was. Confirmation dialogs aren't a good way to specify additional parameters for the operation.

(c) - storing non-default metrics in a favorite coordinate system could be useful, why not. When you want to copy the coordinate system from one image (the copy of the image in which Bob set the coordinate system for you) to another (the copy that you have) and have it be exactly the same, you want to copy the metrics.

Should (a) or (b) be applied only when the chosen coordinate system is a favorite - all of the above is talking only about applying a new coordinate system on top of an existing one by picking the new coordinate system from a drop down menu without going through the coordinate system dialog. If we are going through the coordinate system dialog, it has explicit controls for metrics, there is already logic that keeps / overwrites the metric, it's all visible, etc.

tjhb

8,805 post(s)
#24-Jul-19 08:51

Thanks for explaining so thoroughly.

The first and last paras cover it perfectly I think. They seem 100% correct to me.

Some of your answers to (a) and (c) I could disagree with--on the grounds that favourite projections should not be misused as temporary storage (that is what the clipboard is for, along with SQL)--but I think the answer back would be that you have to design for expert and novice users alike, which is right.

I think what you are planning does exactly that.

danb


1,704 post(s)
#23-Jul-19 20:13

'The UI deficiency:'

It would be great if this behavior changed to retain the local scales / offsets


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


5,491 post(s)
#24-Jul-19 07:45

And then there would be a different UI deficiency, whereby people carefully and diligently specify favorites, apply them, and then they would be surprised to find out the system did not do what they commanded, applying all parameters as specified, including local scales / offsets, but instead did what it thinks they should have commanded, apply all parameters except the local scales / offsets they specified.

Sometimes people want to specify local scales / offsets and sometimes they want to specify everything else but leave those alone. The system should allow the easy specification of either way when creating a new favorite.

I think the best way is for people to be able to define favorites with an option to override or retain any local scales / offsets. That way, if they want their favorite to override, they can do that when they define the favorite (call it "Portugal Military Override"), and if they want their favorite to retain any local values if they exist, they can define the favorite to do that (call it "Portugal Military Retain"). Thereafter they can pick with one click whichever one they want in a given situation.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.