Subscribe to this thread
Home - General / All posts - 168.12 mysterious results of Edit-Merge-Merge Images
KlausDE

6,321 post(s)
#30-Apr-19 22:14

I have imported two images with accompanying MFD8.xml files that share the same coordinate system but have different pixel size (=LocalScaleX/Y) and dropped them in an empty Map. I then ran Edit->Merge->Merge Images different times and got surprising effects.

  • Color Channels R and B got switched in the result in this example, not in others!
  • The result lines up correct but misses tag LocalSaleX/Y in FieldCoordSystem.Tile and sometimes the result is very coarse and shows only dense parts of the original images.
  • BTW each time I open Mfd9 form scratch (i.e. not clicking a map-file) I have to activate Contents pane.

Attachments:
merge_images_test.mxb

tjhb

8,771 post(s)
#01-May-19 02:53

Klaus,

Color Channels R and B got switched in the result in this example, not in others!

The difference between images 1 and 2 and the merged image is not in the channel order, but only in the Styles applied. Images 1 and 2 have RGB interpretation, while the merged image has BGR.

I think this is just because BGR is the Manifold 9 default, applied to any new image. But when two or more RGB images are combined, it might make better sense to give the merged resuklt RGB interpretation as well?

At any rate, restoring RGB interpretation to the merged image here makes things normal again.

(More coming.)

tjhb

8,771 post(s)
#01-May-19 03:30

The result lines up correct but misses tag LocalSaleX/Y in FieldCoordSystem.Tile and sometimes the result is very coarse and shows only dense parts of the original images.

Observations

  • Image 1 has local scales of ~0.026m (X=Y).
  • Image 2 has local scales of ~0.057m.
  • The map has local scales of 1m.
  • The merged image in the example has local scales of ~0.057m.

I assume you want the merged image to have the finer resolution instead, matching image 1.

The notes below explain (I hope) the variations you are seeing.

Workflow variations and results

(a)

Select image 1 alone and create a new map. The map gets the projection (including local scales and offsets) of image 1. Add image 2 to the map and choose Edit > Merge > Merge images. The suggested output resolution is ~0.026m. No change needed.

(b)

Select image 2 alone, create a new map, add image 1. The result is the reverse. The suggested output resolution for the merge is ~0.057m, but you can overrride that to match image 1 (~0.026m), using Edit Projection then Edit Metrics from within the Merge dialog.

(c)

Create an empty map, with no layers. Unless otherwise specified it takes the default projection for new components (set in Tools > Options), usually with implicit local scales of 1 and local offsets of 0.

Now add image 1 to the map. The map's projection is changed to match image 1. Add image 2, then merge. The suggested output resolution is the same as in (a), no change needed.

(d)

Create an empty map as in (c). This time add image 2 alone, then image 1. The suggested output resolution is the same as in (b), but again it can be overridden in Metrics.

(e)

Create an empty map again, then add images 1 and 2 together. This time the map keeps its initial projection, because the projections of the layers added together do not exactly agree (including in their local scales and offsets). Assuming the projections of the images are compatible with the projection of the map (not guaranteed; otherwise the images will not be visible)...

This time the suggested resolution for the merge will be 1 unit, matching the assumed local scales of the map. In your example that is much coarser than either source image. In other cases it may be finer, or the same. Again it can be overidden. If it is not, then local scale parameters will be absent for the output image, because the values are just 1.

tjhb

8,771 post(s)
#01-May-19 08:50

I left out probably the most obvious thing.

(f)

Select both image 1 and image 2 and create a new map. As in (e), the projections of the layers added together do not exactly match (including in their local scales and offets). So unless otherwise specified, the map is given the default projection for new components (from Tools > Options), with implicit local scales of 1 and local offsets of 0, essentially the same result as in (e).

That may not be what is wanted.

The easiest way to fix this is probably to do (a) or (c) instead, i.e. add a single "key" layer with the target projection, then add other layers. But we can also use Edit Query from the New Map dialog, and adjust the auto-generated text.

When the initial layers have projections that don't exactly match, the auto-generated query sets the map's 'CoordSystem' property to a JSON string representing the default projection for new components.

We can replace this string with a function returning the same for the layer whose projection we actually want to use. For Klaus's data,

-- $manifold$

--

-- Auto-generated

-- New Map

--

CREATE MAP [Map 2] (

  PROPERTY 'CoordSystem' ComponentCoordSystem([1 Image])-- adjusted

  PROPERTY 'Item.0' '{ "Entity": "[1 Image]", "Z": 0 }',

  PROPERTY 'Item.1' '{ "Entity": "[2 Image]", "Z": 1 }'

);

Now Edit > Merge > Merge images will suggest the resolution of the specified image layer. No override needed.

adamw


8,584 post(s)
#01-May-19 10:48

Tim laid most things out already. In short, all of this is expected behavior although we could perhaps make a couple of things better.

A couple of notes:

Local scales

Images in the MXB have different pixel sizes. The merge dialog has to select a single pixel size, because it produces a single image. Initially the merge dialog uses the same coordinate system as the map it opens on, but that could be changed. The dialog displays current pixel size as well as the estimated number of pixels in the resulting image above the list, it makes sense to check that prior to performing the merge.

If we open Map, and start the merge dialog on that (View - Merge - Merge Images), the initial coordinate system is that of the map, and the pixel size is 1x1 m. That's too coarse, and it's seen that that's too coarse, because the estimated number of pixels in the resulting image is 40x40. It is also seen that both images are going to be projected during the merge, and that this projection is going to be more than just shift / scale - this is a little of a red herring, because the projection is going to be very close to shift / scale, the culprit is the eccentricity value: the value used by images is very slightly different from the value used by the map.

You can right-click the first image (2 Image) and select Use Coordinate System. This will set the pixel size to 0.057 m and the estimated number of pixels for the resulting image will become 678 x 681 (much better than 40 x 40). The clicked image will hide the coordinate system icon in the list, indicating that it is not going to be projected at all (just shifted, but that does not alter pixel values, merely moves them around in tiles). The remaining image will show the gray coordinate system icon in the list, indicating that it is going to be reprojected, but that the reprojection will be limited to scale / shift, nothing curvilinear.

You can also right-click the second image (1 Image) and select Use Coordinate System for that. This will set the pixel size to 0.026 m, and make the estimated number of pixels for the resulting image increase to 1,505 x 1,510. This is probably the setting you want to use for the merged image.

We think the above works fairly well already. Maybe we might add some sort of a confirmation for when the resulting image seems way too small.

RGB vs BGR

Our images separate data from interpretation of data. Pixel values stored in tiles are data. Style properties direct how that data gets converted into colors for display. This could be a channel mapping or, say, a palette, where each pixel has a single number and you specify that 1 means red (255, 0, 0), 2 means green (0, 255, 0), etc.

The merge dialog operates on data, it ignores styles. If you try merging two images with three channels each, with one image using BGR and another using RGB, the resulting image is going to have a mix of B from the first image and R from the second image in the first channel, etc. Because merging is done on data.

We merge data because that's what is mostly needed. If you want to merge images that use different channel arrangements, you are supposed to take care of that first. Most frequently, the merge is done on a series of images that are similar to each other, so this is a non-issue, but more importantly, we do not want to be reshuffling channels based on temporary rules, because that loses data and makes the operation hard to reason about and use.

That said, we can have an option to merge not data, but rather colors. That way, we would render all images to RGBA applying the rules specified in styles the same way we do when they are displayed, and we would produce a single RGBA image. This is going to lose all floating-point values, etc, but the resulting image would be consistent in terms of display - which is useful. We might add such an option in the future.

Hope this helps a bit.

KlausDE

6,321 post(s)
#02-May-19 07:27

Thx Tim and Adam,

Help covers this aspect in depth and repeatedly, I should have been prepared.

But I didn't recognized that the test images (created in Mfd8) had unexpected order of channels automatically interpreted as intended on import.

I support the clear distinction "Our images separate data from interpretation of data." But for images that is a unusual concept for the ordinary windows user used to pictures and not to raster data in general.

So where would I expect reminders of possible unexpected effects?

In case of channel interpretation the automatism for interpretations on import/link seems to be quite reliable and the surprise arouses from later processing only. Merge Images is only one affected function and one that would allow a warning as it's controlled by a dialog. A reminder for mixed interpretation of input data or a difference between input and default output interpretation would be helpful there.

But what about other (SQL-)functions that come without dialog? So the reminder may as well come on import of those images, preferably additionally and with an opportunity to reshuffle the data on click. Linked data are a problem though.

The default choice of resolution is a Merge Images thing only, I guess. The default is reported in the dialog. So the red exclamation mark we know from other situations like a missing index linked to a warning would be really helpful.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.