Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System 9.0.165.6
adamw

8,037 post(s)
#22-Mar-18 18:25

9.0.165.6

manifold-9.0.165.6-x64.zip

SHA256: 99872b7243e6d7657213e7bd450937129fc44f3a6b202fe56337612f577960dc

manifold-viewer-9.0.165.6-x64.zip

SHA256: a41c22737bf20eb0b1c5555971cc061b28f5662cdd64628b74282fedb231a4a2

adamw

8,037 post(s)
#22-Mar-18 18:26

Changes

Composing contours removes degenerate connections.

(A note: we analyzed differences between contours produced by 8 and 9. Some of these differences were related to 9 producing degenerate connections that 8 was not producing. Although these degenerate connections contained data which would be helpful if one was to restore the original raster from the contours, we decided to remove these connections from the final output of the current contour functions, as above. All other differences were related to the handling of the height range: 8 separates pixels with values exactly equal to the contour height from pixels that are lower, while 9 separates them from pixels that are higher. Neither choice is better than the other, each choice will at times produce geometry that appears smoother than the opposite choice depending on the pixel values. The difference usually appears on integer rasters. Such a raster will usually contain patches where the specific choice worked 'better' than the opposite choice as well as vice versa. The difference between the choices is commonly reconciled by shifting the contour heights to be non-integer. We decided not to sync the behavior of 9 to 8 in this case. The main reason is that we have other ideas on how to handle flat areas for the future which gets better results in all cases.)

Composing contours and tracing areas removes redundant coordinates on horizontal and vertical edges.

The TileContourLinesXxx, TileContourAreasXxx, TileTraceAreasXxx query functions include an additional decompose parameter to specify whether the resulting geometry has to be decomposed into shapes. Contour Lines, Contour Areas, Trace Areas templates in the Transform pane set this parameter on by default.

Decomposing geometry as part of building contours / traces uses a specialized algorithm which performs much faster than the regular algorithm used in the Decompose to Shapes transform. The difference is huge (15x+ and even higher after numerous optimizations). Decomposing geometry as part of building contours / traces helps avoid producing geometry which is too large to fit into 2 GB (~120 million coordinates), which is the limit for the size of a single geometry value. All reports of contours / traces failing with the 'Invalid data buffer size' error were related to geometry values hitting this limit.

The Similarity parameter for the Trace Areas template in the Transform pane is set to 1 by default. (1 produces the same result as 0 on integer rasters, and using 0 on floating-point rasters is frequently a user error.)

Composing contours and tracing areas reports detailed progress and allows canceling operation in all phases, including decompose.

The image library dataport allows using escape sequences for X, Y, etc, in names of intermediate folders in addition to the filename. If the file pattern includes escape sequences in names of intermediate folders, the value of the 'Recurse into subfolders' option is ignored.

(Fix) Exporting SHP with Z values no longer produces malformed data.

(Fix) Reading DBF larger than 4 GB no longer sometimes stops before reaching end of data. (Many packages do not support DBF files larger than 4 GB and do not produce them. These files can, however, appear as a result of export from 9.)

End of list.

tjhb

8,168 post(s)
#22-Mar-18 22:11

Contours look great now. Very clean--no artefacts at all. And no slower!

adamw

8,037 post(s)
#23-Mar-18 08:41

On that:

With decompose off, the performance of contours / traces should be more or less the same as in the previous build. On some tests the new build is a little faster, on others a little slower, but the differences are always small (several percent is typical). Tracing is usually a little faster (because there is a significant number of redundant coordinates which collapse and do not participate in further analysis), contouring can go either way.

With decompose on, things take longer. However, if we compare doing contours / traces + automatic decompose in the new build vs doing contours / traces + manual decompose in the old builds, there is just no contest. The new build wins head and shoulders. (And comparisons to 8 are even better.)

Also:

We are a little worried about still running into the limit on the number of coordinates in a geom even with decompose, when trying to contour / trace a super-huge image. 120 million coordinates is a lot, but with some images things edge close to it. We will likely allow splitting lines that get larger than that into parts - if decompose is on. We could be splitting areas as well, producing artificial edges inside areas of the same height range, but that's more difficult (although certainly doable).

Dimitri


4,941 post(s)
#23-Mar-18 09:39

Very clean--no artefacts at all.

I agree, in that I personally prefer a cleaner, prettier presentation. But I wouldn't use the word "artefacts" to refer to single-line features, like spikes. That word has a connotation that such stuff is artificial and does not reflect what is in the data when the situation is exactly the reverse: those single line contours are there because that is what the data requires in that spot. Get rid of them and you cannot do the reverse transformation to get the raster back.

I don't really take a purist approach to absolutely showing everything that is in the data in a way that allows a reverse transformation (although many people do). I figure the whole idea of converting rasters into vector contours is based on a willingness to approximate, given the fundamentally different nature of vectors and rasters. So if you are going to do that, may as well approximate in a way that delivers an aesthetically pleasing result even if it is not mathematically reversible. :-)

tjhb

8,168 post(s)
#23-Mar-18 13:36

I agree too. “Visual artefacts” might have been better—though it’s hard to find a really good word.

I like the way the judgement has been made (where the line has been drawn). The only reasons I have wanted to “round trip” contours in the past (interpolating back to DEM) have been (a) as a QA/sanity check or sometimes (b) so as to intentionally create a heavily smoothed surface, with visual “noise” removed. (For the (b) I would separately make odd-multiple contours and even-multiple contours, interpolate a DEM from each, then blend.)

My point there is only that I can’t think of a good reason to try to round-trip contours back to a DEM while hoping to preserve full local accuracy. Contours remove (most) data in favour of a characterization, one that the eye can follow intuitively and naturally—which the feet could theoretically follow too. Accuracy is important, but it’s better that “perfect” accuracy should not get in the way of the main purpose.

Maps4planners10 post(s)
#23-Mar-18 13:25

I've been using the Trace Areas transform on some viewshed data created in QGIS and it is working fantastically. To covert the raster viewshed in QGIS to a vector takes forever but it is only seconds in Manifold. Thank you!

I'd like to be able to do viewsheds in Manifold one day!

mdsumner


4,204 post(s)
#23-Mar-18 13:50

Viewsheds could be relatively easy for a Radian expert with the constrained triangulation engine. Would be keen to explore this


https://github.com/mdsumner

Maps4planners10 post(s)
#23-Mar-18 13:56

Something I have noticed is that when I export the traced component it seems to save it using the manifold native coordinates. The original raster was using British National Grid so I was expecting it to be using the full national grid system in the export. When I open the shape file in QGIS or MapInfo it positions it using the Manifold native coordinates.

Attached is the file containing the resulting trace. This positions the viewshed at 3524,3590 (somewhere in the Atlantic) when it should be around about 496988,177046 (Windsor, UK).

Perhaps I'm doing something very silly but I've tried changing all the options I can see. I'm new to Manifold though!

Attachments:
WindsorViewshed Tiles Trace Areas Drawing 2.zip

adamw

8,037 post(s)
#23-Mar-18 14:14

The coordinate system of the drawing produced by Trace Areas coincides with the coordinate system of the image (= pixels). For the export, it would make sense to project the drawing to the same coordinate system but without shifting and scaling left over from the image (forcing local scales to be 1 and local offsets to be 0). Currently, the process is clumsy: the UI does not allow you to just remove shifting and scaling from the existing system, you have to pick the unshifted / unscaled system from the list. We will make this easier.

This was a common issue in 8 as well. Maybe we might do more than just make it easy to remove shifting and scaling pro-actively. Perhaps we could offer to remove shifting and scaling during export using an option (and have it be turned on by default). I guess we could also make Trace Areas and similar transforms remove shifting and scaling as well, but that might be an overkill.

Maps4planners10 post(s)
#23-Mar-18 14:22

I see, so do I have to manually project the resulting shapefile myself to match the image? Seems all the time I save using Manifold to do the trace will be lost making it useable!! Arrghh!!

adamw

8,037 post(s)
#23-Mar-18 14:34

You can project the drawing in 9 before it is exported. This can be done using an addin script, for example.

We agree this is annoying, we will try to make it easier.

Maps4planners10 post(s)
#23-Mar-18 14:55

Thanks - anything to make it useable! It's frustrating having so much potential and it just being useless to me. I've not got time to be doing scripting sadly. If the task was something I'd be doing all day every day I would but it is for a one-off project.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.