Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System

9,480 post(s)
#05-Aug-19 16:38

Here is a new build.

SHA256: 241ca1898fb02f8e4551e654be2a0db4239bc8c93a409864356786b6f02484a3

SHA256: 8b0482b221fd28e1905a9ab37e2f0a38002cfa57ba6f95e5f9833390a3953317


9,480 post(s)
#05-Aug-19 16:40


The Layers pane shows toggle icons for folders which contain layers. If a folder contains both hidden and shown layers, this is shown using a 'mixed state' icon.

The list of layers in the New Map dialog toggles layers using Space / click instead of Enter / double-click.

The list of layers in the Merge Drawings / Merge Images / Merge Labels dialog toggles layers using Space / click instead of Enter / double-click.

Double-clicking a folder in the list of layers in the Layers pane edits its name. The edit box is positioned to start exactly where the folder name starts, unless the list is too narrow for that.

Map window shows tabs for top-level components or folders. Top-level folders that have no components are skipped. Tabs for folders choose one of their layers to be 'current', for visual operation (eg, for adding new objects or for selecting objects using Ctrl-drag) and for panes (eg, Style). If a folder for a tab contains more than one component, the layer context menu for the tab includes the 'Switch to' submenu which allows switching the current tab layer.

Tabs for folders are considered to be visible if any of the layers corresponding to the tab are visible. If any of the tab layers are being cached or painted, the tab displays a chevron of the appropriate color. If any of the tab layers have messages, the tab displays an icon.

Dropping layers from the Project pane into a map window drops them to the top-level before the active tab.

(Fix) Reordering layers by dragging tabs in a map window no longer selects the wrong tab at the end (now correctly selects the tab that was being dragged).

Deleting a layer tab for folder from map using the context menu deletes all layers and folders corresponding to the tab except possibly the layer for the producing component if the window is for a virtual map (eg, if you open a drawing, you cannot delete the layer for that particular drawing).

Zooming to a layer tab for folder using the context menu zooms to the extent of all layers corresponding to the tab.

Toggling a layer tab for folder using the context menu or a double-click shows or hides all layers corresponding to the tab.

The Layers pane allows filtering the list of layers using a filter button, similar to the filter button in the Project pane. Specified filters are kept for the duration of the session.

The Layers pane has been moved out of the Contents pane. This allows having both the Layers pane and one of the panes in the Contents pane visible at the same time. The shortcuts for individual panes in the Contents pane have been remapped: Ctrl-1 = Component (as before), Ctrl-2 = Record, Ctrl-3 = Select, Ctrl-4 = Style, Ctrl-5 = Transform.

The Create Label tool used in layout window is renamed to Create Text.

The context menu for a layer tab in a map window includes the new Show in Layers command, which opens the Layers pane and selects the current tab layer. If the layer is in a folder that is collapsed, the command selects the folder.

Alt-clicking a layer tab in a map window shows the current tab layer in the Layers pane.

Floating / re-docking a window tab is now done via Shift-clicking instead of Alt-clicking. (We are freeing Alt-clicks for other things.)

Favorite coordinate systems allow specifying whether applying the system over an old system should keep or override coordinate system metrics. The default is to keep metrics.

(Fix) Editing favorite coordinate systems no longer suggests adding an incomplete coordinate system to the favorites (could happen if the dialog was launched from a coordinate system picker for a component that has no coordinate system specified).

(Fix) The Fill Sinks transform no longer fails for images with tile sizes different from the default 128x128.

Query functions for watersheds have been reworked to use two-stage create / use pattern. First, you have to create a watershed object using one of the new functions. The created watershed object can then be passed to other functions which compute geometry. Specifically:

  • TileWatershedMake / TileWatershedMakePar - take an image with heights, create a watershed object. TileWatershedMakePar is a parallel variant.
  • TileWatershedMakeDir / TileWatershedMakeDirPar - take an image with directions, create a watershed object. TileWatershedMakeDirPar is a parallel variant.
  • TileWatershed... - all other watershed functions have been reworked to take the watershed object instead of the image.

The change allows using pre-computed directions for computing all watershed geometry without creating a duplicate of each function. It also allows re-using the same watershed object when computing geometry in different variants.

TileWatershedMakeXxx query functions allow creating watersheds with weights (precipitation) for each pixel. To use weights, pass a two-channel image (with either heights or directions in the first channel and weights in the second channel) and pass TRUE for the 'useWeights' parameter.

Computing watershed geometry performs much faster due to multiple optimizations, especially on big images that do not fit into memory cache (and even more so on big images that do not fit into physical memory).

TileWatershedLinesDownstream / TileWatershedLinesDownstreamPar query functions no longer return total flow for computed lines (no longer computed due to changes in algorithm).

Template parameters for Fill Sinks have been renamed: Min delta -> Fill height, Min flow -> Fill flow.

TileMakeNew query function has been renamed to TileMake. (This is consistent with all other XxxMake functions. The only function where we now use XxxMakeNew is UuidMakeNew, which uses New because it generates a new UUID.)

The memory cache performs much better under heavy load. (This includes multiple optimizations which all apply when the amount of RAM available to the memory cache is not big enough to hold all data to be analyzed and some data has to be spilled to disk. In particular, we found that multiple transforms when run with multiple threads on data that is 2-10x bigger than the cache sometimes deteriorate into access patterns which continuously attempt to load data currently residing on disk, making some other data unload, and then attempting to load data that has just been unloaded. Such access patterns are now detected and protected against.)

Destroying big tables in MAP files performs much better when the memory cache is overflown. Destroying temporary tables used by analysis performs enormously faster when the memory cache is overflown. (After doing a lot of optimizations with watersheds, we found somewhat surprisingly that the slowest thing about tables that have a lot more data than fits into the cache is not any of the normal operations like insert or search, but rather deleting the table which has grown to hold a lot of records after it is no longer needed. This item covers several optimizations aimed to help with that. Big tables stored in MAP files will now delete 2-3x faster than before, and temporary tables created for the session now take pretty much no time to delete, at no cost to other operations.)

LAS / LAZ dataport uses less space for spatial index and creates it faster.

(Fix) LAS / LAZ dataport no longer fails to read modern LAS files which store point count only as a 64-bit value with no 32-bit fallback value.

(Fix) LAS / LAZ dataport no longer sometimes crashes on close.

(Fix) LAS / LAZ dataport names the file with the spatial index xxx.LAS/LAZ.MAPINDEXP instead of xxx.MAPINDEXP.

End of list.

181 post(s)
#05-Aug-19 18:29

I can't open the layers pane for a map


181 post(s)
#05-Aug-19 18:34

My mistake -- hadn't kept up with recent changes that layers is a new tab not in components tab

right click tab in map window -- show in layers

view pane > layers

sorry about that


148 post(s)
#05-Aug-19 19:18

Nice round up for the folders.

I would suggest one more thing, though.

Toggling a layer tab for folder using the context menu or a double-click should not show or hide ALL layers corresponding to the tab.

If you could make it so that when a folder is turned off then all layers are off as well, but when you turn it back on it remembers the settings in the layer pane.

Basically unhook the visibility check box of the folder from the ones for the corresponding layers and have the folder visibility override the layer visibility, but only in off-mode.

When a folder is on in the map window it reflects the visibility settings of the layer pane.

When a folder is off in the map, then all layers in the map are switched off as well.

For example, having a folder with aerials from the past 20 years will have the data of the current year visible in the layer pane and all older aerials invisible.

With the current toggle all rule, it would be very inconvenient to have all aerials in one folder.

PS: Thanks for the fix of the LAS import.

The Future of Spatial Data // //


9,480 post(s)
#06-Aug-19 09:13

So, you are suggesting that a layer in a chain of folders should only be visible when it is "turned on" and when all folders in the chain are "turned on" as well. All layers and folders should store their individual "on / off" toggles, flipping a single toggle should not affect other toggles, and visibility should be determined according to the above rule. Correct?

I suppose we could do that. The main concern is - would it be too complex? That would be similar to how controls and windows work - a control on a tab in a pane is visible only if the pane is visible, the tab is selected, and the control is visible itself. So, perhaps a reasonably familiar concept.

139 post(s)
#06-Aug-19 13:30

I agree with BerndD. And I think the folders in the Project table should have the same features (properties) as those on the Layers table. So we could easily use these folders in other maps.

Add: I am also of the opinion, that the folder in a map should keep (respect) the on/off layers; i.e. the layers On will remain On and those Off will remain Off when I turn On or Off the folder..


148 post(s)
#06-Aug-19 15:53

Adam, That's correct.

It's complex. I admit that.

But I think the benefit of this is enormous in terms of organizing big projects.

Please, see the attached screenshot of our MapServer tree. It works with the logic described above.

We literally host hundreds of that sort of trees for our clients.

When this was introduced as part of our MapServer, it instantaneously became a part of how our clients wanted to organize their data.

To us it is a proven concept. I think it would be worth going the extra mile.


The Future of Spatial Data // //


6,438 post(s)
#06-Aug-19 18:38

But I think the benefit of this is enormous in terms of organizing big projects.

100% agree. I think it's a really great idea, a gem. I don't think it is complex in action, but on the contrary has the genius of simplicity and great usefulness. It's such a great idea I bet we'll see this in the next build.

808 post(s)
#07-Aug-19 16:06

Have to admit I don't get the attraction to folders, like everyone seems to. Would a video demo the benefits better than the text I've been reading?


6,438 post(s)
#13-Aug-19 05:45

Even better than a video demo would be trying it. Check out the updated Layers pane topic and the example it gives of using folders to turn off/on several layers at once, and then try it out in one of your own maps where you have many layers.

If you don't have maps (or tables, or layouts) with many related layers (or fields, or frames) that you need to manage, then it's a don't care.

But that's just like folders in projects, and just like files in Windows. If your project only has four or five components you'll keep them all in your root and won't use folders to keep them organized. Same with files in Windows. If you only have a few files you might keep them all in one folder. But if you have hundreds, you'll probably use folders to keep them organized.

In the case of maps, folders make it easy to have maps which have only a few layers that you care about, but where there might be dozens of layers that you occasionally might want to see in context.

Consider the Naperville Gas example ESRI hands out for GDB, where it is a gas utility with dozens of layers, most of which aren't of interest most of the time. Since maps take up zero storage in Manifold, there's no downside to having all those layers in your map, so long as they are conveniently grouped in some folder that with one click turns them all off/on together.

Why might you want to do that? So when you're working with something you care about, like a parcel layer, with a single click you can turn on all the extra stuff to see if there is anything in those layers on a particular parcel, like a gas line crossing it.

Another example is the Natural Earth vector data, which has a staggering number of fields most people don't give a hoot about in attribute tables. The old way of dealing with that was to delete many fields we don't care about. But that's a one-way deal: I don't know about you, but I often think twice about that because then I find myself thinking... "OK... but then it's not Natural Earth anymore, and maybe someday I'll need one of those fields...". In truth, there are reasons why Natural Earth provides so many attributes, and maybe some day I'll understand how to use one of those fields which today seems useless.

No problem. The new way is to just toss a hundred and one fields you don't care about into a folder called Misc and turn them off with one click. If you ever need them again, they're there. Manifold is so fast you don't really care if those fields are there or not in most cases.

Ah, and one last thing: layouts: When layouts introduce legends, a legend will be a collection, a group, of frames. Legends will end up having very many frames within them, perhaps repeated and automatically created from a template or whatever, but nonetheless, many smaller elements within a hierarchy of organization. For example a row in a legend might be a symbology or color sample, a text caption, a range of numbers caption, and so on, and the rows might be repeated in a block where above and below are other blocks or captions.

Structurally, that's all groups of items within other groups, just like layers grouped within subfolders of other folders. If the same mechanism works for all, then you can have more reliable operation and common ways of tinkering with organization of the thing if you want to customize how automated or semi--automated templates do it.

Obviously, you'd have some controls specific to frames within layouts and frames as sub-elements of legends within layouts, but the grouping paradigm can easily accompany that, just as it now works with groups of layers in maps within folders or groups of fields in tables within folders, and just like layers have a transparency control in maps and fields have a width control in tables. Frames within layouts/legends could have left/right/center justification controls, etc.

808 post(s)
#13-Aug-19 22:34

I had already tried it. I had tried it inside a table only and not on a map. After trying it on a map I still don't see the attraction. I can see a good reason to collapse layers into a folder, but I can't think of a time when i would want to turn on all the layers inside that folder at the same time. For example I have lots of labels for lots of layers. Since most of the layers and labels are off most of the time, it might be nice to simplify the map appearance by hiding the seldom used layers into a folder; however, I don't see when I would ever want to turn an entire folder of layers on at the same time. I have had this same sense about other features in M9, but after seeing a more clever approach used in your videos, it made sense. The videos showing color control in imported rasters was extremely helpful to explain an otherwise esoteric concept. I'm hoping there is a clever use for folders that I'm not seeing.


9,551 post(s)
#13-Aug-19 23:19

Think of versioning. Do your clients ever need to see historical data?

Imagine that you save every iteration of the data you prepare for a client as a separate project (as we all should, for other reasons). With each latest generation of data, either as a matter of course or as the need arises, you can link as many past versions into the new project as you like, essentially for free. This could be a truly vast amount of data (possibly impractical to manage in other software). This does not depend on the new features, it has been the case for some time.

With the new features, by arranging historical data into folders (and subfolders), and deploying folders into a map, you can immediately answer client questions such as "How much has this subdivision changed since 1986?", "Has the river moved significantly closer to that levee since the previous aerial survey?" or "How many new houses have been built in the last year?". In two clicks.

And that is the case even if each iteration of the dataset should include tens or hundreds of layers--of vector data, imagery, mosaics, labels--anything. Two clicks.

That's one example. Imagination is the limit.


6,438 post(s)
#14-Aug-19 06:20

Two more examples:

1. Let's say you build a map and you include hydrology features, that is, water features, and you import those from shapefiles. That often means for each such class of features you have three shapefiles, for points, for lines, and for, ahem, "polygons", and so you end up with three drawings and three layers. It's not unusual to have several different types of water features, such as coastal, wetlands, inland waterways and so on.

Because of the sources of the data, you could end up with a dozen layers having to do with "water" in your map, but in the workflow of what you show / don't show in the map or their position in the display stack, all those "water" layers could just as well have been one layer. Put them all in a "water" folder and then you get to work with them as they were just one layer. Much quicker and easier than messing with a dozen layers.

2. I'm helping a friend design his country house, using 9.

There are two keys to using 9 as a CAD package. The first is creating some layers of point grids, for example, a 30 x 30 grid of points spaced every meter, a similar grid of points spaced every 10 cm or 20 cm, etc. Until snaps to virtual grids arrive (soon) you can use such point grid layers as anchors to do snaps when drawing walls and such using areas. You can make the points really small, thin + symbols, and turn them off (transparent stroke color) when you don't need a grid in a layer.

The second key for doing CAD in 9 is to have good editing skills in 9, with good muscle memory. If you fumble using the various commands it's not realistic, but if you learn the interfaces well it is astonishingly fast, quicker than Illustrator and definitely as fast as AutoCAD.

Onward... the house is a two-story house with about 50 layers. To visualize the design it's essential to be able to create different maps that show the two stories in different ways, using different transparencies for different layers. For a cost-effective design, for example, it helps to lay out plumbing so various bathrooms, WC, kitchen and similar share a common plumbing infrastructure in the same part of the house, so you don't have to route water lines and drains to all four corners of the house. I'll have three or four maps opened so I can use whichever I want by just clicking on the name tab for that map.

I've discovered that using folders is a great way to do visualizations. I have a map where the 20 or so layers in the first floor are in one folder, and the 20 or so layers in the second floor are in a different folder., and layers involving both stories, such as stairs, are not in folders. To discuss something using that map, I can very rapidly switch on/off to flick between first floor and second floor, to see how the structure of walls and infrastructure of pipes and similar line up or don't line up. You can't get the same effect switching 20 or so layers on/off one at a time. One of the maps I use has transparency in layers so you can see how the two floors line up, but that can be a more confusing effect for some people than flicking between "first floor" and "second floor" without transparency.

There are other overlays, too. To get a sense of scale I have plans for existing houses he knows that can be overlaid, so if there is a question about how big the kitchen is, just one click and he sees how it compares to the kitchen he has today. If you sit down with a friend or a client, such use of GIS as CAD can be tremendously useful for "what it" planning, and understanding the lay of the land, so to speak. Folders can help with that.


6,438 post(s)
#12-Aug-19 18:50

This has been added in Works beautifully - what a great idea, Bernd!

See the updated Layers pane topic for an example using the idea.


148 post(s)
#05-Aug-19 19:30

Filling a folder with a large group of layers in the right order can sometimes be time consuming.

Therefore it would be nice if one could copy a folder from one map to another.

The Future of Spatial Data // //

Manifold User Community Use Agreement Copyright (C) 2007-2019 Manifold Software Limited. All rights reserved.