Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System 9.0.173.4
adamw


9,552 post(s)
#16-Mar-21 14:48

9.0.173.4

Here is a new build.

manifold-9.0.173.4-x64.zip

SHA256: 1220450d64bb84ce166e099c78711335876ddfa310ac9ae8af2ba385e1873c19

manifold-viewer-9.0.173.4-x64.zip

SHA256: f3d9a738424575e58165db61b4bcb6bf5b165017c468a1704c075e324fd08650

This build contains extensions to geometry editing and raster interpolation. Next: various small things in registration, editing and other areas.

adamw


9,552 post(s)
#16-Mar-21 14:51

Editing

The map window supports several new commands in the context menu used for editing geometry:

Clip

Allows clipping currently edited geometry. The command shows a dialog which allows switching between clipping to all or selected records, and switching between keeping inner or outer part of the result. The default is to clip to all records and to keep the outer part. You can clip newly inserted geometry or existing geometry or tracker geometry.

Clipped geometry can be edited further before being saved.

Merge

Allows merging multiple area, line or point records into one. The command shows a dialog which allows switching between merging all or selected records and specifying transfer rules. The default is to merge selected records (otherwise it is too easy to merge all records). You can only merge into existing geometry (have to Alt-click geometry for the record to merge into).

Merging immediately commits changes to all altered records and clears the picked record.

Transfer rules for merge:

  • (none) - keep the value in the picked record, any field type,
  • copy - keep the value in the picked record if it is not NULL, otherwise use a first encountered non-NULL value from one of the merged records, any field type,
  • minimum / maximum - use the minimum or maximum value, any field type except binary,
  • average / sum - use the average value or the sum of all values, any numeric field type.

The computed fields and unique fields are excluded from the merge. The table has to support updating and deleting records.

When merging selected records, the picked record does not have to be selected.

Split

Allows splitting area or line records into parts producing a separate record for each part. The command shows a dialog which allows switching between splitting all or selected records and specifying transfer rules. The default is to split all records. You can only split using tracker geometry (to split using existing geometry, use the Split transform).

Splitting immediately commits changes to all altered records and clears the tracker.

Transfer rules for split:

  • (none) - use a NULL value, any field type,
  • copy - keep the original value, any field type,
  • split - split the original value equally between parts, any numeric field type,
  • split proportional - split the original value between parts proportionally to their area (for area) or length (for line), any numeric field type.

Splitting the original value between parts makes sure that the sum of the resulting values equals the original value, including for integer fields (eg, splitting 100 into 3 equal parts will produce 33, 33, 34).

The computed fields and unique fields are excluded from the merge. The table has to support inserting, updating and deleting records, and has to have an autogenerated unique field.

All of the above commands remove curves and Z values from processed geometry.

All of the above commands save last used options in the map window to allow quick repeated use.

All of the above commands run progress and allow canceling. This allows using commands with big data sets or with data sets on slow data sources.

Interpolation

The 'gravity' interpolation method is renamed to 'gravity (IDW)', to include a commonly used term and allow using it in the filter box.

Gravity interpolation allows specifying interpolation order. The default value is 2.

Gravity interpolation now uses multiple threads to compute Voronoi neighbors.

There is a new interpolation method: 'natural neighbors' (here is the description of the method on Wikipedia). The current implementation is using Sibson weights. The method is limited to the convex hull of the source data.

There is a new interpolation method: 'thin-plate spline', analogous to 'thin-plate spline' registration. The method works outside of the convex hull of the source data.

(Fix) Kriging interpolation no longer sometimes produces weird results for the power model with radius set to 0 (autocompute) due to selecting the wrong radius.

(Fix) The Interpolate transform no longer applies margins for triangulation / triangulation with segments (the margin parameter was hidden, but the system was still using its value).

(Fix) The Interpolate transform no longer sometimes increases margins for interpolation methods not limited to the convex hull of the source data.

Changed function: TileInterpolateGravity, now also accepts order (any positive number, possibly fractional, a good default value is 2).

New function: TileInterpolateGravityPar, a parallel version of TileInterpolateGravity.

New functions: TileInterpolateNatural / Par, create a natural neighbors interpolation object.

New functions: TileInterpolateSpline / Par, create a thin-plate spline interpolation object, accept radius (0 to autocompute) and the number of neighbors (0 to use Voronoi neighbors).

Changed functions: TileInterpolateXxx, now also accept bounds for the produced image. Passing invalid bounds (eg, VectorMakeX4(1, 1, 0, 0)) sets bounds to those of the source data. Specifying bounds allows not computing values that will be ignored later anyway. This improves performance, particularly for various types of Kriging which were spending significant time computing values near the edges.

Other

(Fix) The library dataport correctly skips image files that refuse to load in all cases. (Previously, some files might cause the dataport to stop initializing.)

The library dataport reports the number of files matching the file mask in the log window. This allows quickly checking if the library shows no data or less data than desired because of issues with the file mask.

(Fix) Linking TIFF without cache no longer fails. This was making the library dataport choke on TIFF files, too.

End of list.

Dimitri


6,511 post(s)
#16-Mar-21 15:34

Quick tips:

Clip - Using Clip to create a new area that is exactly adjacent to existing areas on a

coastline: Choose Create Area for cursor mode. Draw the area so part is offshore

and part overlaps the onshore areas. Right-click and choose Clip. Press OK, and

the provisional area outline is clipped to only that part offshore. Right-click

and choose Save Changes or just press Ctrl-Enter.

Merge - Using Merge: Select objects to be merged, and then Alt-click the object to be merged

into (which means merges can happen only into existing geometry). In the Merge dialog

choose rules for how fields are to be merged (double-click into the default copy

rule to change it) and press OK.

Split - To split an area: Choose Tracker for the cursor mode. Draw a tracker line where

the split is to be. Right-click and choose Split. Specify how fields are to be

split in the Split dialog and then press OK.

Mike Pelletier


1,859 post(s)
#16-Mar-21 16:50

Thanks so much for the new editing tools! Incredibly helpful and indispensable for editing. Seems like the implementation provides handy control over parameters. The interpolation improvements look great too.

Here are a couple thoughts to consider moving forward.

- Would be nice if the clip tool could be made to include other layers. The coastline example is great. This is lower on my list of needs though and I imagine this could get complicated.

- Could the topology transform be allowed to also alter existing layer rather than having to create new layer. This would be useful when dealing with selection only and your sure you won't screw up the whole layer.

- Please make the Coordinate tab default to your last use of either traverse data or X,Y data to help reduce clicks. The traverse dialogue is much more useful for my work.

- Please add options for scale factors to the traverse info dialogue and make it directly editable with an option to enter numbers with simple math (addition, subtraction, multiple, divide). For example, type 51.5*2, press enter and 103 appears in the box.

- Using the tracker tool for splitting makes good sense but it's name is not intuitive for the use. Perhaps that is okay but if the tool evolves more a new name should be considered. It does seem the tracker tool info could be merged into the traverse dialogue to provide a venue for displaying more info and possibly converting the drawing to an object in a layer.

Mike Pelletier


1,859 post(s)
#16-Mar-21 17:59

To add the post above, the clip tool is really great for cleaning topology between two objects. Very helpful tool. However, it really would be better if it could consider the objects in another layer. For example, if your editing parcels or perhaps a tax district layer and you want to clean the edge with a say a public land layer that you know has good boundaries. This also needs to be an option in the topology transform dialogue to allow matching up a layer to another known good layer.

Mike Pelletier


1,859 post(s)
#17-Mar-21 15:04

Another thought on the split tool. Seems like allowing a user to import a line from any layer into the Tracker tool would be really helpful. For example, a user could take a meandering stream line from a river layer and use it to clip an area, rather than taking the time to redraw the stream with the tracker tool.

Alternatively, the Split transform could hopefully be expanded to alter an existing table and use another layer as the splitting line.

dchall8
847 post(s)
#17-Mar-21 17:46

I can see the benefit of using lines from another drawing. When I was in the office I split parcels maybe 5 times a day. In a rural community people are always selling off a few acres for one reason or another. I had a dedicated layer for the custom lines I used to make the splits. That layer collected thousands of bits and pieces which served as a permanent record of the changes made to the data. Using the Tracker tool in 9 is a self deleting custom line which leaves no permanent record.

antoniocarlos

580 post(s)
#17-Mar-21 18:27

The tracker should be renamed multitool after these additions.


How soon?

Mike Pelletier


1,859 post(s)
#18-Mar-21 14:59

Or maybe something like Sketch tool to indicate its temporary nature. Select the tool and it opens up an improved traverse dialogue to display measurement info.

Seems like it could be a place to store temporary vectors that might also help with the desire for undos. Maybe have options to either automatically put copies of vectors into the sketch table or provide options for more manual control.

dchall8
847 post(s)
#18-Mar-21 20:11

It is the Swiss Army Knife of Manifold tools. I would like to copy the distance, area, and/or angle somehow (right click options?).

tjhb

9,627 post(s)
#18-Mar-21 22:04

I don't think it needs renaming. I don't especially like "Tracker", but we are used to it. It was a great idea to give it a new job, in the latest build. (Real imagination. Obvious once it has been done.)

"Multitool" does not help, I think. It only begs the question--what does it do.

If it is to be renamed, I would suggest "Transect", since that is what it does. But is that better than "Tracker"? It is probably less friendly, which counts.

antoniocarlos

580 post(s)
#19-Mar-21 00:54

True. Multitool is ambiguous. But I think it should be changed. The tool is not just for "tracking" anymore. Whatever name Manifold decides is better, will be fine with me. And I guess suggesting it should be a separate tool will probably be frowned upon (this is what I would do). I love the new functionality btw.


How soon?

adamw


9,552 post(s)
#19-Mar-21 12:29

We think it is best to extend the Split transform to allow altering an existing component instead of creating a new one. Before we have this, however, we will likely allow editing tracker geometry in the Info pane. This will allow saving the line you want to split with into a file, then reloading it into the tracker. Somewhat laborious, but will make splitting using an existing line possible until we alter the Split transform.

Graeme

954 post(s)
#18-Mar-21 07:09

Great to see these developments in interactive editing, thanks.

I second Mike's observation about transform / Split needing to offer the target of an existing drawing in addition to a new one, this is almost always the case for us.

The split option with the tracker tool is a good start, but requiring the specification of transfer options for each field means it isn't an option for most of our work, where drawings typically have several hundred fields. This would be better handled by a one-off investment specifying transfer rules in the table schema - in an ideal world, transfer rules, if specified, would come across automatically from an 8 project.

tjhb

9,627 post(s)
#18-Mar-21 07:40

I don’t disagree with you or Mike here Graeme, but I do want to say this.

Anyone dealing with several hundred fields in one table is not just unusual, but clearly working with idiots.

Should idiot colleagues drive Manifold development? Not usually!

Get them some basic DBMS training instead.

apo
141 post(s)
#18-Mar-21 09:15

Being an idiot I might weight this view.

Having hundred fields relying on the geometry split for sure I do not understand any reason for that.

Having hundred fields of variables (socio-demo, eco, etc.) linked to an area is really common with any country census. But those variables have not to be "splitted" "clipped" but might be merged.

Therefore I would prefer not to rely on the interactive tool for that but keep the variables on one side, make the work on the geoms and then use sql to reaffect the variables managing the aggregate clearly in SQL.

Voilà

Dimitri


6,511 post(s)
#18-Mar-21 08:16

where drawings typically have several hundred fields.

Ouch. I feel your pain. Setting aside the counter-productivity of having hundreds of fields, there's something to be said for being able to define in a persistent way transfer tools for tables, at least on a first impression basis. Whether it's ten fields or fifty or many more, somebody who creates a project might want to have a standardized way, in advance, of how fields should be combined.

That has nuances, since you might want different methods for merge and split. You also might want default fields (different, but related) when creating with Clip or just creating new objects. Those might all be part of some schema manager.

Keep in mind that one of the advantages Manifold has, orthogonality with many different data sources, definitely raises the stakes when you're designing such features, because it should all work the same way, ideally, regardless of the data source. Build 173.4 was supposed to come out a few days earlier, but was held back to add engineering so that one of the new commands could correctly work in PostgreSQL/PostGIS despite multiple threads in big remote databases without causing any sort of delay or race condition.

So you can't just assume it's as easy as everything is local and if you want to add a property to a table you do that and all the pre-defined transfer rules will be remembered. What if the data source allows you to edit the table but not to create new ones, where such auxiliary info and more-than-just-the-schema can be saved in a way that reliably stays connected to a given table, despite name changes and other possibilities?

Anyway, such things can be sorted out. Part of rolling out some new editing features is an expectation that, like the new georegistration, there will be plenty of extra details, enhancements, and follow on improvements made.

In the meantime, for setting rules on many fields at once, you can select all the ones you want to set in the usual way (ctrl-click, shift-ctrl-click to select a swath...) and then when you change one all of them change to that same setting as well.

adamw


9,552 post(s)
#19-Mar-21 12:34

Two things on handling hundreds of fields in the Merge and Split tools:

(1) You can set transfer rules for multiple fields at once - select them, then edit the rule. Further, for most fields you will likely want just 'copy' - this is already the default.

(2) We could auto-save transfer rules used for Merge and Split into the component properties. This would make them persist if you close and reopen the window or even the project. Would that work for your case?

Graeme

954 post(s)
#04-Apr-21 02:35

Thanks for the detailed technical explanations and follow up and apologies for the delayed response.

It's mainly problematic for a large set of national drawings stored and maintained on an 8 Enterprise Server, and by extension called to participate in various stand-alone projects, most commonly from the 8 Server Console's "import" option - "link" and "check-out" is primarily for maintenance, where the transfer rules become of primary practical importance - but also via a 9 data source, where the current split transform target excludes an existing drawing and is therefore, at present, a data maintenance dead-end.

(2) We could auto-save transfer rules used for Merge and Split into the component properties. This would make them persist if you close and reopen the window or even the project. Would that work for your case?

If transfer rules could be made to persist in the database object sense, yes it could have potential, providing it didn't render those objects inaccessible to 8.

In the short to medium term we still create revenue-generating cartography in 8. 9 will eventually get layout functionality to make publication quality maps and more besides, but until then we cannot realistically migrate from 8 in a commercial sense. We built much of our business and data models on the capabilities of 8. As with SQL, if we need to make up-front investments to modify approaches to yield equivalent outputs in 9, the increased performance needs to significantly tip the scale in its direction and it isn't always going to be a simple balance. Providing 8 remains viable, we can pick the eyes from 9 on a needs basis (geocoding is a good example, no longer practical in 8 so passed on to 9) and not suffer any disadvantage.

adamw


9,552 post(s)
#06-Apr-21 14:31

If transfer rules could be made to persist in the database object sense, yes it could have potential, providing it didn't render those objects inaccessible to 8.

Understood. Adding properties to an existing component on an Enterprise storage is safe, 8 either doesn't see these properties at all or doesn't understand what they are and ignores them (this depends on the nature of the property). There might be other issues though, ie, the component losing these properties on Check Out from 8. I'll add a note to the request to save the settings for editing tools to try and make this work for components on Enterprise storages specifically.

We'll keep working on cartography, this is a priority (labels, scale bars, etc).

adamw


9,552 post(s)
#19-Mar-21 13:11

One more thing on transfer rules: we have come to regret the design of 8 with common transfer rules editable in the same central place. This had multiple big drawbacks.

First, every tool had to support all transfer rules. While this might seem like a good thing, all this means in practice is that it is harder to add new transfer rules, because even if you really want to extend just one tool, you have to extend all of them. And transfer rules which could have been specific to a single tool either get generalized or do not get added. Example: in the Split tool, we now have 'split proportional'. We do not need to spell out what is the split proportional to - since the tool splits geometry, the 'proportional' bit applies to a measure of the geometry. In 8, we had to generalize the corresponding rule to cite a field. Not only did this add complexity, it also made extensions very hard - which is why we didn't do any in 8 (but likely will in 9).

Second, when transfer rules are defined in some central place other than where they are being applied, people forget about them and this sometimes produces weird results. It is just better to show transfer rules during the operation that is going to use them. This way you know that the transfer rules are going to kick in and what they are going to do. Plus you can change them without canceling the operation and going to a different dialog. Now, we can say that this is a different issue, let's have common transfer rules for all tools, we can still show them and allow editing them in any tool. But if we are going to show transfer rules in every tool, why do these rules have to be common? Do I want to change a transfer rule for some field for Merge when I am changing a transfer rule for that field for Join? Sometimes I do, sometimes I don't. And since I will see what the transfer rule for Merge will be before performing it anyway, I would prefer changing a transfer rule for Join to change it just for Join.

That's roughly why we moved away from having common transfer rules editable in a central place distant from where they are actually applied, to having different transfer rules specific to each tool editable everywhere they are applied.

adamw


9,552 post(s)
#19-Mar-21 12:18

Some response:

Extend Clip to clip using geometry from a different layer - we'll try to do this.

Allow Normalize Topology modify existing component instead of creating a new one - we'd like to add modifying existing component as an option to this and several other transforms. That's the most difficult item on the list though, might have to wait.

Keep the last used mode in the Coordinates tab - will do.

Allow editing traverse commands directly in the Coordinates tab - will do. Note though that changing a single command in the middle of the command list frequently has a cascading effect on all further commands. Perhaps that's what you are after in the first place though.

Allow using simple formulae while editing traverse commands - we'll consider that.

Rename the Tracker tool to something better - we are talking about doing this currently.

Thanks a lot for the notes!

Mike Pelletier


1,859 post(s)
#20-Mar-21 15:14

Great and yes the cascading effect is definitely desired. It is extremely important to be able to interact with a traverse rather than just import them. Having all the other layers in your map displayed helps identify problem's with the traverse. This would be highest on my list of hopes, dreams, and desires :-)

Dimitri


6,511 post(s)
#16-Mar-21 15:52

See the new Newsflash - Merge, Clip, and Split video for a quick demo of editing commands.

tjhb

9,627 post(s)
#16-Mar-21 16:33

Very happy about the extensions and improvements to the interpolation functions, a big deal, thank you! Can't wait to spend a few weeks trying these out.

Manifold User Community Use Agreement Copyright (C) 2007-2019 Manifold Software Limited. All rights reserved.