Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System 9.0.168.1
adamw

8,061 post(s)
#20-Aug-18 18:08

9.0.168.1

This build contains a mix of fixes and features in various areas. Next builds will concentrate on cartography.

manifold-9.0.168.1-x64.zip

SHA256: 7dc736c3c5d4c195914bcc3896d1a07e4d983fbccc88bf3672f890223b0a63ed

manifold-viewer-9.0.168.1-x64.zip

SHA256: 9917ec0a1d565347ec0ee14942ba3021cb1704f77f74684c65bade9e653e59c1

adamw

8,061 post(s)
#20-Aug-18 18:09

Changes

(Fix) Transform templates for triangulation no longer have the margin parameter (was unused).

Launching Kriging with very little data no longer sometimes fails because it is impossible to compute model parameters. Instead, if there is too little data to set up the model, kriging degrades to gravity.

The IMG ERDAS dataport can use data for intermediate levels in the accompanying RRD file. (Try linking a big IMG ERDAS image with an RRD. Previously, this was computing intermediate levels from the base level. This was a one-time thing and the computed data was saved to the MAPCACHE file, however, with a big image this could still take quite a bit of time. Now if the image has intermediate levels stored in an RRD, we just use data from that file.)

(Fix) Rendering unscaled tiles for images in MAP and GPKG no longer fails to make pixels outside of the image invisible. (This fixes a number of issues with merging, projecting and transforming images.)

Merging images whose coordinate systems match except for offsets is faster.

(Fix) Exporting image to TIFF correctly adjusts the origin of image in the projection data written into the file.

Exporting image to TIFF writes values for invisible pixels as legal values set to the reserved void value specific to the pixel type, and advertises used void value for other applications. Visible pixels with values equal to the reserved void value are adjusted not to coincide with it (this loses some data, but better the pixel is adjusted than an invisible pixel made visible or vice versa). Void values for signed integer types are chosen to be the minimum allowed value (in the -128..127 range, -128 is void value, valid -128 is adjusted to -127 on export). Void values for unsigned integer types are chosen to be the maximum allowed value. Void values for floating-point types are chosen to be the maximum allowed value.

Kriging with median polish query function allows specifying processing step for median polish (the default value is 0 = auto).

Kriging transform templates produce a report of used parameters, including resolved values for autocomputed parameters, and put it into the description of the new component.

Global (not per-record) parameters for transform templates are specified using the parameter picker control.

Numeric parameters for distance values in transform templates show the coordinate system unit and allow changing it. The coordinate system is taken from the target field. If the coordinate system uses uneven local scales for X and Y, the parameter is set to use 'native unit' by default, and the parameter value is used without scaling. Otherwise, the parameter is set to use the unit specified in the coordinate system.

The model parameter in Kriging transform templates is selected using a drop-down menu (instead of being entered as a numeric value).

(Fix) Table window no longer sometimes continuously starts and stops the background thread for the preview even when there is no more data to compute.

(Fix) Table window no longer sometimes fails to update preview if the screen contains only one record.

New transform template: Clip - clips a geom with another geom, works between fields.

New transform template: Clip All - clips geoms with the union of areas in the specified drawing.

New query function: GeomBoundedAreas - computes bounded areas for a line geom.

New transform template: Bounded Areas - computes bounded areas for a drawing.

Kriging and gravity interpolation optimize searches for Voronoi neighbors and perform slightly faster.

End of list.

tjhb

8,217 post(s)
online
#20-Aug-18 23:28

Kriging transform templates produce a report of used parameters, including resolved values for autocomputed parameters, and put it into the description of the new component.

This is good (even better than writing to the log).

I have a question.

Let's say I have a very large vector dataset, which I decide to interpolate as overlapping tiles (that is, rects or tiles of vector data, producing overlapping images, which are afterwards cropped and perhaps combined).

Let's say I use Kriging. For each tile of vector data, the interpolation code will automatically determine the best parameters: range, nugget, sill, partial sill, lag, lags (and possibly model and radius if these are not specified).

My question is, would it be better if I had the option to specify all of these parameters explicitly, so that the overlapping images will be mutually consistent?

(If it were possible, I would run several example sample vector tiles, read the parameters automatically chosen for each, then specify a standard set of parameters for use on the whole dataset.)

Does this matter? I suspect it does matter but I don't know.

adamw

8,061 post(s)
#21-Aug-18 07:35

It might make sense to be specifying parameters like this directly, eg, to use the same values for multiple interpolations like you say. We can allow passing all parameters in the future by passing a big JSON string, this part is easy. The main difficulty is figuring out what has to happen when some parameters are specified and others aren't. It is easy to request specifying everything, but some parameters might genuinely need to be computed from the properties of vector data. So this mainly needs specific scenarios to see which combination of parameters we'd like to accept in order to compute the rest.

tjhb

8,217 post(s)
online
#21-Aug-18 10:24

Thanks Adam. A JSON parameter set would be great (returned and then passed back, possibly overspecified).

Do you mean (in last sentence) that you'd like one or more specific overlapping datasets to test with?

adamw

8,061 post(s)
#21-Aug-18 13:28

No, I just meant that we have to look into which combinations of "these parameters I set and the rest I want the system to autocompute" make sense and if anyone stumbles onto a specific case which can speak to that, we'd welcome that input.

vincent

1,716 post(s)
#21-Aug-18 21:56

(Fix) Exporting image to TIFF correctly adjusts the origin of image in the projection data written into the file.

No more offset :)

Mike Pelletier


1,508 post(s)
#22-Aug-18 18:26

Adam, just thought I'd mention, in case it somehow fell off the list, that we still cannot assign a projection for a linked .ecw file. The error message is "Cannot set value".

adamw

8,061 post(s)
#23-Aug-18 15:36

Right, I'll bump that up a bit, thanks. Linking an ECW (or JPEG2000) file should create a .MAPCACHE file to store adjustments like this. This item was waiting for two other items, one of which has been completed, so there's one more item left to complete and then we can fix and close it.

danb


1,649 post(s)
online
#22-Aug-18 19:10

Great work and a quick note to let you know that the fixes for the issues I was experiencing while merging images and exporting the results work flawlessly.

I am however experiencing a few issues with the RRD cache, but will package up some problem examples and send them to tech in the near future. Without any real basis, I suspect that they might be to do with ESRI written RRD as I have found RRD files associated not only just with img but also with other file types such as adf, tiff. When I link an IMG file with an associated RRD file I get a 'Can't read data from file' error on all that I have tried thus far.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


4,980 post(s)
#23-Aug-18 08:58

I don't think the feature in 9.0.168.1 is aimed at any RRD file other as used in ERDAS IMG.

ERDAS IMG format as it utilizes RRD files is slightly different than what ESRI does with RRD for file formats other ERDAS IMG. When ESRI creates RRD files for formats other than ERDAS IMG it puts the pointer to the pyramids in an .aux file, not in the .img file as does ERDAS, so the two are close but not identical.

I expect that over time 9 will consume RRD with other formats, and probably even export using RRD as ESRI does.

When I link an IMG file with an associated RRD file I get a 'Can't read data from file' error on all that I have tried thus far.

Might be an ESRI written IMG in some format other than ERDAS IMG, in which case ESRI might have written the pointer into an .aux file. Just guessing. Or, some ESRI packages are known to have issues writing and using intermediate levels. Or, it could be a bug in Manifold. :-)

If you have a problem with a specific IMG file, send it in as a bug report.

adamw

8,061 post(s)
#23-Aug-18 15:45

Thank you and the more examples of the failing RRDs, the better. We are planning to do more with RRDs as well, this is an expanding area in our code.

danb


1,649 post(s)
online
#23-Aug-18 19:41

Great. I will send some examples to tech today. My main concern was that if I encounter an ESRI RRD that it might cause the import of a legitimate(?) IMG to fail. The legitimacy of the IMG that I will send is something that I am sure you will be able to determine, though I know it loads fine in at least ArcMap. Anyway, I'll get something sent to tech with a bit more of a description.

I think this is an important addition to Manifold as it plays to perception and psychology that other GIS vendors and particularly Google use to great effect. With other vendors it somehow seems to matter less that your interaction with the data is potentially hobbled by being tied to the original file and data structure because it appears to load instantly!

Straight away it becomes more of a hard sell to get non Manifold users to appreciate the richness and benefit that comes from the (largely) minimal import times that Manifold imposes to bring the information into its own data structures.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


4,980 post(s)
#24-Aug-18 10:27

I will send some examples to tech today

Perfect! They can go to work on those and fix whatever is the issue. Manifold's rule of thumb is that if Arc opens it, Manifold must open it too.

RRD is an ERDAS thing. It was smart of ESRI to adopt it and put it to use as an auxiliary in formats besides ERDAS IMG. That's something Manifold can, and will, do as well.

dchall8
479 post(s)
#31-Aug-18 16:36

Manifold's rule of thumb is that if Arc opens it, Manifold must open it too.

Do you have a rule about M9 exporting file types?

Each week I use M8 to export a kml file which includes HTML coding for Google Earth. M8 works great for that, but as of yet, M9 cannot export the table field that builds the HTML. Here is a sample of the text within that field.

<head><style>body {background-color: linen;}p {color: maroon;}</style></head><hr /><p><strong>Owner:</strong><br />BANDERA COUNTY</p> <hr /><p><strong>Owner&#39;s Address:</strong><br /> PO BOX 368<br /> BANDERA, TX</p> <hr /><p><strong>Legal Description:</strong><br />BANDERA RNG 12 ***COURTHOUSE*** 2.561 ACRES</p> <hr /><p><strong>Situs Address:</strong><br />500 MAIN ST</p> <hr /><p><strong>Driving Directions:</strong><br /><p>Click <a href="https://maps.google.com/maps?saddr=29.7264041+-99.0729435&daddr=29.7267271651482+-99.0724994758346">HERE</a> for directions from <strong>BANDERA</strong>.</p> <p>Click <a href="https://maps.google.com/maps?saddr=29.424565+-98.494713&daddr=29.7267271651482+-99.0724994758346">HERE</a> for directions from <strong>SAN ANTONIO</strong>.</p>

The field is an active field that builds the text inside using data from our office database. When I export to kml I select that field for export and Google Earth interprets the code. Inside Google Earth the user can click on one of the map parcels and get a balloon with information about the parcel as well as driving directions to the parcel. I've attached an image of the bubble described above.

Attachments:
Google Earth Bubble.jpg

Dimitri


4,980 post(s)
#31-Aug-18 19:20

So what does 9 export in that case? Have you sent it in as a request? Don't be shy about such things as they are often very easy and can get worked into builds along with more difficult stuff. 9 in particular tends to have way better text processing within fields than 8 [look at all the transforms that work super with multi-line text...] so my guess is this wouldn't be difficult at all.

Speaking of text processing in 9, I did a really weird thing today. I'm often working with text that comes out of HTML editors and going back and forth to real text editors. You probably know the drill if you've done that, it's always a mess with CR-LF or newlines or whatever with the text often ending up double-spaced when you don't want it to be. Or, there are special character issues like tabs in unicode you want to replace with spaces and so on.

Today I had some blocks of text I wanted to fix and I didn't want to remember how to use Notepad++ or whatever so I just copied and pasted them into a text cell in 9. I could then use the replace all template to brute-force edits, using expressions like '\u0009' to grab tab characters, fix the ends of line sequences to what I wanted them to be and so on. Because you can right-click on the cell to paste / copy into and out of it, it's remarkably quick. Weird, I agree, but if you have a 9 session running, well, why not? :-)

I think the next build will extend RRD to many more formats, by the way...

dchall8
479 post(s)
#31-Aug-18 21:56

M9 exports the parcels just fine, but there is no option to include data with it like there is in M8. I'll find the Suggestions site and send it in.

I guess M9 would not be my first choice for a word processor. My headache with those is when people mix tabs and spaces trying to align text vertically from line to line. I published a newsletter for many years back in the early 90s. One of the guys in the local Mac club was a pretty good teacher. He helped me learn MS Word on a Mac and then move it into a program that's been dead for so long I can't remember the name - a document processor for final production. If you had spaces instead of tabs you could go crazy trying to figure out the problem.

danb


1,649 post(s)
online
#31-Aug-18 20:44

Off topic. I was at your bubble location a month ago. There is a terrific museum in Bandera, well worth a visit and stuffed with curiosities.

I haven't sent a suggestion , but being able to embed formatted bubbles or tables into kml/kmz is a useful thing to be able to do and something that I know Arc makes a good job of.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

KlausDE

6,204 post(s)
#25-Aug-18 10:24

Here is the German UI file for v 9.9.168.1

Attachments:
ui.de.txt

KlausDE

6,204 post(s)
#26-Aug-18 09:55

Being ahead of the times its v 9.0.168.1

adamw

8,061 post(s)
#04-Sep-18 16:10

Status update: we are planning to issue the next cutting edge build late this week. It will contain a big new addition for cartography.

adamw

8,061 post(s)
#10-Sep-18 08:20

A follow-up: we need a little more time. We have implemented most of what we wanted to implement and are working on the rough edges making the feature future-proof in terms of performance and extensions. We expect to be done in 3-5 days.

Mike Pelletier


1,508 post(s)
#11-Sep-18 01:59

Will it be worth the wait?

Of course it will! Thanks for these updates Adam.

Dimitri


4,980 post(s)
#18-Sep-18 19:28

Taking a bit longer but very close. Wanting to add more pre-built styles, etc. What you see below is all one layer, not stacks of layers. There are just endless, endless combinations with the controls already there, and yet more controls soon. Click the image to make it bigger.

Attachments:
styles_experiment.png

tonyw
459 post(s)
#19-Sep-18 00:04

adamw:

Status update: we are planning to issue the next cutting edge build late this week. It will contain a big new addition for cartography.

Will the cartography features include legends and scale bars? Presently I'm using M9 for most of the work then creating the layouts in M8. I know it's bad form, but sometimes I'll just take a screen shot from M9 and paste it into an email. Can't wait until I can produce the end product in M9. I'm sure the wait will be worth it.

Dimitri


4,980 post(s)
#19-Sep-18 07:19

Will the cartography features include legends and scale bars?

When the time is right, sure. Nothing of interest to cartography will be neglected in 9.

A gentle reminder: this is a Cutting Edge thread. It's a place to discuss builds that have been issued and to participate in the community driven process by saying what you want and what you think should come next, but it isn't a place to ask "what will be in the next build?"

If interested, the Cutting Edge way to answer such questions is to download the next build and see for yourself. :-)

geozap78 post(s)
#19-Sep-18 11:02

It would be great to have in m9 the flexibility and virtually unlimited options QGIS offers when it comes to symbolology. For example you can easily define a line style that consists of as many individual lines or point symbols you like, each with its color, width, dash pattern, offset from the line axis, draw order etc. Same flexibility exists with points or areas.

Dimitri


4,980 post(s)
#19-Sep-18 12:53

You'll have endlessly flexible styles like that in 9 and more, over time. It's not difficult building an endlessly extensible system. What takes more effort is building an endlessly extensible system that gives average people many options in a way that is easy to use.

It's also important to build the system in a way that all of the fine details work in all settings, all of the time. 9 does a good job at that, much better than 8: when something is declared to be 6 points or 11 points that's what it is on screen, in print and in PDF.

And last, but not least, the thing has to be incredibly fast, never crash, and be a good platform for maintainability and extensibility. To achieve that you can't just pile stuff on. It all has to derive from deeper foundations.

Lucky for all of us, 9 already has those deeper foundations: the core that is already there is a great foundation for a thousand times more than what the current GUI calls upon it to do. Lots of headroom to add whatever people want. Send in those Suggestions for whatever is your top priority.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.