Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System 9.0.168.9
adamw


8,634 post(s)
online
#16-Feb-19 15:56

9.0.168.9

The focus of the build was analysis. We originally planned to deliver the accompanying UI, but late in the build cycle we had a revolutionary idea on how to make it more powerful and at the same time much easier to follow and use. We didn't want to hold up the build so we are delivering the analysis features now and the UI will follow shortly.

manifold-9.0.168.9-x64.zip

SHA256: 7e5a3ef643e4be00e912fec5ad9033f99dc406d89341ed6c91396f50d4497368

manifold-viewer-9.0.168.9-x64.zip

SHA256: 577ae15db17616ebe4fc7c99a1ca21a381728851c4d9f370fa96bb6f9dc2a0ce

adamw


8,634 post(s)
online
#16-Feb-19 15:59

Changes

(Fix) JoinBitAnd, JoinBitOr, JoinBitXor query aggregates ran on a single value no longer fail to convert it to int32 (leaving whatever type it was).

(Fix) Median query aggregate ran on two values no longer returns a random value out of the two and always returns the highest one.

(Note: the behavior of Median on an even number of values is an implementation choice. 8 selects the lowest value of the pair of central values. 9 selects the highest value - for historical reasons. We discussed adjusting the code to selecting the lowest value like 8 does simply to avoid breaking compatibility for little reason. We might still do it, however we think that what is needed here more is a variant of Median that takes an average.)

Median query aggregate is optimized to perform significantly faster. Performance increases depend on the data type and are highest for numeric values.

Quick performance numbers for Median before and after optimization:

  • Median on float64: 15.580 sec before optimization -> 1.883 sec after optimization
  • Median on float64x2: 16.820 sec -> 3.114 sec
  • Median on nvarchar: 32.102 sec -> 21.943 sec

StDev, StDevPop, Var, VarPop query aggregates use much less memory. Corr, Covar, CovarPop query use memory more efficiently (less pressure for cache).

New query aggregates: Diversity, DiversityIndex, Major - all numeric. Diversity computes the total number of different values. DiversityIndex computes a measure of diversity: 1 - sum(individualcount^2) / (totalcount^2), 0 means that all values are equal. Major computes the most frequently occurring value.

(A word on limitations: Median, Diversity, DiversityIndex, Major are currently limited to 2 billion values. Corr, Covar, CovarPop are currently limited to 1 billion pairs of values. These limits are not new, Median, Corr, Covar, CovarPop were subject to them from the very beginning. We are going to remove the limits altogether in the future. StDev, StDevPop, Var, VarPop were also limited to 2 billion values before, we removed the limits for these aggregates in this build.)

New transfer rules for transforms like Overlay, Touching: diversity, diversity index, major.

New query function: TileFilterDefCircle - takes a radius and a center value, creates a definition for a circular filter.

Renamed query functions: TileFilterDefBlur -> TileFilterDefSquare ('square' better describes what the function does), TileFilterDefBlurDirection -> TileFilterDefDirection, TileFilterDefBlurGaussian -> TileFilterDefGaussian.

New transform: Average - a variant of Blur that uses a fixed center of 1. Provided for convenience, it might not be awfully clear that Blur with a center of 1 computes an average value.

New query functions: TileFilterMax, TileFilterMin - take an input tile, a radius and a filter definition, return a tile. For each input pixel, take values in the pixel vicinity corresponding to non-zero filter weights and compute maximum / minimum. New transforms: Maximum, Minimum.

New query functions: TileFilterSum - takes an input tile, a radius and a filter definition, returns a tile. For each input pixel, take values in the pixel vicinity and compute their weighted sum. New transform: Sum.

There are GPGPU versions of TileFilterMax, TileFilterMin, TileFilterSum. Limitations: radius must be at most 8.

New query functions: TileFilterDiversity, TileFilterDiversityIndex, TileFilterMajor - take an input tile, a radius and a filter definition, return a tile. For each input pixel, take values in the pixel vicinity corresponding to non-zero filter weights and compute diversity, diversity index, major value. New transforms: Diversity, Diversity Index, Major.

There are GPGPU versions of TileFilterDiversity, TileFilterDiversityIndex, TileFilterMajor. Limitations: radius must be at most 8, the number of non-zero coefficients must be at most 32.

(Note on pixel window computations like TileFilterMax / TileFilterMajor and others: there is a slight difference from 8 in handling invisible pixels. The surface evaluator tool in 8 was producing a visible pixel if at least one pixel in the vicinity was visible, this was making some of the invisible pixels visible. In 9 we changed that so that invisible pixels stay invisible and visible pixels generally stay visible and only turn invisible if non-zero filter coefficients all fall onto invisible pixels - so, if filter center is non-zero, then visible pixels will always stay visible.)

Blur, Diversity, Diversity Index, Major, Maximum, Median, Minimum, Sum transforms for images specify filter shape using a dropdown control. Available shapes: square (default), circle, cross. Blur, Cross and Median, Cross transforms are removed (no longer needed).

New query functions: TileGeomAvg, TileGeomCount, TileGeomDiversity, TileGeomDiversityIndex, TileGeomMajor, TileGeomMax, TileGeomMedian, TileGeomMin, TileGeomSum - take an image and a geom, and compute specified statistic on all pixels under the geom. The image must have a single channel. The geom must be in the coordinate system of the image.

Notes on geometry to raster conversion, mostly compatible with 8:

  • each pixel is only counted once, even for multipoints which fall onto the same pixel - this is consistent with not counting pixels multiple times for lines and areas even if they have multiple coordinates which fall onto the same pixel, same as in 8 (save for multipoints which were impossible),
  • pixels for lines do not necessarily include pixels for line ends if they are converted to points (sometimes a neighbor pixel is a better match) - same as in 8,
  • pixels for areas do not necessarily include pixels for area boundary if it is converted to a line (horizontal shapes thinner than 1 pixel might be ignored) - same as in 8.

In addition, pixels for areas are computed more accurately than in 8 - 8 used to perform a trick to ensure correctness that we no longer need.

Ultimately, like in 8, pixel shapes covered by a geom are an approximation that aims to be reasonably good without being slow. We will likely provide versions of functions that compute the share of a pixel covered by a geom in the future, for more accurate computations, should these be required.

Quick performance numbers for TileGeomAvg in 8 and 9:

  • Transferring heights from 10k x 10k pixels to 8142 small areas: 8 = 37.800 sec, 9 forced to use single thread = 7.835 sec, 9 with 8 threads = 1.769 sec, max difference in the result between 8 and 9: 9.3e-11 with the height range of 100-1500 (9 counts slightly more accurately, as above).
  • Transferring heights from 10k x 10k pixels to 1050 big areas: 8 = 74.050 sec, 9 forced to use single thread = 7.256 sec, 9 with 8 threads = 1.680 sec, max difference in the result between 8 and 9: 4.7e-11 with the height range of 100-1500.

We do not currently expose transforms for transferring heights from images to drawings - this is coming - so here's an example query:

--SQL

 

-- prepare to convert coordinates from drawing to image

VALUE @conv TABLE = CALL CoordConverterMake(

  ComponentCoordSystem([image]),

  ComponentCoordSystem([drawing])

);

 

-- set 'height' field in drawing to average height taken from image

UPDATE (

  SELECT [mfd_id][height],

    TileGeomAvg([image], CoordConvert(@conv, [geom])) AS [computed]

  FROM [drawing]

  THREADS 8

SET [height] = [computed];

Image servers support {-y} escape sequence in custom URLs (reverses Y direction within level).

Importing SDTS data reads Z scale and offset. Exporting SDTS data writes Z scale and offset.

End of list.

Dimitri


5,491 post(s)
#18-Feb-19 09:31

Don't be put off by using a quick SQL query to transfer heights from images to drawings. It's much easier than you might think. There's a new, very short example topic that shows how: Example: Transfer DEM Terrain Heights to Areas in a Drawing.

lionel

567 post(s)
#18-Feb-19 09:34

since meany months i have offset in mind ... what mean local offset in different context and even understand what tool can be use to define () or update ( shift transform) I can't have an image in my mind !! In which direction offset can be define ? only vertical and horizontal for projection ( shift) but for edge width of shape border in a drawing ...Does the computation is done using for reference start the middle of width border ?

from manifod 9 doc offset return many links but sometimes no explanation only use the term 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

https://en.wikipedia.org/wiki/Offset_(computer_science)

https://en.wikipedia.org/wiki/UTC_offset

a offset seem to be group a of value than can use in a specific way to have another value .

I think manifold doc "try to avoid" the use of the name of border or width ( use in ui) to explain thing ( in the good sense ) !!

why local near offset ? there is global offset ? like global projection that at suitable for earth compâre local projection choice suitable for preserved distance angle .....

image from manifold doc about composeRect transform from point (0,0) here with offset 40h,20v

regard's

Attachments:
composeRectangleOffset.png
offset_style.png


join image

"Because my dad promised me" ( interstellar ) but blackhole don't exist

best hardware with no ads focus on quality features price like manifold see xiaomi

lionel

567 post(s)
#18-Feb-19 09:48

think my post should be a new thread /post !! sorry


join image

"Because my dad promised me" ( interstellar ) but blackhole don't exist

best hardware with no ads focus on quality features price like manifold see xiaomi

adamw


8,634 post(s)
online
#25-Feb-19 13:08

Status: we are planning to issue the next cutting edge build at the end of this week. The focus is various UI features, plus a couple of additions to databases and images.

adamw


8,634 post(s)
online
#28-Feb-19 05:55

One more update: the build will cross a few days into the next week in order to incorporate a big chunk of changes and additions related to Unicode.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.