Home - General / All posts - TileFilterVar
 dyalsjas147 post(s) #17-May-20 01:16 Quick question to confirm my assumptionDoes the TileFilterVar SQL function (and Variance template) provide the max minus min of values contained within the radius, filter shape, of the input image tile?What is the difference between the TileFilterVar and TileFilterVarPop, and Variance / Variance Population templates?
 tjhb9,223 post(s) #17-May-20 03:40 TileFilterVarPop should calculate the sum of squared differences between each value within the defined radius (weighted by the defined filter shape) and the average of all those values, the sum then being divided by the number of values (or sum of weights) including the centre value itself.TileFilterVar (without Pop, meaning sample) should calculate almost exactly the same, except at the end, where it will divide by the number of values minus 1 (or if weights are used, then by the sum of weights divided by the number of values minus 1, I think).The Pop version assumes we know all relevant neighbouring values; the sample version assumes that we don't (e.g. because NULLs are prevalent in the image, and the NULL values represent not true empty areas but unknowns).That is simplistic. Anyone who knows statistics better (that is ~everyone), please correct it.
 dyalsjas147 post(s) #17-May-20 14:25 Tim, thanks for confirming my assumption. It's been waaayyy too long since I thought hard about statistics.I can probably still use variance instead of the cheetah flips needed to subtract the output of a TileFilterMin from the output of a TileFilterMax.
 tjhb9,223 post(s) #18-May-20 03:38 I didn't think I had confirmed your assumption, but contradicted it. Perhaps we were not clear enough together?There is no direct relation between TileFilterMin orTileFilterMax and TileFilterVar(Pop).TileFilterVar(Pop) does not use Min or Max. It uses every value independently, and their combined average.They can't be substituted or translated one for another.
 tjhb9,223 post(s) #18-May-20 03:57 If you need the local range, then subtracting TileFilterMin from TileFilterMax is correct. Variance won't give a similar measure.Take a tile like1  1  11 19  11  1  1The minimum is 1, the maximum 19, so the range is 18. The average is (8 + 19) / 9 = 3. The population variance is ((1 - 3) ^ 2) * 8 + ((19 - 3) ^ 2) * 1 = 32 + 256 = 288.The population standard deviation is the square root of 288, slightly under 17.
 adamw9,135 post(s) #18-May-20 11:29 You forgot to divide the sum of squares by the number of measurements. So, the result is not 288, but 288 / 9 = 32.--SQL9VALUE @t TILE = StringJsonTile('[ 1, 1, 1, 1, 19, 1, 1, 1, 1 ]',  3, 3, 1, false); ? TileJson(TileFilterVarPop(@t, 1, TileFilterDefSquare(1, 1)))-- nvarchar: [--  null, null, null,--  null, 32, null,--  null, null, null-- ]:-)
 tjhb9,223 post(s) #18-May-20 21:03 Thanks! How silly of me.Better working of the example too.
 tjhb9,223 post(s) #17-May-20 10:49 So to answerDoes the TileFilterVar SQL function (and Variance template) provide the max minus min of values contained within the radius, filter shape, of the input image tile?more directly, no.
 adamw9,135 post(s) #18-May-20 11:13 As Tim says, TileFilterVar / VarPop compute sample variance and population variance.You seem to need Max - Min, so you need to do two calls: TileFilterMax(...) - TileFilterMin(...). We have been thinking about adding functions to compute that directly - perhaps calling the measure Range.
 dyalsjas147 post(s) #18-May-20 13:48 How can I tackle nesting these two functions (TileFilterMax and TileFilterMin) into a query?I have a set of models that sum/subtract the outputs of two or more filters, either from the results of a filter or from the data the filter operates on.For this one, I need to determine the range of values within a filter radius e.g. subtract TileFilterMin from TileFilterMax.If nesting two filters into a query is easy, then perhaps a function/template for range isn't necessary, a suitable example for the help docs would suffice.If there are performance benefits for having a specific function, then, yes, please.I think "Range" is an easily understood description.
 adamw9,135 post(s) #18-May-20 15:32 Well, wherever you are calling TileFilterMin or TileFilterMax, you can define a custom function called TileFilterRange and call that instead, with the same arguments:--SQL9FUNCTION TileFilterRange(@t TILE, @radius FLOAT64, @filter TILE) TILE AS  TileFilterMax(@t, @radius, @filter) - TileFilterMin(@t, @radius, @filter)END;We will still likely add built-in variants of various XxxRange functions, but until we do, you can roll your own.
 dyalsjas147 post(s) #21-May-20 16:41 So, I'm trying to roll a range function, and I'm starting with the Function description above, then trying to merge it into the generated SQL from the Maximum template. I also read the Process Images using Dual 3x3 Filters help topic.I've attempted to update the SQL from the Maximum template as follows:My source image is called [n30 UTM Source Image]My output table is called [n30 Tiles Range]My ouput image is called [n30 Tiles Range Image]My desired radius is 1My desired filter type is squareI suspect I've declared @t, @radius, and @filter incorrectly.Here's my attempt at the query, it does not parse.Thoughts? Ideas? Wry expressions at novice coding?--SQL9FUNCTIONTileFilterRange(@t TILE, @radius FLOAT64, @filter TILE) TILE AS TileFilterMax(@t, @radius, @filter) - TileFilterMin(@t, @radius, @filter)END;VALUE @t = [n30 UTM Source Image]VALUE @radius = 1Value @filter = 1CREATE TABLE [n30 Tiles Range] (  [mfd_id] INT64,  [X] INT32,  [Y] INT32,  [Tile] TILE,  INDEX [mfd_id_x] BTREE ([mfd_id]),  INDEX [X_Y_Tile_x] RTREE ([X], [Y], [Tile] TILESIZE (128,128) TILETYPE INT16),  PROPERTY 'FieldCoordSystem.Tile' '{ "Axes": "XYH", "Base": "World Geodetic 1984 (WGS84)", "CenterLat": 0, "CenterLon": -93, "Eccentricity": 0.08181919084262149, "FalseEasting": 500000, "LocalOffsetX": 500000, "LocalOffsetY": 3318785.352608442, "LocalScaleX": 27, "LocalScaleY": 27, "MajorAxis": 6378137, "Name": "Universal Transverse Mercator Zone 15 (N)", "ScaleX": 0.9996, "ScaleY": 0.9996, "System": "Transverse Mercator", "Unit": "Meter", "UnitScale": 1, "UnitShort": "m" }',  PROPERTY 'FieldTileSize.Tile' '[ 128, 128 ]',  PROPERTY 'FieldTileType.Tile' 'int16');CREATE IMAGE [n30 Tiles Range Image] (  PROPERTY 'Table' '[n30 Tiles Range]',  PROPERTY 'FieldTile' 'Tile',  PROPERTY 'FieldX' 'X',  PROPERTY 'FieldY' 'Y',  PROPERTY 'Rect' '[ 0, 0, 3574, 4122 ]');PRAGMA ('progress.percentnext' = '100');INSERT INTO [n30 Tiles Range] (  [mfd_id], [X], [Y],  [Tile]) SELECT  [mfd_id], [X], [Y],  CASTV ((TileRemoveBorder(TileFilterRange(TileCutBorder(@t, VectorMakeX2([X], [Y]), 1), 1, TileFilterDefSquare(@radius, @filter)), 1)) AS INT16)FROM [n30 UTM Source Image]THREADS SystemCpuCount();TABLE CALL TileUpdatePyramids([n30 Tiles Range Image]);
 adamw9,135 post(s) #21-May-20 16:58 Well, the query does not parse because VALUE declarations miss type - should be VALUE @t TABLE = [n30 ...], VALUE @radius FLOAT64 = ..., VALUE @filter FLOAT64 = ... (the numeric types could also be INT32, the conversion is automatic). You should also end each VALUE statement with a semicolon - depending on further text the query engine might find that the statement ended without a semicolon, but it is better to use one, otherwise you might accidentally make an error and the query engine will execute something other from what you intended.Frankly, since you are using each variable exactly once after it is declared, you could have just used the values you assigned to them instead.
 dyalsjas147 post(s) #21-May-20 18:01 Wry facial expressions at novice coding.... yep, well deservedCorrected the declared values and it works like a champ.
 tjhb9,223 post(s) #21-May-20 19:05 Frankly, since you are using each variable exactly once after it is declared, you could have just used the values you assigned to them instead.All the same, in my opinion two good reasons to declare them at the top are to make the given parameters more visible (they are not constants, but current assumptions, which can be nicely isolated), and to facilitate code re-use.I suppose that it only one reason really, but I like doing it that way too.A slightly different spin: I like using VALUE, and sometimes FUNCTION, even when they are not necessary, to leave breadcrumbs for a future self, to help understand what I did and to make arbitrary changes easier.
 dyalsjas147 post(s) #22-May-20 04:09 Adam (and Tim),Thanks for the guidance, your patience for noob errors is remarkable. I've included a working example of the query (with source data from the inport_dem_SDTS project in the Manifold.net examples.I've added a number of questions about declaring VALUES for Image or Table names in the range query. I didn't see an easy answer in the user manual.Finally, I asked earlier this week about subtracting the values in the Tiles of one band from the values in the Tiles of another band.To tie my interest to the Range function, I would like to combine the output of Range (filter radius 1) and Range (filter radius 4) by dividing the Tile values in each output by 2 and summing the results into a single band. See the Example Combine Query for step 1.Right now my only Manifold option is to export one of the bands as a set of points and performing a Drawing to Image join.Attachments: 2020_0521_Range_Example.mxb