Subscribe to this thread
Home - General / All posts - New video showing how to create custom filters
Dimitri


7,411 post(s)
#08-Feb-19 14:24

There's a new video on the Gallery page, together with an accompanying example project containing all the queries used in the video. It's a long video that shows how to create custom filters, giving live examples of how to create new Emboss, edge detection and other filters, all of which are automatically GPU accelerated.

Simple copy/paste and a few adjustments to the examples lets you add some of the very many convolution matrix filters you can find on the web for image processing. The video also shows how to do multi-step filters, like Sobel, Prewitt, and Kirsch edge detection, where multiple convolution matrix filters are used.

It seems technical, and the first time you try it may take a few minutes, but even if you don't really know how the StringJsonTile function works, just by plugging the numbers you find on the web for a filter of interest you can very easily, very quickly, create your own custom filter that runs at GPGPU speed. With a bit of practice it goes very quickly, enabling creation of new filters in just a minute or so.

lionel

995 post(s)
#08-Feb-19 15:02

error


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

lionel

995 post(s)
#08-Feb-19 16:11

wouah i always learn new things and a lot with manifold !!

manifold support image filter like photoshop but in a coding SQL way using tiles and no gui menu

all is about convolution matrix /image kernel ( gimps , setosa.io,photoreactor, wiki )

https://en.wikipedia.org/wiki/Image_embossing

https://en.wikipedia.org/wiki/Canny_edge_detector

http://rastergrid.com/blog/2011/01/frei-chen-edge-detector/

https://en.wikipedia.org/wiki/Gaussian_blur

http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/

https://en.wikipedia.org/wiki/Edge_detection

https://en.wikipedia.org/wiki/Kirsch_operator

Attachments:
photoshopCS6_filter.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Dimitri


7,411 post(s)
#08-Feb-19 18:14

manifold support image filterlike photoshop but in a coding SQL way using tiles and no gui menu

Yes and no. The Transform functions encapsulate image filters in a gui menu, very point and click, no SQL required. There are many, many more filters coming.

Manifold could easily do a custom filter dialog like the ones illustrated from Photoshop. That's probably not a bad idea. But what's interesting about the system today is you don't have to wait for that: you can do custom filters today, and with even more flexibility than the usual "fill in the boxes" GUIs people do in graphics editors like Photoshop or GIMP:

1. The "fill in the boxes" GUI is limited to only radius 1 (3x3) or radius 2 (5x5). You can't do radius 3 (7x7) or greater, but it's trivial in Manifold to do whatever radius you want.

2. With a "fill in the boxes" GUI, it is not possible to create multistep filters like shown in the video for Sobel, Prewitt or Kirsch. If you start with the example project in the download, those are easy to do.

3. The ability to utilize arithmetic and functions in a filter is just superb. When you read the casual literature (people writing for each other on Internet) you run into a lot of hand-wringing over the complexity of programming what in the end are very simple things. The Sobel filter for example, is just squaring two different filter results, adding those squares together and then taking the square root of the sum. It's amazing you can do that in Manifold as simply as...

(TileFilter(@t, 1, @f1)^2 + TileFilter(@t, 1, @f2)^2)^0.5

... using the + and ^ operators just like you were working with ordinary numbers and not tiles.

It's also very, very convenient to be able to do things like toss a CASTV in there to make the data what type you want it to be, use ordinary arithmetic operators to scale as you like and so on. Anyway, I think all that opens the doors to ad hoc experimentation and invention of new types of filters, because now it is so easy to try out something sophisticated that before might have been too much of a hassle to code.

The ESRI blogs, for example, often talk about combining things like Plan and Profile curvature results, but with Manifold you can do it as part of a special filter you want, and not have to combine manually.

I agree it would be fun to have a "fill in the boxes" dialog for simple filters, but I'm glad the system started with a fully open and accessible way to do this in SQL. Plus... it's all GPU parallel and CPU parallel. :-)

Mike Pelletier

2,122 post(s)
#08-Feb-19 20:17

I like that you can do this in the Expression tab so that you can see the result quickly in your map without having to load up a new component. You can also include all the border removing functions as well. Toogle back and forth between the Expression and Template tab acts like a Preview checkbox. Certainly plenty of room for more user friendliness but wow its really powerful.

How do we convert a BGRA image to single channel pallette type image?

Dimitri


7,411 post(s)
#09-Feb-19 05:50

Add a column to the table that is a Tile type using the desired data type (instead of uint8x4, make it uint8, for example), and then do SQL arithmetic on the channels in the uint8x4 tile to say what the value in in the uint8 tile should be.

The simplest way is to average the R, G, B values. Just add them up and divide by 3.

Discard alpha (if it has an alpha channel it is not a single channel palette image, at least not in Manifold) and then for the remaining channels add them up and divide the result by three.

(TileChannel([Tile], 0) + TileChannel([Tile], 1) + TileChannel([Tile], 2))/3

If [Tile] is your RGBA data, and [TileOne] is your one channel tile, the above expression I think would do the trick in the Expression tab when applied to TileOne as the target.

[Forgot the channel collapsing... will add in companion post]

There are, of course, more sophisticated ways to derive grayscale from RGB if you know something special about the various channels, like whether one should be weighted more than another, clever adjustments for how the human eye responds to different colors and so on. But averaging is the usual way.

Dimitri


7,411 post(s)
#09-Feb-19 06:08

It's slightly more complicated, since the expression is correct but you want to deal with the infrastructure of how to use CASTV to create a UINT8. The expression is:

CASTV (((TileChannel([Tile], 0) + TileChannel([Tile], 1) + TileChannel([Tile], 2))/3) AS UINT8)

But the best way to use that is to open your RGBA image, in the transform panel choose Channels, press the Edit Query button and then replace the simple, one channel CASTV line with the above. You get the following, which creates a grayscale image that averages the RGB channels. Call me lazy, but I love that Edit Query button. :-)

--

-- Auto-generated

-- Transform - Channel - Add Component

--

CREATE TABLE [m_4012230_nw_10_h_20160717 Tiles Channel] (

  [X] INT32,

  [Y] INT32,

  [Tile] TILE,

  [TileOne] TILE,

  [mfd_id] INT64,

  INDEX [mfd_id_x] BTREE ([mfd_id]),

  INDEX [Y_X_x] BTREE ([Y][X]),

  INDEX [X_Y_Tile_x] RTREE ([X][Y][Tile] TILESIZE (128, 128) TILETYPE UINT8),

  PROPERTY 'FieldCoordSystem.Tile' 'EPSG:26910,mfd:{ "LocalOffsetX": 552708, "LocalOffsetY": 4490226, "LocalScaleX": 0.6, "LocalScaleY": 0.6 }',

  PROPERTY 'FieldTileSize.Tile' '[ 128, 128 ]',

  PROPERTY 'FieldTileType.Tile' 'uint8'

);

CREATE IMAGE [m_4012230_nw_10_h_20160717 Tiles Channel Image] (

  PROPERTY 'Table' '[m_4012230_nw_10_h_20160717 Tiles Channel]',

  PROPERTY 'FieldTile' 'Tile',

  PROPERTY 'FieldX' 'X',

  PROPERTY 'FieldY' 'Y',

  PROPERTY 'Rect' '[ 0, 0, 9410, 12140 ]'

);

PRAGMA ('progress.percentnext' = '100');

INSERT INTO [m_4012230_nw_10_h_20160717 Tiles Channel] (

  [X][Y][TileOne],

  [Tile]

SELECT

  [X][Y][TileOne],

  CASTV (((TileChannel([Tile], 0) + TileChannel([Tile], 1) + TileChannel([Tile], 2))/3) AS UINT8)

FROM [m_4012230_nw_10_h_20160717]

THREADS SystemCpuCount();

TABLE CALL TileUpdatePyramids([m_4012230_nw_10_h_20160717 Tiles Channel Image]);

Mike Pelletier

2,122 post(s)
#09-Feb-19 20:33

This is all super helpful stuff including the NDVI example! Your enthusiasm for what can be done is great and I too really like the Edit Query button and how it lets us learn the inner workings a bit. I'm lazy too :-)

Is it possible at this time to get a one channel image that looks much like the original, such that each value is preassigned a color from the palette of the original? My goal is to ultimately classify the pixel values with various image filters and convert to vector areas.

Dimitri


7,411 post(s)
#10-Feb-19 12:56

Is it possible at this time to get a one channel image that looks much like the original, such that each value is preassigned a color from the palette of the original?

Sure. That's converting an RGB image into an indexed color image (single channel image with an accompanying palette). Using SQL almost anything is possible, especially if you use an embedded function within the SQL. :-) You can add a StylePixel property with whatever styling value you want to a new component that you create. See examples in my new post on NDVI. But that's a lot of work to, basically, write a function embedded in your SQL that basically converts an RGB image into a palette image. Unless you need that right now, I'd wait until Manifold does it as part of the big images/raster cycle.

My goal is to ultimately classify the pixel values with various image filters and convert to vector areas.

You don't need to make it look the same to use Trace Areas. I'd suggest applying the filters you want to use and then use Trace Areas against the final result. Color the resulting areas as desired.

adamw


10,447 post(s)
#20-Feb-19 07:47

If we are talking about taking a color image (RGB / RGBA) and converting it to an adaptive palette using most frequently used colors + choosing for best contrast + with or without dithering, then we don't have such a function yet. But once you go from colors to color indexes with an adaptive palette, you won't be able to apply filters, you'd have to be translating color indexes back to colors to do that.

If we are talking simply about collapsing colors into grayscale using intensity or something similar, that's fine, that's going to allow filtering - and that's doable right now - eg, instead of summing channel values and dividing them by 3 like Dimitri's query does above, we could apply weights: TileChannel(tile, 0)*0.11 + TileChannel(tile, 1)*0.59 + TileChannel(tile, 2)*0.3, if channels are 0=B, 1=G, 2=R. (We'll add a separate query function and transform to do this, yes.)

Dimitri


7,411 post(s)
#09-Feb-19 06:36

A quick comment in addition to the other answers: In answer to the question "Can I have a one click converter of image types?" The answer is "sure!" ... when Manifold cycles though the big images evolution that will happen sooner rather than later.

Keep in mind that 9 is evolving from a spatial data engineering environment in which absolutely everything is possible. The key thing is to build that infinite engine, which can do anything first, and then, second, add short cut conveniences in the order in which they are desired. The point-and-click transforms, as the Edit Query button shows, are just canned conveniences which provide a pre-written, special case of very general capabilities.

If you create the infinite engine first, then it's easy to add zillions of Photoshop style conveniences in very short order on an assembly line basis, as you can see from a quick look at the filters in the example project for the most recent video. But if you hard-code that into a graphics editor like Photoshop, you can't go from there to infinite engine infrastructure. So it makes sense to do the general purpose engine first and then, second, build it out.

So there's nothing difficult about any of the Photoshop stuff for image editing. It's all easy to do given 9/Radian infrastructure. But there is a lot of it and a lot of it hangs together in groups of features, where you really want to do a few dozen related features all together at the same time for features within that group to make sense. That will come in the not too distant future as Manifold goes through a big images cycle.

Dimitri


7,411 post(s)
#09-Feb-19 11:35

One last example... You can use identically the same workflow to create an NDVI display from four band NAIP. The example I used for grayscale is from the image I used in various comments about Color Infrared (CIR) displays from NAIP. That's where the image name came from.

So, here's how we do NDVI:

1. Open the four band image. (make sure it's one that's already been inverted, or which uses the build that is going to come out next week which brings in the near-IR band without any need to invert).

2. In the Transform panel click on the Channel transform and then choose Edit Query so it writes the query for you. I love that button. :-)

3. NDVI is a single band image created from the Red band and the near-IR ("NIR") bands of a four band image. The Green and Blue bands are not used. The formula to create the single band in NDVI is trivial:

(NIR - Red) / (NIR + Red)

In Manifold, here is the line that does the calculation, ready to drop in as a replacement for the CASTV line in the query created by the Edit Query button:

CASTV (((TileChannel([Tile], 3) - TileChannel([Tile], 2 )) / (TileChannel([Tile], 3) + TileChannel([Tile], 2 ))) AS FLOAT64)

We need to cast it as a float64 because NDVI is within the range of -1 to 1, with all the action of interest happening from 0.2 to around .9.

In addition to making that change, you also have to change every place where it says UINT8 or uint8 to FLOAT64 or float64. The resulting query is below. Run it and it creates NDVI. For the image to make sense, you have to style it and apply a palette. The easiest way to do that is to right-click on the image, choose Properties and in the Properties dialog add a StylePixel property with the value one of the text strings below. There is no standard for NDVI so people use all sorts of things. Here are some I've used:

Useful for leaf cover from http://web.nmsu.edu/~tcarrill/what_is_ndvi.htm:

{ "Channel": 0, "Fill": "boundaverage", "Value": 4333842, "Values": { "-0.31": 4333842, "0.05": 10441002, "0.16": 13412885, "0.25": 16580099, "0.3": 15068672, "0.35": 13688576, "0.4": 12177152, "0.45": 10665984, "0.5": 9154560, "0.55": 7708160, "0.6": 6196736, "0.65": 4554752, "0.7": 2977792, "0.75": 1532160, "0.8": 20992 } }

USDA Crop Explorer palette:

{ "Channel": 0, "Fill": "boundaverage", "Value": 15183459, "Values": { "0": 15183459, "0.1": 16769940, "0.2": 15204269, "0.3": 10289052, "0.4": 6553443, "0.5": 3264305, "0.6": 39424, "0.7": 25856 } }

A classic palette as used by many third party applications, applied to the USDA intervals:

{ "Channel": 0, "Fill": "boundaverage", "Value": 14102567, "Values": { "0": 14102567, "0.1": 16018755, "0.2": 16625249, "0.3": 16703627, "0.4": 14282635, "0.5": 10934634, "0.6": 6733155, "0.7": 1742928 } }

The CE Spectral palette applied to the USDA intervals:

{ "Channel": 0, "Fill": "boundaverage", "Value": 13975119, "Values": { "0": 13975119, "0.1": 16018755, "0.2": 16625249, "0.3": 16703627, "0.4": 15136152, "0.5": 11263396, "0.6": 6734501, "0.7": 3311805 } }

Here is the query:

-- $manifold$

--

-- Create NDVI

--

CREATE TABLE [m_4012230_nw_10_h_20160717 Tiles Channel] (

  [X] INT32,

  [Y] INT32,

  [Tile] TILE,

  [mfd_id] INT64,

  INDEX [mfd_id_x] BTREE ([mfd_id]),

  INDEX [Y_X_x] BTREE ([Y][X]),

  INDEX [X_Y_Tile_x] RTREE ([X][Y][Tile] TILESIZE (128, 128) TILETYPE FLOAT64),

  PROPERTY 'FieldCoordSystem.Tile' 'EPSG:26910,mfd:{ "LocalOffsetX": 552708, "LocalOffsetY": 4490226, "LocalScaleX": 0.6, "LocalScaleY": 0.6 }',

  PROPERTY 'FieldTileSize.Tile' '[ 128, 128 ]',

  PROPERTY 'FieldTileType.Tile' 'float64'

);

CREATE IMAGE [m_4012230_nw_10_h_20160717 Tiles Channel Image] (

  PROPERTY 'Table' '[m_4012230_nw_10_h_20160717 Tiles Channel]',

  PROPERTY 'FieldTile' 'Tile',

  PROPERTY 'FieldX' 'X',

  PROPERTY 'FieldY' 'Y',

  PROPERTY 'Rect' '[ 0, 0, 9410, 12140 ]'

);

PRAGMA ('progress.percentnext' = '100');

INSERT INTO [m_4012230_nw_10_h_20160717 Tiles Channel] (

  [X][Y],

  [Tile]

SELECT

  [X][Y],

  CASTV (((TileChannel([Tile], 3) - TileChannel([Tile], 2 )) / (TileChannel([Tile], 3) + TileChannel([Tile], 2 ))) AS FLOAT64)

FROM [m_4012230_nw_10_h_20160717]

THREADS SystemCpuCount();

TABLE CALL TileUpdatePyramids([m_4012230_nw_10_h_20160717 Tiles Channel Image]);

Enjoy!

lionel

995 post(s)
#10-Feb-19 13:25

from manifold doc CASTV

castv 12( 3 sql)

CASTV (<value> AS <type>)

Takes a vector or tile value and converts the data type of all contained values to the specified type.

CASTV ([vector-of-float64] AS INT32)

Produces a vector of int32 values.

CASTV ([tile-of-uint8x3] AS FLOAT32)

Produces a tile of float32x3 values. The number of channels does not change.

1) so CASTV apply not on one value like CAST but group of same type value (ie vector ) .

2) Does it works for typex3 typex4 ... typexn ?

3) I think there is a logic about cast ( upgrade , downgrade , truncation ). some cast don't make sense since we loose information ( refer to memory space precision )

4) in this post i paste content that define vector that can contain different type ! but manifold vector not ?!!

Does vector in manifold 9 is a array ( same type value )


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

adamw


10,447 post(s)
#20-Feb-19 07:52

Regarding 2 - does CASTV work for xN types - yes, it does:

CASTV (VectorMakeX2(1.1, 2.2) AS INT32) produces [ 1, 2 ].

RonHendrickson
283 post(s)
#10-Feb-19 16:16

Wonderful video, guys. Let's have more like that, I like the technical training in SQL.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.