Subscribe to this thread
Home - General / All posts - Type mismatch using sign()
JoeyD21 post(s)
#11-Nov-20 07:56

I want to turn a DEM into an image only consisting of the values 0 and 1, divided by a height H. I.e. values above H should have the value 1 and all others the value 0. The data type of the source image is int32 stored in 128x128 tiles. I thought to do this with a transform with the following expression

sign( [TILE] - H )

followed with a transform lower limit = 0. The "[TILE] - H" part of the expression seems to work fine. When I enter the sign() function into the expression editor I always get a "sign": type mismatch error when trying to leave the editor with OK. This happened for all destination data types that I tried. I would assume that the data type int8 would be appropriate to store the results of the sign() function: +1, -1, or 0. Where is the problem that I don't see?

tjhb

9,475 post(s)
#11-Nov-20 08:47

I would guess you have some invisible pixels. (Around tile edges for example, which are normal to ensure image alignment.)

Try using

Sign(Coalesce([Tile] - H, -1))

tjhb

9,475 post(s)
#11-Nov-20 19:29

Silly me. I am very good at missing the obvious. (An expert.)

The type mismatch is due to passing a tile to the function Sign(), which needs a scalar argument.

To get the sign of (every pixel in a) Tile, you need TileSign() instead.

(And yes, the result of <tile> minus a constant is implicitly <tile>.)

JoeyD21 post(s)
#12-Nov-20 22:23

OK thanks, this hint worked!

Manifold User Community Use Agreement Copyright (C) 2007-2019 Manifold Software Limited. All rights reserved.