Subscribe to this thread
Home - General / All posts - understand image tile channel
lionel

995 post(s)
#13-Apr-20 22:54

the first time i use manifold i test how concat work on <geom> and understand that concat 2 lines in one is in final create a new list of coordinate . For this i use wkt format !

I have the same question with raster that is <tile> . in a way i shure that this tile contain a list of value . So the 128x128 is not refer to visible size area of 128x128 pixel. Since image can contain many layer Does it mean that each tile contain many layers ? . For understand <tile> i need to use SQL function like i do to study <geom> htat have tile in argument and some text in output ( json or table with char content) .

I test SELECT TileJson([Tile]) FROM [color_32bit Tiles];

so each ell contain [ , , ] with many 0< value< 255 separate by ","

All image files 4 16 32 bit have 4 raws in the table , but number of values change for each row using differents bits

32 bit => 65 538 items ( number of ",") /raw 16x16x16x16= 65 536

24 bits => 49151 items (number of ",") /raw 65 536 - 16384 = 49 152

4 bits => 16383 items (number of ",") /raw 65 535/4 = 16 384 =128x128 like in manifold tile

is there a way to see the real content after the ... in the end of pop up of cell inside manifold ?

The editor seem limited to some content compare to external editor like notepad or wordpad or word !

I really like the manifold documentation since the beginning because describe in a simple way with many capture screen , infographic how is structure data (geom and image cover by manifold doc) and tile ( not find it) .

Is there a page about Tile & image & channel in manifold documentation ?

Which raster file alone on OS can manage projection ( embedding projection inside its own file format like pdf with some custom selected fonts )

thank's

----------manifold doc-------------------

channels 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109

Tiles

Example: How Images use Tiles from Tables

Transform Templates - Tile

Tilted Perspective Projection

Attachments:
color_manifold-json.png
color_test.map
color_use_inimage4-24-32bits.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Dimitri


7,413 post(s)
#14-Apr-20 04:12

Tiles do not contain many layers. An image is a single layer that consists of pixels, organized into tiles. Pixels can have different data types, since pixels in some images are a single value (single channel) or a palette index, or a vector (multiple channels). That doesn't mean they are multiple layers, just one layer where the value in each pixel is a vector (a set of values), not a scalar (a single value).

See the illustrations in the Images topic. Get at data within tiles using Tile... SQL functions.

Most operations on rasters are done using tile functions since it is more efficient to fetch and to operate on many pixels at once than it is one pixel at a time. For examples, see the series of matrix filter examples, like this one.

Matd4 post(s)
#14-Apr-20 14:38

Hi, I have a question on image tiles.

Have been producing large image mosaics around 200K by 200K pixels, up around 97 GB uncompressed and they will not show up as an image when imported by Manifold 9.0 just a list of tiles. Smaller orthos 100K by 100K pixels seem to be Ok. Any tips on how to load them?

Many Thanks Matt

Dimitri


7,413 post(s)
#14-Apr-20 16:06

What format are you trying to import?

Matd4 post(s)
#15-Apr-20 10:57

Geotiff, uncompressed and un tiled

Dimitri


7,413 post(s)
#15-Apr-20 11:32

A 97 GB GeoTIFF could decompress into a terabyte, bigger than the current 256 GB limit on the size of a .map file. I don't quite get that limit, since I know full-planet PBF imports generate a project that is about a terabyte in size. I'll ask around.

If you are using 32 bit Manifold, the limit is 64 GB, which would be a show stopper. Other possibilities are running out of free space on your hard disk.

1. What is the build you are using? What version of Windows? How much free space do you have on your hard disk?

2. What exactly do you do, step by step? For example, do you import or do you link, and whatever you do, what are all details of the procedure you use?

3. Where is the GeoTIFF... local on your own hard disk, or coming over a network?

4. What is the result in the Project pane, in terms of components created?

5. Opening up an image does not show a list of tiles. Opening a table shows a list of tiles. Could it be you are opening the table and not the image?

6. If you open up an image, what happens?

Matd4 post(s)
#15-Apr-20 13:27

1. I was using the latest Manifold build 64 bit, Windows 10 1803, Heaps of space about 500 GB on the OS and 20TB of storage (12 HDD). I have just done fresh install of windows - so will reinstall manifold and give it another crack.

2. Open Manifold, Select Import - Choose the image (takes a while to load) and then there is no actual image layer just a tiles layer. On smaller images there are two layers in the Project pane and image layer and below that a tile layer. With the big images just the tiles.

3. Local Nvme m.2 drive

4. Just the tiles

5. There is no image to open inside of Manifold once imported just tiles - I can view it in other software.

adamw


10,447 post(s)
#15-Apr-20 14:42

Are there any complaints in the log window? (See View - Panes - Log Window.)

What happens if you try linking the file instead of importing? Do you see the image component in the Project pane? If yes, do you see the image data if you open it?

Can you reproduce the problem on a smaller image (eg, 1 GB)? If so, it'd be easiest to upload that smaller file to tech support so they could take a look at what exactly is going on.

Matd4 post(s)
#16-Apr-20 02:48

A fresh install of windows and Manifold did the trick. I have it loaded now.

Thanks

Matt

Dimitri


7,413 post(s)
#17-Apr-20 04:10

A 97 GB GeoTIFF could decompress into a terabyte, bigger than the current 256 GB limit on the size of a .map file. I don't quite get that limit, since I know full-planet PBF imports generate a project that is about a terabyte in size. I'll ask around.

OK, I asked around and the Limitations topic had two errors in it, since corrected. A .map file can be up to 64 TB (terabytes) in size, not 64 GB (gigabytes). Amazing what a difference a single letter makes. :-). Here is the new, relevant info:

64 TB max size for .map files - Using either 64-bit or 32-bit Manifold, the maximum amount of data stored within a .map file cannot be larger than 64 Terabytes (TB), although a .map file can contain data sources linking to a DBMS that contain far larger data, petabytes, in size. As a practical matter, the most pressing limitation on the size of data is the limit of 2 billion records per table. That will be increased in future builds.

256 GB max size for temporary databases within projects - The amount of data stored within temporary databases created on the fly for the duration of a query cannot be larger than 256 GB, in 64-bit Manifold and 64 GB with 32-bit Manifold.

lionel

995 post(s)
#29-Apr-20 01:49

Sub image (tile) is in a way a grid or table with column and row.cell contain vector data .

1) Does number of row and column is define contain by the structure ( what i think) or define using metadata?

2) we visually see in documentation that the image that is shown is define by 4 tiles = 4 area=4 ROW but store in the same SQL type ! How are choose the 2 (values in this specific case) of row and column for each 4 area ? I think area should perhaps be a multiple of 2 : 2 4 8 Todo list or middle of midle)

( I understand that each cell can contain a list ( separate by virgulecomma ) of 1 to n value ( separate Spaceby ) .

3) is it exact to write that for uncompress image , the size of an image is fix and relative the number of cells and memory allocated for celll not The differents values sstore inside this cell !

4)saying I use a 16 bit image don' t inform us about how the structure of data is (in cell) ? (1 bit = 0 or 1=> 2 choices)

-. 4 4 4 4 0<X<16

- 8 8 0<X<256

And 16 bit refer to the size of item in vector ( what I Truththink )

-16 16 .16 for rgb......

-16 16 16 16 for rgba

Thank..

BIG IMAGE ROW COL --- tile

Sub image row col --- channel


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

lionel

995 post(s)
#17-May-20 04:17

i read again your comment and this week i use and know ( i was thinking to know ) tiff format since many years ....but discover in my daily work that tiff create from fax communication can contain many pages or raster image or many layers . So same thing but use differents words , wikipedia use the word subfiles not layer or channel !! I add this post now because there is a question in this same post about geotiff and was thinking tiff is a old past file format and not use a lot nowadays ( may be wrong )

so Does a tiff with 4 pages so do i write something wrong if i say that a tiff file with 4 ( images = pages) is one image with for each pixel of "4 channel"= "vector of 4 values " or tiff data structure is different ?!!

Many tools can be use to open this file in windows 10 :paint, paint 3D, Photos, sketchBook ( autocad), Snip & Sketch , Windows Photo Viewer . But all those image software even they can open tiff file don't mean that they let us see all the content of a RICH = multiple page tiff file !!

--Paint.net open show only first page no acces to other pages

--Paint3D open show only first page no acces to other pages

--Autodesk sketchBook pro open rich tiff but some issues could exist if we edit using differents images tools .Note:

Only the TIFF and PSD image formats preserve layers. Saving them as any other image format flattens them. Only use Autodesk SketchBook to read TIFF files containing layers. We do not advise you to open these images in other programs because we cannot guarantee layer preservation.

--Paint3D open show only first page no acces to other pages

--Microsoft Photos open show only first page no acces to other pages

--Snip & Sketch open show only first page no acces to other pages

So by default there is no softwares by default in windows 10 that can open rich tiff . And microsoft support forum post is not usefull compare to a google that link (How to Restore Missing Windows Photo Viewer in Windows 10 ) that describe to a step by step process to enable Windows Photo Viewer(=WPV) . It is very strange that this file format is not supported by default in windows 10 OS since fax communication are still use in business in 2020 !! shure we use PC connect to copy/fax/Scan device to manage fax to avoid to print in real time all receive fax to papers !!.

I use Total commander to find if *.dll ( dll= dynamic-link library) to know the differents paths and versions available in my windows 10 OS ( for imageres.dll : 4 path but 1 one version 10.0.18362.1 and is not the same compare to tutorial link) . imageres.dll is can be use by many software like WPV but WPV is hidden in windows 10 OS so we need to import a custom reg file using regedit to make WPV accessible / runnable ! ( i do a *.reg backup /export size of 330 Mo before run the custom *.reg . The process work ...see capture screen before /after !

Attachments:
before_after.png
imageres.dll_path-version.png
multipage_tiff_example.tif
tiff_opewith_window10.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

lionel

995 post(s)
#17-May-20 04:18

Attachments:
after_test.png
reg-custom_import-regedit.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

lionel

995 post(s)
#17-May-20 04:59

Attachments:
before_after.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

lionel

995 post(s)
#17-May-20 05:20

--manifold doc ---------

tif 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

tiff 1 2 3 4 5 6 7 8 9 10 11 12 13 14

strange tif is not the same than tiff ? ( why create many extension/ mime type for same thing ? )

-A classic example are image formats such as TIF or LiDAR storage formats which often store data in compressed form

-Some formats, such as TIF images, specify alpha as opacity, so that is how Manifold imports alpha channel data for TIF, interpreting 255as full opacity. Other packages and formats use transparency for alpha, and sometimes alpha data in TIF is stored using transparency.

-The zip file unzips into a GeoTIFF with .auxand.tfwsidecar files. The GeoTIFF provides correct georeferencing information, but the two sidecar files provide bogus information to override the coordinate system information in the GeoTIFF. To import the file into Manifold, first delete the auxand.tfwsidecar files. The .tifwill then be imported with the GeoTIFF coordinate system information and will be accurately georegistered.

-Setting aside all concerns about malware the hiding of extensions complicates work in GIS and DBMS because so many different file formats are used in with extensions that are unknown to Windows, or which have misleading associations with consumer programs. It is important to be able to see what a three letter extension is to know what a file is likely to be, and not to have to guess what, say, a TIF file is based on whether or not it has a Microsoft Paint or PhotoShop or whatever icon on it.

-20 : TIF,TIFF,GeoTIFF

where is locate the metadata projection for geo inside tiff file ?


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Dimitri


7,413 post(s)
#17-May-20 06:53

When there is an interest in a specific format, make sure to read carefully the topic for that format, for example, the TIF, TIFF, GeoTIFF topic in the Formats and Data Sources section of the documentation.

The topic discusses some of the points you raise, for example, the difference between TIF and TIFF: "(TIFF or TIF, same thing)", and the discussion of alpha channels in TIF. The key point to understand is that even though the TIF standard says alpha is opacity, which Manifold (of course) honors, there are very many software packages which ignore that and instead write alpha as transparency.

where is locate the metadata projection for geo inside tiff file ?

Besides what the TIF, TIFF, GeoTIFFtopic says about tags being used in GeoTIFF, I don't know specifically, but if you Google "geotiff standard" you'll get plenty of links to the definition, which will give you specifics.

tjhb
10,094 post(s)
#17-May-20 07:31

It would be worth searching on gdalinfo as well. That is part of the GDAL toolset, and will give you every bit of detail stored in any TIF(F) file, among many other formats.

I learned this here from Michael Sumner.

adamw


10,447 post(s)
#15-Apr-20 15:00

is there a way to see the real content after the ... in the end of pop up of cell inside manifold ?

Yes, right-click the cell and select Edit, this will open a dialog which will show the full value. Or right-click the cell, select Copy, then paste the result into a text editor.

lionel

995 post(s)
#22-May-20 22:58

if a camera photo is export to tiff ...what is the pojection setting we must choose ?

Attachments:
autumn.png


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

adamw


10,447 post(s)
#23-May-20 08:55

For photos like this it mostly makes sense only to save the location of where the photo was taken. This is normally done in lat/lon (WGS84).

lionel

995 post(s)
#12-Jan-21 03:07

put here because relative to tif and discover this term in the forum

Raster pyramids—ArcMap | Documentation (arcgis.com)

from Manifold System 9.0.172.3 (georeference.org)

TIFF supports pyramids stored within the file. (This is the feature that we were working on. It allows rendering a big TIFF without spending any time on preparing data for intermediate levels, if the file already has that data prepared and stored internally.) There is a limitation in that the pyramids are only supported if both the main image and all sub-images within the file are tiled, but that's how these files are usually organized anyway.

New function: TileUpdateFieldPyramids - updates pyramids for a tile field in the specified component. If the tile field is not part of a spatial index, the call does nothing.

New function: TileUpdatePyramidsAll - updates pyramids for all tile fields in the specified component.

from tjhb Color Balance difference in images imported into M8 and M9 (georeference.org)

The image will initially become blank, because pyramids need to be rebuilt. You can either do this via the Messages route (F7), or include it in the SQL:

TABLE CALL TileUpdatePyramids([Image])

;

from Manifold System 9.0.168.3 (georeference.org)

ERDAS IMG dataport automatically creates RRD pyramids when exporting raster data.

From Manifold System 9.0.168.1 (georeference.org)

ERDAS IMG format as it utilizes RRD files is slightly different than what ESRI does with RRD for file formats other ERDAS IMG. When ESRI creates RRD files for formats other than ERDAS IMG it puts the pointer to the pyramids in an .aux file, not in the .img file as does ERDAS, so the two are close but not identical.

TODO Key concepts of raster dataset color maps—ArcMap | Documentation (arcgis.com)


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.