Subscribe to this thread
Home - General / All posts - Kriging - 26 hours in
dchall8
1,008 post(s)
#19-May-20 01:52

I downloaded one of the USGS LiDAR files for my area of Texas. It has about 900,000 records in the file. Viewing it was weird from the start because it seems to sit in the Sahara Desert. I imported the file, changed coordinates to Lat/Long, and it's thousands of miles away from home. The lat/long in the lower right corner of the image below shows the location. But I don't really care about that, yet. I wanted to change the LiDAR points to an image using Kriging.

I tried setting the Resolution to .00001 and .000001. Both times I got an artifact seen in the image. At least it was clear enough that I can recognize the junior school and the bend of the river in our area. I'm confident I got the right tiles from USGS. So I tried setting the Resolution to .0000001 and, "off we go" as they say in the videos on the topic. All my CPU cores maxed out and the RAM maxed out for over 26 hours. I'm using Win 7, so I can't see what the GPU was doing. It felt cool, but all the video on the computer for this time had been slow to refresh and came with a lot of quirky pixellation.

When it finally finished Kriging Manifold came up with an error, Cannot write data. When I tried to save the project, it gave an Invalid object reference error in addition to Cannot write data. I'm not concerned about spending the time and getting the error. I'm just trying to learn the best way to use the software. Is there a better way to approach this problem of the artifact in the Kriged image? I measured the points and found them to be roughly 0.5 to 1 meters apart.

Attachments:
USGS LiDAR Image Kriging.jpg

Dimitri


7,413 post(s)
#19-May-20 04:23

Best to start with the original data, to clear up the projection issue, and to thus understand the actual extent to see what size image has been commanded. For example, if the kriging setup asks the system to create a 2 terabyte image and there is only 800 GB free on your disk, that's a problem. Just guessing, to illustrate why it is useful to start with the original data.

What is the download link to the original USGS LiDAR file? Any info on that file, like what projection it is supposed to be in? Also, info on the machine, like free space on disk, free space on the volume where TEMP is located, etc., helps.

tjhb
10,094 post(s)
#19-May-20 06:18

Yes, this is nuts.

I tried setting the Resolution to .00001 and .000001.

Why?

So I tried setting the Resolution to .0000001 and, "off we go"

Why?

These suggested resolutions (whether measured in metres or in feet) are not just strange but almost certainly orders-of-magnitude wrong. How ever to use the results?

You might have a total AOI measuring, say, 1 metre by 1 metre. Soil virus, bacteria, that could work.

But instead there ~must be a simple misunderstanding.

dchall8
1,008 post(s)
#21-May-20 01:23

I'm going to give a very incomplete reply, because I keep getting kicked off in the middle of typing.

I got the EPSG projection from the metadata and used that to put the point cloud over the location I was expecting it to be.

I disassembled the point cloud using the classifications as described in the metadata. Then I ran the Kriging on only the ground layer of points, accepting the defaults, and got what I wanted. Here's the revised image.

Attachments:
USGS LiDAR Image Kriging 2.jpg

Dimitri


7,413 post(s)
#21-May-20 05:53

That's a great image!

adamw


10,447 post(s)
#19-May-20 16:12

I would guess that the system ran out of space on the temp drive. When you increase the linear resolution 10x, the size of the raster grows 100x, that could easily produce the overrun.

Perhaps we should make transforms that produce rasters estimate the size of the output data, show it somewhere in the pane and warn if it gets too large. I will file a couple of requests into the wishlist.

antoniocarlos

609 post(s)
#19-May-20 16:16

Is krigging the best interpolation method for this purpose?


How soon?

adamw


10,447 post(s)
#21-May-20 09:23

It's hard to say, need to experiment with the specific data. My default with a big and dense set of points would be triangulation and then I would pick a subset of the set and try various forms of kriging, cutting points off by radius, choosing the cutoff radius so that there are maybe 15-20-30 points within it on average. If kriging would produce a smooth result with not many artifacts, I'd try using the same parameters on the whole raster.

PS: The color image of the result posted above looks pretty smooth indeed, at least on the posted scale.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.