Subscribe to this thread
Home - General / All posts - Bringing LiDAR clouds into Future as a projected drawing
dchall8
313 post(s)
#12-Oct-17 00:09

I'm trying to simplify bringing 2,000 LiDAR files into Manifold Future. Each file has roughly 20 million points of various classifications. Ultimately all I need out of these files are the points where the value of the classification field = 6. This is a zero to 1% fraction of each file. Ultimately these points will all be in one drawing for the county. Is it possible to write a single query to link to all these files, one at a time or in batches, extract records where classification = 6, bring them to Manifold Future as a table, create a drawing in UTM 14N, and color the points red? If this cannot be done with a link, can I bring the .las file in and make the computer do the rest of the processing? I have the SELECT INTO FROM WHERE syntax to get just the class 6 points, but can that be extended to make (or append into) a projected drawing with red points?

danb


1,590 post(s)
#12-Oct-17 23:32

With some help from Adam and Tim, I have been experimenting with the following process (which admittedly will be a little cumbersome for 2000 files), though should work well if you could script it.

Firstly following Adam's advice (http://www.georeference.org/forum/t138408.11#138461) I link to all the las/las files I want to use.

I then write a query (example below) to return the filtered, scale corrected data.

Finally I use Edit > Export Results to write the filtered points back to a las or laz file*

-- $manifold$

SELECT

VectorValue(GeomCoordXYZ([Geom], 0), 0) AS [X],

VectorValue(GeomCoordXYZ([Geom], 0), 1) AS [Y],

VectorValue(GeomCoordXYZ([Geom], 0), 2) AS [Z],

[Geom]

FROM

(

SELECT [Geom] FROM [C_BG36_1K_1337]::[C_BG36_1K_1337] WHERE [classification] = 2

UNION

SELECT [Geom] FROM [C_BG36_1K_1338]::[C_BG36_1K_1338] WHERE [classification] = 2

UNION

SELECT [Geom] FROM [C_BG36_1K_1339]::[C_BG36_1K_1339] WHERE [classification] = 2

UNION

SELECT [Geom] FROM [C_BG36_1K_1435]::[C_BG36_1K_1435] WHERE [classification] = 2

);

* Note: I have found that when I use export results, the resulting las/laz seems to ignore the scaling value and write the file with a value of 1. I have reported this to tech.

At the current time, you have to build a temporary index on point clouds which can be painful. As I understand however there is a specialist point cloud index in the pipeline.

The process above would also be much easier if you could treat/define a folder or directory tree of las/laz files as a single virtual dataset.

Attachments:
Clipboard-1.png


Landsystems Ltd ... Know your land | www.landsystems.co.nz

dchall8
313 post(s)
#13-Oct-17 22:03

BIG HELP! Thanks Dan. Your query worked great for me. I'm doing them in batches of 8 files. That way I'll only have to run it 250 times... for each classification.

As you are well aware, the first tedious part of this is making the individual links from Future to the LAS files in a folder on your computer. LAS file names are written in a code with spaces and subtle differences which make them hard to target individually I created an Excel file with the LAS file names and check them off as I go, so I don't duplicate them. I copy the file name from Excel, click File>Link in Future, and paste the file name into the search box at the upper right of the Link dialog box. Windows finds the file for me. Then doubleclick on the file to complete the link. That way I don't have to hunt for it in the list of look-alike names. It would be nice if the import could be done in a multiple select or batch mode. Or it would be nice if a script could iterate through the remote folder to make these links. The second tedious part is reforming the query in the [las_data_source]::[las_table] format. I have been dragging the table into the Query Builder lower right frame, double-click selecting the old text in the query, and double clicking the table name in the Query Builder lower right frame. That replaces the old text with the new text. The reason I limit these queries to 8 files is that it becomes hard to distinguish one line of code from another when all the file names are so subtly similar. So I tried dragging one of the linked tables into a folder on Future. What happened is Future imported the file instead of retaining it as a link. That defeats the purpose of linking. So I dragged the entire link into a folder. I'm not sure that helps. It would be nice if a script could use that folder in Future to iterate through the tables buried therein. If I were to ask for the world, it would be very nice if a script could

  1. go to a named LAS files folder source,

  • link to the first LAS file,

  • run the query to extract data where classification = 6 (in my case),

  • delete the link,

  • iterate to the next file in the LAS files folder,

  • link to the selected LAS file,
  • run the query to append new results to the old results,

  • delete the link,
  • repeat steps 5-7 until the folder of LAS files is exhausted
  • use the results to create a table
  • use the table to create a drawing in a selected projection.

    I can take it from there. If I knew how to do any of that I would be all over it.

    Does it seem odd that you have to export the results of the query and bring them back in to use them in your project?

  • otm_shank
    39 post(s)
    #14-Oct-17 06:46

    I've been trying to do similar, however I haven't been able to find the answer on how to import or link (or delete a link) via scripting:

    http://www.georeference.org/forum/t138528.17

    If that were possible, you could create a list of your LAS files using a DOS command into a text file, the script could open/link these one at a time and query your required data into a table within the MAP project.

    rk
    198 post(s)
    #16-Oct-17 10:47

    Dan, question below.

    I used very similar query for importing lines from linked dwg-s and experienced nonlinear slowdown when adding more and more UNIONs. I ended up using INSERT for every linked file, otherwise it got too slow. It happened even if I imported dwgs first into project - adding UNION made query more slower than adding extra INSERT.

    Didn't you have that problem?

    I see that my query uses extra values in every UNION, that is different.

    --SQL9

    insert into [result] ( ... )

    select

      ...

    from

      (

    -- 

      SELECT [ObjID][Geom]'B1' as [BLOCK]'1' as [LEVEL] from [B1]::[1 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B1' as [BLOCK]'2' as [LEVEL] from [B1]::[2 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B1' as [BLOCK]'3' as [LEVEL] from [B1]::[3 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B1' as [BLOCK]'4' as [LEVEL] from [B1]::[4 Table] WHERE GeomIsLine([Geom])

    -- 

    UNION ALL SELECT [ObjID][Geom]'B2' as [BLOCK]'1' as [LEVEL] from [B2]::[1 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B2' as [BLOCK]'2' as [LEVEL] from [B2]::[2 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B2' as [BLOCK]'3' as [LEVEL] from [B2]::[3 Table] WHERE GeomIsLine([Geom])

    UNION ALL SELECT [ObjID][Geom]'B2' as [BLOCK]'4' as [LEVEL] from [B2]::[4 Table] WHERE GeomIsLine([Geom])

    Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.