Subscribe to this thread
Home - General / All posts - Surface related capabilities in M9
tonyw
462 post(s)
#10-Feb-18 17:23

In http://www.georeference.org/forum/t141662.8 Dimitri wrote

I think 9 can do more than 8 (way better than the surface transform pane in 8, for example)

This is promising. I have elevation points from LiDAR data collection, in .laz files, from which I would like to create contours, say at 5m and 1 m intervals. I've looked in the manual but don't know what terms or phrases to look for in M9. Dimitri, can you point me to the subject areas of the manual please?

Dimitri

5,035 post(s)
#10-Feb-18 19:34

That's a very ambitious project, not for the faint of heart.

The SQL Functions topic lists the arsenal you have if you want to use SQL and the API doc is here if you prefer scripting. SQL in 9 is way more capable than the surface transform pane in 8, but that doesn't mean you have to use SQL for project like this. (Always check the query builder in the Command Window for the most up to date list of SQL functions).

lionel

459 post(s)
#10-Feb-18 22:21

hardware device that use depth sensing camera to create point cloud need specific sofware/library to process this data .

it is very technical .

http://recherche.ign.fr/labos/matis/pdf/articles_revues/gressin_ICP_IJPRS_2013.pdf

https://www.crcpress.com/Topographic-Laser-Ranging-and-Scanning-Principles-and-Processing/Shan-Shan-Toth-Toth/p/book/9781420051421

some software fugroviewer spdlib CloudCompare Viewer https://github.com/PulseWaves/Specification/blob/master/specification.rst

http://opentopo.sdsc.edu/usiei

Attachments:
point_clound_software_comparision.png


join image "Because my dad promised me" interstellar from Manifold: Time by Stephen Baxter. power Math destruction

Dimitri

5,035 post(s)
#11-Feb-18 05:49

hardware device that use [...]

Well,... maybe not that ambitious. :-)

The big difference between 9 and 8 right now as regards rasters is that 9 has huge, more powerful infrastructure than 8, but you have to build what you want out of that infrastructure, while 8 has many more packaged features done for you in rasters within what is a more constrained infrastructure.

Short term, right now with rasters you can do more with 8 easier. Long term, there is no comparison because when 8 tops out that's it, while 9 doesn't top out. Within months it will be a case where 9 will do more packaged stuff than 8 with rasters as well as having better infrastructure.

To take Surface Tools as an example, in 8 you have the surface transform dialog which allows construction of expressions to operate on surfaces using a defined set of functions. That's a very good idea, and there is an impressive set of functions available in that dialog, many of which accomplish in one function what would require a chain of simpler building blocks within 9. But those building blocks within 9 have a way more infinite applicability and won't slow down to a halt with bigger data, or at least nowhere near as quickly as the infrastructure in 8.

There is also the innate flexibility. Don't kid yourself: what you are doing when using that surface transform dialog is you are writing a program. It is a fair question, therefore, just what kind of programming environment does the surface transform dialog provide, and how does that environment compare to 9.

Without doubt that dialog is a much more constrained environment than 9. 8 is very good - I'm very proud of the work we did in 8 - and it is a better programming environment, as far as I can tell, than other classic GIS for such things. After all, 8 has spatial SQL and neat things like the surface transform dialog in it. But compared to 8, 9 really hits it out of the park as a more sophisticated, more powerful, more general, and overall a much better programming environment. The experience of 8 made that possible.

For all that while 8 may have a more constrained infrastructure, for people who want point and click capability (a great thing, in my book), 8 has years of accumulated, packaged dialogs and functions. They may top out way sooner than 9 in terms of capacity, but there's lots of them and they work great. 9 doesn't have those yet, so while, sure, you can get down to it and roll your own by recombining what 9 does provide, that's often not at all realistic for most nor is it desirable.

When I mentioned applying 9 infrastructure I had in mind what most uses for the surface transform dialog historically have been: relatively straightforward applications of comparisons and manipulations between rasters. Those are cases where not being constrained by an expression within the specific rules of a one-of-a-kind dialog, and instead being able to use a full and very rich SQL, that can also tap into an even more rich scripting environment, are a good example of how the better infrastructure in 9 can work for you.

Setting out to duplicate very big and complex, multi-step packaged dialogs like starting with vector point clouds, creating an intermediate interpolated raster surface, computing raster contours on those in the usual way, and then vectorizing them is a very different kettle of fish. That you could do such stuff given skill and effort in 9 is impressive. It shouldn't be a surprise, because, after all when Manifold's engineering staff adds such capabilities to 9 that's what they will be using: 9 itself. But whether you should do such things as opposed to just letting those features appear in point and click fashion is a different question.

My advice is to start simpler if this interests you, since there is little point other than intellectual interest in starting on a feature that Manifold likely will introduce before you get it done. The focus in 9 clearly is to bring all vector operations to a reasonably complete level at the current set of aspirations. That means things like adding a few more snap modes, another pass through the vector editing infrastructure and restoring the ability to manipulate curved segments that was temporarily lost in the shift from modal dialogs to non-modal panes.

In the months ahead Manifold will turn to very fully articulated implementations of everything raster under the sun. We can make it a lot easier for people by implementing automatically a pixel-based programming and UI approach, which now must be manually derived in an extra step from the tile-based storage model, and by adding a variety of common raster tasks, say as raster templates, for things like watersheds, viewsheds, contours, and the like. Like I said, all that is implemented using 9 itself so in theory anybody could do it right now, but in practice it is easier to let Manifold do it for you. Besides a probably greater familiarity with the product the Engineering staff has the advantage that if they ever see the opportunity to add a helper function that eliminates the need for some very roundabout approach, well, they can add that function.

The good news about rasters, by the way, that raster features tend to be very easy and often (not always, but often) are completely obvious how to parallelize. So it's possible to implement very many of them very quickly. :-)

tjhb

8,273 post(s)
#11-Feb-18 06:49

Wonderful post Dimitri. This frankness is really useful. Thanks!

tonyw
462 post(s)
#11-Feb-18 19:55

Dimitri wrote:

That's a very ambitious project, not for the faint of heart.
and instead being able to use a full and very rich SQL, that can also tap into an even more rich scripting environment, are a good example of how the better infrastructure in 9 can work for you.

OK, I'll wait for the packaged tools. Anything technical involving writing extensive SQL or any scripting is beyond my abilities. The packages will be out ages before I'm anywhere done. Someone else in another company is working on mapping for my current project and is able to use the LiDAR data we purchased. I thought it would be interesting to also explore the data on my own but that will have to wait.

Packages or templates maybe limiting by their design but for users like me, that's about the only way I will be able to use GIS and Manifold to its full capabilities. For the technically inclined or purists who like to roll their own it is great Manifold 9 allows that. In my tool kit, Manifold sits there with MS-Word, Excel, a web browser, and an email client. I need to write a memo, the memo needs a map and a table so out comes Manifold and Excel. Memo done, attached to an email and sent. Onto next task. I have an intellectual interest in trying things but the opportunities are limited to available time and that I'm yet unsuccessful in finding any clients willing to pay me a salary to learn. It's great Manifold is such a technically advanced tool but there are also people like me who are less technical users and depend on packages, templates, and menu driven processes.

dchall8
501 post(s)
#12-Feb-18 15:08

I'm with you on waiting for the surface tools to develop in M9. I work in the appraisal business. While I am interested in LiDAR surfaces, I am also interested in finding buildings hidden in the trees. Our LiDAR files have different classifications of data. One is the ground surface, then they have low, medium, and high vegetation, building roofs, and culverts. So I was interested only in one class of points. Each LiDAR files has millions of points but only hundreds of thousands of roof points. Complicate that import by having over 2,000 .las files and I was not seeing the light at the end of that tunnel. I wrote in here and got help from several people. Read through that and you can almost see the gears turning. There are two steps. First is to create a DOS script to generate a list of .las files. That takes no time at all. Second is the script to link to each file and extract the points I needed. There were some iterations, but the final script processed each .las file in a little less than a minute, but at least it was all automatic.

tonyw
462 post(s)
#13-Feb-18 07:29

dchall8,

Interesting, I'll check more if my elevation points are classified. I wonder how an algorithm can distinguish an elevation as being a roof top or the ground? The elevation point data I received has a file name of "Ground/Non-ground" suggesting there is some differentiation however on a quick look I don't see anything in the table that distinguishes ground from non-ground points (maybe tree tops). I don't have the documentation yet that will tell me the assumptions or contents of the fields in the tables.

dchall8
501 post(s)
#13-Feb-18 22:33

The LiDAR should be in XYZ or YXZ format in the .las file.

When I view my data in a real LiDAR viewer, you can tilt the world and see the roof top angles floating there. Tree and roofs look like trees and roofs as you spin around them. That would be very slick if Manifold could show the LiDAR data in 3-d. It probably has the horsepower to do that.

Dimitri

5,035 post(s)
#14-Feb-18 08:15

That would be very slick if Manifold could show the LiDAR data in 3-d. It probably has the horsepower to do that.

Yes, it does have the horsepower to do that. Radian from the beginning was designed to support 3D visualizations of larger data. That's always been in the plan.

We have several (many) months of work ahead of introducing 3D visualizations, both for LiDAR and for other things. We'll get there.

tonyw
462 post(s)
#14-Feb-18 08:23

When I view my data in a real LiDAR viewer

Thanks dchall8 for the tip, I searched and found several LiDAR viewers, some open source, others not open source but nonetheless free. Any recommendation on one to try?

dchall8
501 post(s)
#14-Feb-18 14:13

No suggestion. Some of them won't do what you want for free and some will. The last time I used one was 2 years ago. As I recall it took so long for the data to load I did not get much value out of it. Once it loaded, though, spinning the image around was very speedy with no lag.

antoniocarlos

512 post(s)
#14-Feb-18 14:53

LIDAR is not my forte but if you want to work with smallish datasets not huge datasets. Try Whitebox GAT for free. I also understand that Global Mapper is pretty good bang for the buck as far as their LIDAR extension is concerned but have not used it.


How soon?

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.