Support for LAS version 1.4

[jtfell]

From my research there is no pure javascript LAS reader library that can read files in version 1.4 that is open-source. Is developing this functionality on your roadmap?

I can see that your existing parser for LAS files is a fork of plasio which is based on an enscripten build of laz-perf. If you are going to add this, how difficult do you expect the upgrade to be considering that toolchain? If you expect it to not be a major undertaking I could be interested in contributing.

Really, I'm just looking to get the conversation started around this as it seems to be a key piece missing from this ecosystem

[ibgreen]

@jtfell Welcome to loaders.gl!

Is developing ... a LAS reader library that can read files in version 1.4 that is open-source ... on your roadmap?

• The LASLoader is a cornerstone of loaders.gl and we'd love to have support for the latest version of the LAS format.
• (It would also be nice to have a presumably smaller, non-emscripten based parser for LAS).

how difficult do you expect the upgrade to be considering that toolchain? If you expect it to not be a major undertaking I could be interested in contributing.

• Taking a quick look at the code in https://github.com/hobu/laz-perf/tree/master/laz-perf it appears to be a somewhat complicated format so it probably would take a couple of days to port to JavaScript, with proper test cases etc.
• If you are seriously interested in doing this work in loaders.gl, I would be happy to support and guide you.

The alternative is of course to upgrade the laz-perf C++ implementation and we'd get the JS implementation for free by rebuilding the emscripten library.

@hobu Given that you are behind the LAZ-perf implementation that is currently used by loaders.gl:

• Do you recall which LAS version laz-perf supports? I couldn't easily see it in the repo.
• Do you have a sense for how big the delta to support 1.4 is?
• Do you have any thoughts about implementing a 1.4 compatible parser in pure JavaScript. Or would contributing to the laz-perf C++ code base be a better choice?

@jtfell Out of curiosity, do you just happen to have 1.4 files that you wish to load, or is there a specific feature from LAS 1.4 that interests you? Googling, features added with version 1.4 include:

• Backward compatibility with LAS 1.1 – LAS 1.3 when payloads consist of only legacy content
• Extension of offsets and field sizes to support full 64-bit values.
• Extension of several features to allow higher quality data and support more powerful equipment.
• Support for Well Known Text (WKT) definitions for Coordinate Reference Systems (to replace use of GeoTIFF conventions).
• Support for Domain Profiles. These profiles may extend the LAS format in controlled ways.

[jtfell]

Thanks for the in-depth response @ibgreen. We are most interested in the capability to store > 32 classification categories. We also expect to need to deal with existing 1.4 from our customers. There are of course workarounds to use PDAL to downgrade before reading using loaders.gl, but this adds complexity.

Is your preference for a pure-js implementation primarily because of the filesize? It would seem a lot more effort to port and maintain a separate implementation rather than making improvements to the existing codebase.

I'd be very interested to hear your thoughts on this @hobu

[ibgreen]

Is your preference for a pure-js implementation primarily because of the filesize?

Well, that certainly used to be the case (I believe the loaders.gl 1.3 LAS module adds 700KB to the JS bundle size). but is no longer a strong reason, since in loaders.gl 2.0 we support dynamic loading of loader libraries from CDN.

We haven't configured LASLoader to use dynamic loading yet, but it should not be hard and should effectively remove that concern.

I would agree that updating the existing C++ code base would seem like the right approach, especially as the new features in 1.4 seem fairly limited. A new emscripten build of laz-perf would be the deliverable that loaders.gl would need. Also perhaps we could add a few test cases to loaders.gl, e.g. a few LAS files with different formats.

[ibgreen]

• We are most interested in the capability to store > 32 classification categories.
• We also expect to need to deal with existing 1.4 from our customers.
• There are of course workarounds to use PDAL to downgrade before reading using loaders.gl, but this adds complexity.

@jtfell Based on your feedback, there seems to be 3 levels of support for LAS v1.4 we could provide in loaders.gl

1. las-perf/loaders.gl could load 1.4 files, but any 1.4 features would be ignored. At least your customers would not need to process files externally in 1.4 before loading them into the browser.
2. las-perf/loaders.gl could add partial 1.4 support, e.g. just for the classification limit raise you are interested in.
3. las-perf/loaders.gl could add full 1.4 support.

Presumably in increasing order of effort.

Have you tried loading 1.4 files in loaders.gl and confirmed it doesn't work?

[hobu]

There is a notional roadmap item to add LAS/LAZ 1.4 to the laz-perf library, but because the LAZ encoder changed so significantly from 1.0-1.3 -> 1.4, it is a big lift of effort. It will require porting the new LAZ 1.4 encoder/decorder to laz-perf and implementing the new (wider) types of LAS 1.4 in laz-perf. It is likely to be a completely new API added to the library because of the widening of the types in LAS 1.4.

We haven't yet done this because in our experience, we have found that we can work around the need by using LAS 1.3 with the LAS 1.4 extra bytes VLR. This is not a long term solution as more LAS 1.4 content is being developed, especially content with wider types or eVLRs.

Another option is to compile the LASzip library itself using emscripten. laz-perf was a reaction to the overly virtual'd design of a previous LASzip version, and with that stuff removed, it is probable that LASzip would compile to WASM with reasonable performance. The library API is quite messy though, and hooking it up to JS will require some shims, shackling, and spackle.

@connormanning

[sguergachi]

Hi there, wanted to chim in that we have a project that would really benefit from LAS 1.4 support. Trying to load a 1.4 .las file gives this error:

Any idea support will be added anytime soon?

[ibgreen]

@sguergachi As you can read above, loaders.gl depends on a web assembly library and there has not been any activity there to my knowledge.

I have taken a quick look in the past at implementing LAZ directly in JavaScript but it seemed quite complex and the need simply hasn't been strong enough.

Unfortunately, I am starting to suspect that making progress here may require spending some money, i.e. I could put you in touch with resources that do this kind of work.

[sguergachi]

Thanks @ibgreen understood this isn't likely to happen.

It might be worth indicating on documentation that LAS 1.3 and below is supported, as I only discovered this after implementing a proof of concept in code.

[ibgreen]

It might be worth indicating on documentation that LAS 1.3 and below is supported,

@sguergachi Good suggestion. Docs updated at https://loaders.gl/docs/modules/las/api-reference/las-loader (and apologies for the wasted time).

Just for reference, if someone ever needs support for LAS 1.4 badly enough to consider paying for it, a good option would be working with ActionEngine, they have a team that has been contributing to loaders.gl for a long time.

[ibgreen]

Reopening ticket as we now have some traction: #3165

[ibgreen]

Converted this ticket to a discussion to have a place to continue to discuss the ongoing work.

At this stage the open PR is #3168.

@alekzvik @connormanning