I3S server, removed slpk-extractor and parseSLPKArchive

[bortomar-intellmaps]

Hi, I would like to ask why the "slpk-extractor" tool was removed from @loaders.gl/tile-converter (maybe @dariaterekhova-actionengine) and why i3s-server currently only allows reading data from the slpk archive and not from directory with extracted archive?

If I run i3s-server@4.3.1 with the I3SLayerPath parameter pointing to the slpk archive (frist screenshot) and then use address of this local i3s-server as tileset url in the example @loaders.gl/examples/website/i3s-buildin-scene-layer, I get 200 in the server console for the first two requests, but all other requests end up with 404 (see second screenshot).

Content of the extracted root metadata.json (click to unfold)

{
   "folderPattern":"EXTENDED",
   "archiveCompressionType":"STORE",
   "resourceCompressionType":"GZIP",
   "I3SVersion":"1.8",
   "nodeCount":197
}

If I run i3s-server@4.2.5 with the I3SLayerPath parameter pointing to the directory containing structure SceneServer/layers/0 where the previously extracted slpk is located, everything works as it should.

Content of the extracted root 3dSceneLayer.json.gz (click to unfold)

{
   "id":0,
   "name":"PavilonP_BL",
   "layerType":"Building",
   "alias":"PavilonP_BL",
   "spatialReference":{
      "wkid":4326,
      "latestWkid":4326,
      "vcsWkid":5773,
      "latestVcsWkid":5773
   },
   "fullExtent":{
      "xmin":16.5744992240586235,
      "ymin":49.18788311011717,
      "xmax":16.5780159930023885,
      "ymax":49.1896941398643648,
      "spatialReference":{
         "wkid":4326,
         "latestWkid":4326,
         "vcsWkid":5773,
         "latestVcsWkid":5773
      },
      "zmin":206.44999999999709,
      "zmax":225.110000000000582
   },
   "activeFilterID":"",
   "statisticsHRef":"./statistics/summary",
   "sublayers":[
      {
         "id":200,
         "layerType":"group",
         "name":"Full Model",
         "alias":"Full Model",
         "modelName":"FullModel",
         "visibility":false,
         "sublayers":[
            {
               "id":210,
               "layerType":"group",
               "name":"Architectural",
               "alias":"Architectural",
               "modelName":"Architectural",
               "visibility":true,
               "sublayers":[
                  {
                     "id":8,
                     "layerType":"3DObject",
                     "name":"IfcWall_surface",
                     "alias":"IfcWall_surface",
                     "modelName":"IfcWall_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":7,
                     "layerType":"3DObject",
                     "name":"IfcSlab_surface",
                     "alias":"IfcSlab_surface",
                     "modelName":"IfcSlab_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":6,
                     "layerType":"3DObject",
                     "name":"IfcRailing_surface",
                     "alias":"IfcRailing_surface",
                     "modelName":"IfcRailing_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":5,
                     "layerType":"3DObject",
                     "name":"IfcPlate_surface",
                     "alias":"IfcPlate_surface",
                     "modelName":"IfcPlate_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":4,
                     "layerType":"3DObject",
                     "name":"IfcMember_surface",
                     "alias":"IfcMember_surface",
                     "modelName":"IfcMember_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":3,
                     "layerType":"3DObject",
                     "name":"IfcDoor_surface",
                     "alias":"IfcDoor_surface",
                     "modelName":"IfcDoor_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":2,
                     "layerType":"3DObject",
                     "name":"IfcColumn_surface",
                     "alias":"IfcColumn_surface",
                     "modelName":"IfcColumn_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  },
                  {
                     "id":1,
                     "layerType":"3DObject",
                     "name":"IfcBuildingElementProxy_surface",
                     "alias":"IfcBuildingElementProxy_surface",
                     "modelName":"IfcBuildingElementProxy_surface",
                     "discipline":"Architectural",
                     "visibility":true,
                     "isEmpty":false
                  }
               ],
               "isEmpty":false
            }
         ]
      },
      {
         "id":0,
         "layerType":"3DObject",
         "name":"Overview",
         "alias":"Overview",
         "modelName":"Overview",
         "visibility":true,
         "isEmpty":false
      }
   ],
   "version":"1.8"
}

If I use the same archive in the @loaders.gl/examples/experimental/slpk-in-browser I get an error in the browser console:

Uncaught (in promise) Error: No such file in the archive: nodes/root
    at SLPKArchive.getFile (slpk-archieve.ts:101:11)
    at async ZipFileSystem.fetch (zip-filesystem.ts:107:26)
    at async load (load.ts:90:12)
    at async normalizeTilesetData (parse-i3s.ts:97:12)
    at async parseTileset (i3s-loader.ts:108:32)
    at async Object.parseI3S [as parse] (i3s-loader.ts:87:12)
    at async parseWithLoader (chunk-5WBTQHG4.js?v=39f4736d:4174:12)
    at async parse (chunk-5WBTQHG4.js?v=39f4736d:4151:10)
    at async load (chunk-5WBTQHG4.js?v=39f4736d:4380:97)
    at async CustomTile3DLayer._loadTileset (@deck__gl_geo-layers.js?v=f80ba189:40416:25)

I would like to know if the above slpk archive is non-standard, or is it a problem on the @loaders.gl side?

EDIT: The same behavior can be observed in the case demo data from: https://github.com/Esri/i3s-spec/blob/master/docs/1.9/BSL_ReadMe.md

[dariaterekhova-actionengine]

Hi,

It was decided that Loaders.gl no longer supports i3s datasets in the form of local folder structure, so all the functionality connected to it was removed. The only function of the slpk-extractor was to convert an slpk to a folder structure so it was removed too. So currently SLPK is the only format for i3s datasets that you can use in Loaders.gl.

Regarding the building scene layer issue - it was an issue with sublayers reading and it is fixed in v4.3.2 of Loaders.gl

[bortomar-intellmaps]

@dariaterekhova-actionengine Hi, thank you very much for the fix. However, while testing, I noticed that i3s-server still returns a 404 error for the path "/statistics/summary" (see screenshot).
So I looked at the changes you made in the last patch and based on that, I added a fix for the case described above.
Can you confirm that my fix is acceptable and ready for merge?
#3154

[ibgreen]

@dariaterekhova-actionengine Are there any updates to whats-new or upgrade-guide docs that we could add to help users find this information on their own without asking? Or is it already covered?

[dariaterekhova-actionengine]

@ibgreen We have this info in the tile-converter section of v4.3 there https://loaders.gl/docs/whats-new