Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add GeoJSON(bool area) lua function #4

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Commits on Oct 3, 2024

  1. add GeoJSON(bool area) lua function

    Returns the geometry serialized as a GeoJSON geometry.
    cldellow committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    9c25495 View commit details
    Browse the repository at this point in the history
  2. Push Docker image even on branches

    Previously, we pushed a docker image only when building `master`.
    
    Instead, push it for any branch.
    cldellow committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    a196c9d View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. docker: luarocks, luaflock, lua-sql-sqlite3

    Some useful packages that a user might want available to their
    Lua scripts:
    
    - `luaflock`: BSD-style flock
    - `lua-sql-sqlite3`: SQLite bindings for Lua
    
    Also print the lua error code and error message string in the error
    handler. It seems like not all Lua errors have stack traces/tracebacks,
    e.g. requiring a non-existent module:
    
    before:
    
    ```
    lua runtime error:
    terminate called after throwing an instance of 'OsmLuaProcessing::luaProcessingException'
      what():  std::exception
    ```
    
    after:
    
    ```
    lua runtime error 2:
    ./file_append.lua:7: module 'flock' not found:
    	no field package.preload['flock']
    	no file './flock.lua'
    	no file '/usr/local/share/lua/5.1/flock.lua'
    	no file '/usr/local/share/lua/5.1/flock/init.lua'
    	no file '/usr/local/lib/lua/5.1/flock.lua'
    	no file '/usr/local/lib/lua/5.1/flock/init.lua'
    	no file '/usr/share/lua/5.1/flock.lua'
    	no file '/usr/share/lua/5.1/flock/init.lua'
    	no file './flock.so'
    	no file '/usr/local/lib/lua/5.1/flock.so'
    	no file '/usr/lib/x86_64-linux-gnu/lua/5.1/flock.so'
    	no file '/usr/lib/lua/5.1/flock.so'
    	no file '/usr/local/lib/lua/5.1/loadall.so'
    terminate called after throwing an instance of 'OsmLuaProcessing::luaProcessingException'
      what():  std::exception
    ```
    cldellow committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    8b6c67a View commit details
    Browse the repository at this point in the history
  2. be less chatty when run non-interactively

    This introduces a minimum threshold for writing the progress updates
    like:
    
    - Scanning for nodes in ways
    - Block M/N
    - Writing tile M of N
    
    The threshold applies when run non-interactively, for example, when in a
    GitHub action. (Related to this, I've learned that GitHub Actions get
    angry at you when your action generates 500 MB of logs. :)
    
    Interactive use should be unaffected.
    cldellow committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    1a86dcd View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. use libdeflate rather than zlib

    The zlib implementation that ships with most distributions is fairly slow,
    even when you allow for the zlib algorithm itself being quite slow (vs
    lz4 and zstd).
    
    There are faster implementations. `libdeflate` [1] is one such example.
    According to benchmarks [2], it's maybe 2-3x faster than zlib.
    
    This PR updates helper.cpp's compression routines to use libdeflate.
    
    It saves ~2-3% of total execution time for me:
    
    Creating a GB mbtiles (zlib):
    
    ```
    real 2m1.706s
    user 28m24.186s
    sys 0m41.886s
    ```
    
    Creating a GB mbtiles (libdeflate):
    
    ```
    real 1m58.450s
    user 27m32.579s
    sys 0m51.848s
    ```
    
    Snapshotting an external dependency is sorta distasteful - I worry
    about how much is too much from a maintenance POV. To mitigate that,
    the snapshot is a direct copy of the upstream folders, so it should
    be easy to update in the future if needed.
    
    This also lets us drop the zlib1g-devel and boost-iostreams
    dependencies, which is nice.
    
    [1]: https://github.com/ebiggers/libdeflate
    [2]: zlib-ng/zlib-ng#1486
    cldellow committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    250f942 View commit details
    Browse the repository at this point in the history
  2. D'oh, ban copy/move constructors

    Oops: if you ever passed a non-default compression level, we'd
    try to create a new compressor. But because we were using the
    default constructors, we'd happily create a temporary, copy its
    pointer over to the `compressor` field, then destruct the temporary,
    freeing the underlying native resource, and the world would end.
    
    Instead, delete the default constructors and require an explicit
    `setLevel` call.
    
    Also update tests to verify that the different levels work.
    cldellow committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    880d553 View commit details
    Browse the repository at this point in the history