Questions on new nigiri core deployment

[mariodavid]

Hi @felixguendling,

I read that with the 0.9+ versions there is a new core engine that requires way less memory for graph building and the running application. That is really great news.

Since in the documentation there is not very much information about what modules are already supporting nigiri. There is this blog post about 0.11 which says:

we’re happy to announce that we are almost finished with transitioning the intermodal door-to-door real-time routing over to the new nigiri core

where I'm not 100% sure what "almost finished" means :) I have seen this image: https://github.com/motis-project/motis/wiki/Modules#modules-compatible-to-the-new-core - but it is not 100% clear to me which of the ones are considered done & if the docs are up to date.

We are trying to upgrade from 0.8 to 0.11 in order to reduce memory footprint and potentially also speed up graph building process (not sure if I understand that correctly).

With the following config.ini I was hoping to have the correct settings in place to use it:

modules=nigiri
modules=intermodal
modules=osrm
modules=routing
modules=tripbased
modules=intermodal
modules=ppr
modules=lookup
modules=cc

intermodal.router=nigiri

# Configure datasets
import.paths=schedule-de:/input/schedule_germany
import.paths=osm:/input/osm.pbf

# OSRM profiles (add more if needed)
osrm.profiles=/motis/osrm-profiles/car.lua
osrm.profiles=/motis/osrm-profiles/bus.lua

# Configure schedule range
nigiri.first_day=TODAY
nigiri.num_days=7

With the input data collected as follows:

#!/bin/bash

wget -q https://download.geofabrik.de/europe/germany/bayern-latest.osm.pbf -O ./osm_bayern.pbf
wget -q https://download.geofabrik.de/europe/germany/berlin-latest.osm.pbf -O ./osm_berlin.pbf
wget -q https://download.geofabrik.de/europe/germany/brandenburg-latest.osm.pbf -O ./osm_brandenburg.pbf

osmium merge osm_bayern.pbf osm_berlin.pbf osm_brandenburg.pbf -O -o ./input/osm.pbf

wget -q https://de.data.public-transport.earth/gtfs-germany.zip -O ./schedule_germany.zip
unzip ./schedule_germany.zip -d ./input/schedule_germany

When I run it like this, I get the following error during graph building:

          cc: FINISHED
   de_nigiri: FINISHED
      lookup: FINISHED
      nigiri: ERROR: 10 already in shared_data
    osrm-bus: FINISHED
    osrm-car: FINISHED
    parse de: FINISHED
         ppr: FINISHED
     routing: FINISHED
    schedule: FINISHED
   tripbased: FINISHED
2023-11-19T18:18:56Z [VERIFY FAIL] some imports were not successful: nigiri

initialization error: some imports were not successful: nigiri

When I on the other hand try a stipped down version of the config, at least the server starts:

modules=nigiri
modules=intermodal
modules=osrm

intermodal.router=nigiri

# Configure datasets
import.paths=schedule-de:/input/de
import.paths=osm:/input/osm.pbf

# OSRM profiles (add more if needed)
osrm.profiles=/motis/osrm-profiles/car.lua
osrm.profiles=/motis/osrm-profiles/bus.lua

# Configure schedule range
nigiri.first_day=TODAY
nigiri.num_days=7

dataset.no_schedule=1

Which leads me to conclude that 1. it has something to do with the dataset.no_schedule or with the amount of modules. The no schedule setting I found in the 0.9 blog post, but other than that not really found anything. Since it says it is required to prevent to go down the old approach, I assumed it is required to achieve the small memory footprint.

Perhaps you can answer a couple of those questions. I know there are multiple, but it would be of great help :)

Thanks!

[mariodavid]

Additionally here is the log output for the nigiri module: nigiri.txt
nigiri.txt

[felixguendling]

This week I updated the "Installation & Setup" instructions to the new core:
https://github.com/motis-project/motis/wiki/Installation-and-Setup

The nigiri module now takes over the functionality from the following modules: guesser, lookup, railviz, routing, tripbased, csa, raptor (so those are now deprecated and won't see any further development except for maybe small bug fixes). The path module functionality (nice lines on the map instead of direct lines) is currently not available with nigiri - the idea is to use shapes.txt in the future (but there are no concrete projects planned yet to do this). That means, you probably also don't need the OSRM bus profile (which was only used by the path module).

Regarding nigiri.num_days=7: with nigiri it should be easy to cover a way larger time period with not much memory overhead. Every bitfield in nigiri is 512 bits anyway. So setting it to one year is good.

Depending on your use case, it might make sense to turn on tiles and address, parking, and ppr.

To speedup preprocessing times for the parking module, I use parking.ppr_exact=false which reuses footpath ways for one direction in the opposite direction (A-B = B-A), which assumes symmetry. This is almost always the case except for special cases like escalators.

To speedup restart times, I sometimes use ppr.check_integrity=false which skips the integrity check for the foot routing graph.

I hope this helps. Feel free to ask more questions or tell me which questions I missed :)

[felixguendling]

Corrrect, the cc (as well as all pax* and revise, ris, rt) module is not compatible yet to nigiri.

[ch33hau]

Hi @felixguendling ,

I tried to start the server with nigiri, however, as long as when I have module=nigiri, I got the following error:

nigiri: ERROR: boost::filesystem::creation_time: Function not implemented [system:38]: "/data/import/gtfs"

I tried with the very minimal settings like what mariodavid posted here.

Without module=nigiri, the server can be started, but the UI is missing.
When I include server.static_path, then railviz requests were failed due to the module was missing.

Do you have any ideas about having nigiri with railviz and other modules work together?

Thanks

[felixguendling]

Do you have any ideas about having nigiri with railviz and other modules work together?

nigiri replaces railviz and delivers the same APIs.

For the other problem, I would need to reproduce it first.

[gbastkowski]

The boost::filesystem::creation_time appeared on MacOS with host-mounted volumes. I solved it by mounting the data volume as a Docker volume.

[felixguendling]

Should work even with host-mounted volume with the latest release.