You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As discovered in #187, pickling the road map rtree is currently failing which disables us from leveraging multiprocessing for a batch of traces. Until the pickling issue is resolved we could explore workarounds to allow traces to be processed in parallel. Road maps could be very large and so a solution that allows the road network data to be shared would be ideal.
The text was updated successfully, but these errors were encountered:
@jhoshiko - I seem to recall that you've run into a similar issue in the past. Do you have any notes from trying to map match large road networks in parallel that might be useful in this context?
@nreinicke In the past, I would initialize a copy of the road network on each worker, scaling the number of workers based on available memory and the size of the road network. I haven't personally experimented with matching to a road network in shared memory, but I agree that it would be ideal. I think multiprocessing/joblib might have options for instantiating objects into shared memory, but I haven't personally tested anything.
As discovered in #187, pickling the road map rtree is currently failing which disables us from leveraging multiprocessing for a batch of traces. Until the pickling issue is resolved we could explore workarounds to allow traces to be processed in parallel. Road maps could be very large and so a solution that allows the road network data to be shared would be ideal.
The text was updated successfully, but these errors were encountered: