Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rendering specific access tags #4952

Merged
merged 19 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ jobs:
osm2pgsql -G --hstore --style openstreetmap-carto.style --tag-transform-script openstreetmap-carto.lua -d gis -r xml <(echo '<osm version="0.6"/>')
- name: Create indexes
run: psql -1Xq -v ON_ERROR_STOP=1 -d gis -f indexes.sql
- name: Load functions
run: psql -1Xq -v ON_ERROR_STOP=1 -d gis -f functions.sql
- name: Load empty shapefiles
run: scripts/get-external-data.py --no-update --cache -D scripts/empty_files
- name: Test queries are valid
Expand Down
7 changes: 7 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ The indexes can be created in parallel with
scripts/indexes.py -0 | xargs -0 -P0 -I{} psql -d gis -c "{}"
```

### Database functions
Some functions need to be loaded into the database for current versions. These can be added / re-loaded at any point using:

```sh
psql -d gis -f functions.sql
```

## Scripted download
Some features are rendered using preprocessed shapefiles.

Expand Down
89 changes: 89 additions & 0 deletions functions.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/* Additional database functions for openstreetmap-carto */

/* Access functions below adapted from https://github.com/imagico/osm-carto-alternative-colors/tree/591c861112b4e5d44badd108f4cd1409146bca0b/sql/roads.sql */

/* Simplified 'yes', 'destination', 'no', 'unknown', NULL scale for access restriction
'no' is returned if the rendering for highway category does not support 'restricted'.
NULL is functionally equivalent to 'yes', but indicates the absence of a restriction
rather than a positive access = yes. 'unknown' corresponds to an uninterpretable
access restriction e.g. access=unknown or motorcar=occasionally */
CREATE OR REPLACE FUNCTION carto_int_access(int_highway text, accesstag text)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combined with the other changes I think this can be removed but the other changes would need to be done first.

RETURNS text
LANGUAGE SQL
IMMUTABLE PARALLEL SAFE
AS $$
SELECT
CASE
WHEN accesstag IN ('yes', 'designated', 'permissive') THEN 'yes'
WHEN accesstag IN ('destination', 'delivery', 'customers') THEN
CASE WHEN int_highway IN ('road', 'pedestrian') THEN 'restricted' ELSE 'no' END
dch0ph marked this conversation as resolved.
Show resolved Hide resolved
WHEN accesstag IN ('no', 'permit', 'private', 'agricultural', 'forestry', 'agricultural;forestry') THEN 'no'
WHEN NULL THEN NULL
ELSE 'unknown'
END
END
$$;

/* Try to promote path to cycleway (if bicycle allowed), then bridleway (if horse)
This duplicates existing behaviour where designated access is required */
CREATE OR REPLACE FUNCTION carto_path_type(bicycle text, horse text)
RETURNS text
LANGUAGE SQL
IMMUTABLE PARALLEL SAFE
AS $$
SELECT
CASE
WHEN bicycle IN ('designated') THEN 'cycleway'
WHEN horse IN ('designated') THEN 'bridleway'
ELSE 'path'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be footway rather than path if you want to eliminate highway_path from MSS altogether.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conservative option is to keep the option of rendering highway=path differently from highway=footway (there already seems to be some difference in that footway can have an area render, but not path).

END
END
$$;

/* Coalesce highways that will be treated in the same way, e.g. all roads become 'road'
Note that bicycle, horse arguments are only relevant if considering highway=path */
CREATE OR REPLACE FUNCTION carto_highway_int_highway(highway text, bicycle text, horse text)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This SQL function is called in exactly one place and consists of a single CASE statement. This makes it equivalent to

SELECT
CASE CASE WHEN highway in (..) THEN 'road' WHEN ... END
WHEN 'road' THEN ... END
WHEN 'pedestrian' THEN ... END
END

Just modify the SQL statement calling it to remove the need for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not quite that simple since int_highway is passed through to carto_int_access. But I would propose splitting carto_int_access into 2 vs 3 state functions, removing the need for int_highway.

RETURNS text
LANGUAGE SQL
IMMUTABLE PARALLEL SAFE
AS $$
SELECT
CASE
WHEN highway IN ('motorway', 'motorway_link', 'trunk', 'trunk_link', 'primary', 'primary_link', 'secondary',
'secondary_link', 'tertiary', 'tertiary_link', 'residential', 'unclassified', 'living_street', 'service', 'road') THEN 'road'
WHEN highway IN ('footway', 'steps') THEN 'footway'
WHEN highway = 'path' THEN carto_path_type(bicycle, horse)
ELSE highway
END
END
$$;

/* Return int_access value which will be used to determine access marking.
Only a restricted number of types can be returned, with NULL corresponding to no access restriction */
CREATE OR REPLACE FUNCTION carto_highway_int_access(highway text, "access" text, foot text, bicycle text, horse text, motorcar text, motor_vehicle text, vehicle text)
dch0ph marked this conversation as resolved.
Show resolved Hide resolved
RETURNS text
LANGUAGE SQL
IMMUTABLE PARALLEL SAFE
AS $$
SELECT
CASE carto_highway_int_highway(highway, bicycle, horse)
WHEN 'road' THEN
carto_int_access('road', CASE
WHEN motorcar <> 'unknown' THEN motorcar
WHEN motor_vehicle <> 'unknown' THEN motor_vehicle
WHEN vehicle <> 'unknown' THEN vehicle
ELSE "access" END)
WHEN 'pedestrian' THEN carto_int_access('pedestrian', CASE WHEN foot <> 'unknown' THEN foot ELSE "access" END)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use NULLIF(foot, 'unknown')

WHEN 'footway' THEN carto_int_access('footway', CASE WHEN foot <> 'unknown' THEN foot ELSE "access" END)
WHEN 'cycleway' THEN carto_int_access('cycleway', CASE WHEN bicycle <> 'unknown' THEN bicycle ELSE "access" END)
WHEN 'brideway' THEN carto_int_access('bridleway', CASE WHEN horse <> 'unknown' THEN horse ELSE "access" END)
dch0ph marked this conversation as resolved.
Show resolved Hide resolved
ELSE carto_int_access(NULL, "access")
END
END
$$;

/* Uncomment lines below to create generated column for int_access
ALTER TABLE planet_osm_line DROP COLUMN IF EXISTS int_access;
ALTER TABLE planet_osm_line
ADD int_access text GENERATED ALWAYS AS (CASE WHEN highway IS NOT NULL THEN carto_highway_int_access(highway, "access", foot, bicycle, horse, tags->'motorcar', tags->'motor_vehicle', tags->'vehicle') ELSE NULL END) STORED;
*/
81 changes: 18 additions & 63 deletions project.mml
Original file line number Diff line number Diff line change
Expand Up @@ -444,32 +444,24 @@ Layer:
(SELECT
way,
(CASE WHEN feature IN ('highway_motorway_link', 'highway_trunk_link', 'highway_primary_link', 'highway_secondary_link', 'highway_tertiary_link') THEN substr(feature, 0, length(feature)-4) ELSE feature END) AS feature,
horse,
foot,
bicycle,
tracktype,
int_surface,
access,
int_access,
construction,
service,
link,
layernotnull
FROM ( -- subselect that contains both roads and rail
SELECT
way,
'highway_' || highway AS feature, --only motorway to tertiary links are accepted later on
horse,
foot,
bicycle,
'highway_' || (CASE WHEN highway = 'path' THEN carto_path_type(bicycle, horse) ELSE highway END) AS feature, --only motorway to tertiary links are accepted later on
tracktype,
CASE WHEN surface IN ('unpaved', 'compacted', 'dirt', 'earth', 'fine_gravel', 'grass', 'grass_paver', 'gravel', 'ground',
'mud', 'pebblestone', 'salt', 'sand', 'woodchips', 'clay', 'ice', 'snow') THEN 'unpaved'
WHEN surface IN ('paved', 'asphalt', 'cobblestone', 'cobblestone:flattened', 'sett', 'concrete', 'concrete:lanes',
'concrete:plates', 'paving_stones', 'metal', 'wood', 'unhewn_cobblestone') THEN 'paved'
END AS int_surface,
CASE WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
carto_highway_int_access(highway, access, foot, bicycle, horse, tags->'motorcar', tags->'motor_vehicle', tags->'vehicle') AS int_access,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reveals an inconsistency of how we're treating access tags. foot, bicycle, and horse get columns while motorcar, motor_vehicle, and vehicle don't.

If we're storing the raw tag text here, let's put them all in the tags column. If we're preprocessing we could have a smaller number of columns

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, our current database schema has foot, bicycle, and horse as columns (because these have been interpreted in rendering of highway=path for a long time) and not the other more specific access tags. I agree that with this change this distinction would not make that much sense any more and sourcing all of them from hstore would be reasonable. But since that would involve a database reload i would suggest to do this separately. Overall, i am not sure if this is worth breaking backwards compatibility of the database.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any issue with doing a reload. Being consistent on columns is a plus, even if we aren't able to reach a consensus on any preprocessing

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with removing foot/bicycle/horse from the columns. I am just not sure if it is worth doing a database layout change breaking backwards compatibility just because of this.

We could also source foot/bicycle/horse from hstore in anticipation of such a change now and defer the actual removal of the columns to when we do a database reload anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also source foot/bicycle/horse from hstore in anticipation of such a change now and defer the actual removal of the columns to when we do a database reload anyway.

This would make sense. But if we are sourcing all the detailed access tags via hstore, I suggest passing the hstore tags as a function argument to both carto_highway_int_access and carto_path_type. As well as being neater, it is presumably slightly more efficient to selectively access tags as needed rather than unpack them all to pass as function arguments [but I'm no SQL expert]?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to correct myself here: I wrongly remembered that tags in columns are also available in hstore, but they are not. Hence my suggestion would not work.

construction,
CASE
WHEN service IN ('parking_aisle', 'drive-through', 'driveway') THEN 'INT-minor'::text
Expand All @@ -491,15 +483,9 @@ Layer:
WHEN (railway = 'rail' AND service IN ('spur', 'siding', 'yard')) THEN 'INT-spur-siding-yard'
WHEN (railway = 'tram' AND service IN ('spur', 'siding', 'yard')) THEN 'tram-service'
ELSE railway END) AS feature,
horse,
foot,
bicycle,
tracktype,
'null',
CASE
WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
NULL,
construction,
CASE WHEN service IN ('parking_aisle', 'drive-through', 'driveway') THEN 'INT-minor'::text ELSE 'INT-normal'::text END AS service,
'no' AS link,
Expand All @@ -514,7 +500,7 @@ Layer:
z_order,
CASE WHEN substring(feature for 8) = 'railway_' THEN 2 ELSE 1 END,
CASE WHEN feature IN ('railway_INT-preserved-ssy', 'railway_INT-spur-siding-yard', 'railway_tram-service') THEN 0 ELSE 1 END,
CASE WHEN access IN ('no', 'private') THEN 0 WHEN access IN ('destination') THEN 1 ELSE 2 END,
CASE int_access WHEN 'no' THEN 0 WHEN 'restricted' THEN 1 ELSE 2 END,
CASE WHEN int_surface IN ('unpaved') THEN 0 ELSE 1 END
) AS tunnels
properties:
Expand Down Expand Up @@ -681,32 +667,24 @@ Layer:
(SELECT
way,
(CASE WHEN feature IN ('highway_motorway_link', 'highway_trunk_link', 'highway_primary_link', 'highway_secondary_link', 'highway_tertiary_link') THEN substr(feature, 0, length(feature)-4) ELSE feature END) AS feature,
horse,
foot,
bicycle,
tracktype,
int_surface,
access,
int_access,
construction,
service,
link,
layernotnull
FROM ( -- subselect that contains both roads and rail/aero
SELECT
way,
('highway_' || highway) AS feature, --only motorway to tertiary links are accepted later on
horse,
foot,
bicycle,
'highway_' || (CASE WHEN highway = 'path' THEN carto_path_type(bicycle, horse) ELSE highway END) AS feature, --only motorway to tertiary links are accepted later on
tracktype,
CASE WHEN surface IN ('unpaved', 'compacted', 'dirt', 'earth', 'fine_gravel', 'grass', 'grass_paver', 'gravel', 'ground',
'mud', 'pebblestone', 'salt', 'sand', 'woodchips', 'clay', 'ice', 'snow') THEN 'unpaved'
WHEN surface IN ('paved', 'asphalt', 'cobblestone', 'cobblestone:flattened', 'sett', 'concrete', 'concrete:lanes',
'concrete:plates', 'paving_stones', 'metal', 'wood', 'unhewn_cobblestone') THEN 'paved'
END AS int_surface,
CASE WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
carto_highway_int_access(highway, access, foot, bicycle, horse, tags->'motorcar', tags->'motor_vehicle', tags->'vehicle') AS int_access,
construction,
CASE
WHEN service IN ('parking_aisle', 'drive-through', 'driveway') OR leisure IN ('slipway') THEN 'INT-minor'::text
Expand All @@ -731,20 +709,14 @@ Layer:
WHEN (railway = 'rail' AND service IN ('spur', 'siding', 'yard')) THEN 'INT-spur-siding-yard'
WHEN (railway = 'tram' AND service IN ('spur', 'siding', 'yard')) THEN 'tram-service'
ELSE railway END)) AS feature,
horse,
foot,
bicycle,
tracktype,
CASE WHEN surface IN ('unpaved', 'compacted', 'dirt', 'earth', 'fine_gravel', 'grass', 'grass_paver', 'gravel', 'ground',
'mud', 'pebblestone', 'salt', 'sand', 'woodchips', 'clay', 'ice', 'snow') THEN 'unpaved'
WHEN surface IN ('paved', 'asphalt', 'cobblestone', 'cobblestone:flattened', 'sett', 'concrete', 'concrete:lanes',
'concrete:plates', 'paving_stones', 'metal', 'wood', 'unhewn_cobblestone') THEN 'paved'
ELSE NULL
END AS int_surface,
CASE
WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
NULL,
construction,
CASE WHEN service IN ('parking_aisle', 'drive-through', 'driveway') OR leisure IN ('slipway') THEN 'INT-minor'::text ELSE 'INT-normal'::text END AS service,
'no' AS link,
Expand All @@ -763,7 +735,7 @@ Layer:
z_order,
CASE WHEN substring(feature for 8) = 'railway_' THEN 2 ELSE 1 END,
CASE WHEN feature IN ('railway_INT-preserved-ssy', 'railway_INT-spur-siding-yard', 'railway_tram-service') THEN 0 ELSE 1 END,
CASE WHEN access IN ('no', 'private') THEN 0 WHEN access IN ('destination') THEN 1 ELSE 2 END,
CASE int_access WHEN 'no' THEN 0 WHEN 'restricted' THEN 1 ELSE 2 END,
CASE WHEN int_surface IN ('unpaved') THEN 0 ELSE 1 END,
osm_id
) AS roads_sql
Expand Down Expand Up @@ -913,32 +885,24 @@ Layer:
(SELECT
way,
(CASE WHEN feature IN ('highway_motorway_link', 'highway_trunk_link', 'highway_primary_link', 'highway_secondary_link', 'highway_tertiary_link') THEN substr(feature, 0, length(feature)-4) ELSE feature END) AS feature,
horse,
foot,
bicycle,
tracktype,
int_surface,
access,
int_access,
construction,
service,
link,
layernotnull
FROM ( -- subselect that contains both roads and rail/aero
SELECT
way,
'highway_' || highway AS feature, --only motorway to tertiary links are accepted later on
horse,
foot,
bicycle,
'highway_' || (CASE WHEN highway = 'path' THEN carto_path_type(bicycle, horse) ELSE highway END) AS feature, --only motorway to tertiary links are accepted later on
dch0ph marked this conversation as resolved.
Show resolved Hide resolved
tracktype,
CASE WHEN surface IN ('unpaved', 'compacted', 'dirt', 'earth', 'fine_gravel', 'grass', 'grass_paver', 'gravel', 'ground',
'mud', 'pebblestone', 'salt', 'sand', 'woodchips', 'clay', 'ice', 'snow') THEN 'unpaved'
WHEN surface IN ('paved', 'asphalt', 'cobblestone', 'cobblestone:flattened', 'sett', 'concrete', 'concrete:lanes',
'concrete:plates', 'paving_stones', 'metal', 'wood', 'unhewn_cobblestone') THEN 'paved'
END AS int_surface,
CASE WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
carto_highway_int_access(highway, access, foot, bicycle, horse, tags->'motorcar', tags->'motor_vehicle', tags->'vehicle') AS int_access,
construction,
CASE
WHEN service IN ('parking_aisle', 'drive-through', 'driveway') THEN 'INT-minor'::text
Expand All @@ -960,15 +924,9 @@ Layer:
WHEN (railway = 'rail' AND service IN ('spur', 'siding', 'yard')) THEN 'INT-spur-siding-yard'
WHEN (railway = 'tram' AND service IN ('spur', 'siding', 'yard')) THEN 'tram-service'
ELSE railway END) AS feature,
horse,
foot,
bicycle,
tracktype,
'null',
CASE
WHEN access IN ('destination') THEN 'destination'::text
WHEN access IN ('no', 'private') THEN 'no'::text
END AS access,
NULL,
construction,
CASE WHEN service IN ('parking_aisle', 'drive-through', 'driveway') THEN 'INT-minor'::text ELSE 'INT-normal'::text END AS service,
'no' AS link,
Expand All @@ -983,7 +941,7 @@ Layer:
z_order,
CASE WHEN substring(feature for 8) = 'railway_' THEN 2 ELSE 1 END,
CASE WHEN feature IN ('railway_INT-preserved-ssy', 'railway_INT-spur-siding-yard', 'railway_tram-service') THEN 0 ELSE 1 END,
CASE WHEN access IN ('no', 'private') THEN 0 WHEN access IN ('destination') THEN 1 ELSE 2 END,
CASE int_access WHEN 'no' THEN 0 WHEN 'restricted' THEN 1 ELSE 2 END,
CASE WHEN int_surface IN ('unpaved') THEN 0 ELSE 1 END
) AS bridges
properties:
Expand Down Expand Up @@ -1923,8 +1881,7 @@ Layer:
CASE
WHEN oneway IN ('yes', '-1') THEN oneway
WHEN junction IN ('roundabout') AND (oneway IS NULL OR NOT oneway IN ('no', 'reversible')) THEN 'yes'
END AS oneway,
horse, bicycle
END AS oneway
FROM planet_osm_line l
WHERE highway IN ('motorway', 'motorway_link', 'trunk', 'trunk_link', 'primary', 'primary_link', 'secondary', 'secondary_link', 'tertiary',
'tertiary_link', 'residential', 'unclassified', 'road', 'service', 'pedestrian', 'raceway', 'living_street', 'construction')
Expand All @@ -1949,15 +1906,13 @@ Layer:
table: |-
(SELECT
way,
highway,
CASE WHEN highway = 'path' THEN carto_path_type(bicycle, horse) ELSE highway END AS highway,
construction,
name,
CASE
WHEN oneway IN ('yes', '-1') THEN oneway
WHEN junction IN ('roundabout') AND (oneway IS NULL OR NOT oneway IN ('no', 'reversible')) THEN 'yes'
END AS oneway,
horse,
bicycle
END AS oneway
FROM planet_osm_line
WHERE highway IN ('bridleway', 'footway', 'cycleway', 'path', 'track', 'steps', 'construction')
AND (name IS NOT NULL
Expand Down
Loading
Loading