Apply OpenMapTilev3.14 changes #47
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Make a few spacing adjustments for ease-of-reading and consistency * fix bbbike naming and other source urls * remove unneeded `override` * added `list-bbbike` target
Closes openmaptiles#904 * Make all data-related targets like `download*`, `import-osm`, `import-borders`, and `generate-tiles` into `area`-aware -- making it possible for multiple data files to coexist inside the `./data` dir. * Add `make download area=... [url=...]` command to automatically download any kind of area by checking Geofabrik, BBBike, and OSM.fr, optionally from a custom URL. Supports `area=planet` too. * Do not re-download area with `make download-*` if it already exists. * Automatically rename `<area>-latest.osm.pbf` into `<area>.osm.pbf` * If `area=...` parameter is not given to `make`, see if there is exactly one `*.osm.pbf` file, and if so, use `*` as the `area`. * Configure many variables in the .env file, overriding the defaults in tools * If `<area>.osm.pbf` exists, but `<area>.dc-config.pbf` is missing, generate it using `download-osm make-dc` command. Also: * closes openmaptiles#614 * closes openmaptiles#647 * partially addresses openmaptiles#261
…es#915) * Set `MAX_ZOOM` to 7 by default. * Remove `QUICKSTART_MIN/MAX_ZOOM` - unneeded complexity with two env vars. We can just use `MIN_ZOOM` and `MAX_ZOOM`. See also openmaptiles#261 * Generate dc-config yaml file with a new `make generate-dc-config` step. It will compute BBOX based on the downloaded data file. This step is not needed for planet generation. * Generate Imposm replication file only when `DIFF_MODE` is `true`. Not needed otherwise. If the data source does not support it, it will throw an error.
Running './quickstart monaco' twice in a row will cause an error the second time.
I would like to reformat all of our SQL to have a concise coding style. This makes it far easier to understand the code for a casual contributor, and lets us spot errors more easily. Most importantly, it makes it much easier to grep (search) the code because it is more likely to be in the same syntax Some key changes: * SQL keywords are always UPPERCASE, e.g. `SELECT WHEN AS END ...` * types, variables, aliases, and field names (identifiers) are always lower case * `LANGUAGE 'plpgsql'` is now `LANGUAGE plpgsql` (no quotes) * a few minor spacing/semicolon cleanups P.S. Per @TomPohys request, `TABLE` is spelled using upper case despite being a type for consistency with PG Docs. Same for `LANGUAGE SQL` vs `LANGUAGE plpgsql`.
Make sure that core targets (i.e. part of the `all` target like build-sql and build yaml files) do not run multiple times if they already exist. Makefile for some reason does not like it when a real target depends on a PHONY target, and re-runs it. I added an `if` statement to skip building targets if their result already exists.
Use `update_water.sql` for all queries that update db to be consistent with the other layers.
Add missing `-q` flag
…iles#911) Resolve openmaptiles#776 Thanks @zstadler for this PR.
Allow users to override how many import-sql files are loaded in parallel at the same time. This change should be a noop
* Current code incorrectly passes `IMPOSM_CONFIG_FILE` to the `generate-tiles` image, but should pass it to the tools. * add a test to ensure imposm config exists * add a test to ensure area is set during updates
Minor optimization - in function `layer_aerodrome_label(bbox, zoom_level, pixel_width)` last parameter is not being used, so removing.
* Change GitHub Action to the dedicated server Ralph * bugfix - let quickstart generate-tiles up to zoom 14
It's ok if integrity test fails -- makes the whole testing process faster by 10 minutes at a cost of some extra CPU cycles.
* All functions that access database must be declared as `STABLE`, not `IMMUTABLE` -- because database can change at any moment, e.g. during an update * there are a few functions that could be made `STRICT` -- passing `NULL` as a parameter will always result in a `NULL`, but for some reason that causes a significant decrease in perf. * tagged one function as parallel safe NOTE: somehow `ST_AsMVT()` method of tile generation is showing 70-90% slowdown with this patch. I am not sure of why this is happening. If the reason is the `IMMUTABLE` -> `STABLE` change, we may have to dig deeper into PG optimization
* Building aggregation at zoom 13
…penmaptiles#949) Improve 97216c5 and openmaptiles#853 In case of replay update it may fails because of already existing primary key on osm_id. Add a on conflict clause to make it fail safe.
* Remove alignment of AS in SQL * Remove alignment of CREATE TABLE in SQL
…#961) * Replace ILIKE by lower() for insensitive case compare * Refactoring SQL insensitive case compare
Move the hard coded `openmaptiles.yaml` from Makefile to a variable. Allows use other tileset definition. A step forward easy alternative layer definition.
Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged openmaptiles#896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses openmaptiles#814. * Remake update_peak_point use incremental update openmaptiles#814 * Make update_aerodrome_label_point use incremental update openmaptiles#814 * Make housenumber_centroid use incremental update openmaptiles#814 * Make update_continent_point use incremental update openmaptiles#814 * Make update_island_point use incremental update openmaptiles#814 * Make update_island_polygon use incremental update openmaptiles#814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update openmaptiles#814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update openmaptiles#814 * Make osm_poi_polygon use incremental update openmaptiles#814 Thanks @frodrigo
…aptiles#951) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) to ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. It is a based on the already merged openmaptiles#944 It is a separated PR as less obvious than previous. It replaces the reset of the `rank` field to NULL by missing value resulting of `LEFT JOIN`. It avoid triggering a new update on the table by reset the value then re-seting it to initial or new value. It addresses openmaptiles#814. Thanks @frodrigo
Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) to ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. It is a based on the already merged openmaptiles#944 It is a separated PR as less obvious than previous. It replaces the reset of the `rank` field to NULL by missing value resulting of `LEFT JOIN`. It avoid triggering a new update on the table by reset the value then re-seting it to initial or new value. It addresses openmaptiles#814. Thanks @frodrigo
…maptiles#1389) This PR fixes openmaptiles#1388 by doing a test cast for the type and if it fails adding the type.
This PR adds support for `capital=6` in the place layer, which is used for tagging county seats in the US. A county seat is the capital of a county. In addition, I reorganized the code a bit to be simpler. As noted in ZeLonewolf/openstreetmap-americana#384, we would like to render county seats along with state and national capitals, as this is typical styling on American-style maps. Update with also with `capital=3` and `capital=5`
This PR removes support for `waterway=riverbank`. Usage of waterway=riverbank has [declined to globally insignificant levels](https://taginfo.openstreetmap.org/tags/waterway=riverbank). In other projects: * Support for `waterway=riverbank` was removed from the iD editor in openstreetmap/iD#5591 * Removal of `waterway=riverbank` was tracked for JOSM in [JOSM ticket 21630](https://josm.openstreetmap.de/ticket/21630) and was completed in revision [18519](https://josm.openstreetmap.de/changeset/18519/josm) * Removal of `waterway=riverbank` from openltmap (Lithuania vector map) occurred in [[1]](https://github.com/openmaplt/vector-map/commit/b636ef85efaaae4f6bfeecc15bd95729c4cac21a),[[2]](https://github.com/openmaplt/vector-map/commit/507f095b35e7f5a5afef9267b4ec4eafd7d6f9b5)
Guided busways (or bus guideways in OSM terminology) are special roadways along which bus operators can travel smoothly at high speeds without steering. These are tagged highway=bus_guideway on OpenStreetMap. Currently, highway=busway is surfaced in OpenMapTiles as the busway class in the transportation layer, but highway=bus_guideway is not present. These two types of roadways serve generally similar purposes, so it would make sense to have guided busways at the same zoom levels.
- Add `make import-diff` and `make generate-changed-tiles` - Update `UPDATE.md` accordingly - Other updates and clarifications in `UPDATE.md`
Adding missing requires Natural Earth tables for layers.
This PR updating Natural Earth dataset from v4.1 to 5.1.2. It is in cooperation with openmaptiles/openmaptiles-tools#414, where is `import-data` docker image updated. PR removes manual fixes for lake merging (already fixed in version 5.1.2) Switched comparison from `gn_ascii` (removed from NE5) to `name_en`. Could be merged after openmaptiles/openmaptiles-tools#413. After OMT-T is merged, it will use `latest` image (until the release of OMT-T v7)
This PR creates larger polygons of `residential` land-use areas for upper zoom levels (especially for zoom 6 - 10). Clusters are created with the same logic as for [`building on z13`](https://github.com/openmaptiles/openmaptiles/blob/b14da2eafb4d3dc5a1278f1bec9ecdf961acded4/layers/building/update_building.sql#L34-L49). The generalization of land-use prolongs the `import-sql` step of the `landuse` layer. But it is still faster than the `building z13` generalization which with the parallel importing keeps the whole `import-sql` time untouched.
In `transportation_name` layer for `shipway` used Instead of `subclass` use `class` attribute for `ferry`. But `subclass` remains.
This includes quarry in the list of landuse features Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
Update documentation about a place - capital possible values. Based on openmaptiles#1401
* OpenMapTiles style * Rename style snippets from layer_name.json to defaultstyle.json * Add README to icons dir. * Add README to icons dir. * Rename snippets to style.json * Fix spritezero - move icons to svg subdir * Move icons from style/icons/svg directly to style/icons Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
Update documentation before release v3.14.
…emove all_tags mapping temporarily
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.