Meteorological meanderings, part 4: creating interactive maps

As with part 3, this is not exactly meteorology, but rather a necessary component of a modern meteorological application. The ability to view a map, pan and zoom in it, and click on locations within the map is something we take for granted today, with sites and apps like Google Maps having popularized the concepts. Bringing this functionality to a weather application is only natural.

Map tiles and pyramids

Have you ever noticed, right after zooming in on an interactive map, that things sometimes look blurry for a moment before clearing up? Or, more jarringly, that occasionally there are gray squares where pieces of a map should be? This is because of the way maps are generated and loaded.

In order to optimize loading times and minimize data transfer, square tiles of map data are lazy-loaded whenever we access a particular area on a map. These tiles are most often 256×256 pixels, and they contain a “good enough” representation of the desired area at the current zoom level. This way, a minimal number of tiles needs to be loaded in order to show us what we want to see. When we pan to another area, the tiles that haven’t yet been loaded are fetched from the server and displayed. When we zoom, fresh tiles at the new zoom level are loaded and displayed. In order to minimize the jarring effect of clearing the screen and then having all new tiles load when we zoom in, the previous zoom level’s tiles are retained, but at a higher zoom level (which is why they look blurry), and then gradually replaced by tiles at the proper zoom level as they load.

This concept is often represented as a pyramid:

Pyramided tiles make up the majority of the UI in services like Google Maps. In fact, often there are multiple sets of pyramided tiles layered on top of each other: for example, we can enable an overlay of current traffic conditions “above” the base map. These layers allow for rich customization of displayed data, and it just so happens that layering can be very useful for displaying weather information.

Overlaying actual weather visualizations is beyond the scope of this post, but we’ll get a good start by creating two map layers: a “bottom” layer with some base information and a “top” layer with geopolitical division lines. The reason for this is, when showing particularly turbulent (and, therefore, colorful) weather, it’s helpful to see where it’s occurring. If we only have one map layer and one weather layer, the weather layer may obscure too much of the map layer, making it difficult to orient ourselves. Having the weather layer sandwiched between bottom and top map layers helps to both see the weather activity clearly and still be able to determine its location relative to known positions without much trouble.

Generating map tiles from mapping data

OpenStreetMap data was used for geocoding in part 3, and we’re going to use it again for map tile generation. This time, however, we need to apply different filters to the data. In addition to keeping the names of administrative boundaries and such, we need to see things like rivers, major motorways, and other recognizable landmarks to assist users in orienting themselves. Here is an example of what an osmfilter command might look like for this purpose:

$ osmfilter source-data.o5m --drop-version --keep="highway= place= boundary= leisure=park =nature_preserve landuse=recreation_ground natural=water protect_class=" --out-o5m -o=filtered-data.o5m

At this point, it’d be good to take a step back and note that the steps below are rather manual. This is because they serve as a good explainer of how map tiles work, and they are suitable for certain customization that we’re looking to do. There are, however, easier and quicker ways to generate map tiles. For information on that, take a look at OpenMapTiles.

Instead of letting the Nominatim Docker application take care of importing our data into the database, we’ll do that ourselves using osm2pgsql, which can be installed via apt in Debian- and Ubuntu-based distributions. The import process looks like this:

$ createdb -E UTF8 gis
$ psql -c "CREATE EXTENSION postgis;" -d gis
$ psql -c "CREATE EXTENSION hstore;" -d gis
$ osm2pgsql --slim -d gis --hstore --multi-geometry --number-processes 8 -C 2000 filtered-data.o5m

The above script creates a new database named gis, enables the postgis and hstore Postgres extensions (which, of course, requires PostGIS to be installed), and finally runs osm2pgsql using our filtered dataset.

To generate the map tiles, we’ll need a few things. First is a map style definition, which controls how all of the various map elements are rendered. A good starting point is HDM-CartoCSS. It’s a relatively minimal map style, which makes it easy to modify to remove things we don’t need for either the bottom or top layer.

The second thing we’ll need is a way to compile the map style into a format that can be used for map tile generation. In this case, we’ll use CartoCSS. It’s easy to install using npm, as described in their documentation. CartoCSS takes the YAML (or .mml) file from a style definition like HDM-CartoCSS and outputs a Mapnik XML file, which can then be consumed by tile generation tools compatible with this format.

The third and final thing we’ll be using here is the actual map tile generator. There are a number of options out there. A pretty simple one is PolyTiles, which consists of a single Python file.

The HDM-CartoCSS wiki specifies some prerequisites that may be useful to download. Additionally, the project.yml file should be modified to point to our gis database. And, of course, we’ll want to make some design changes to the map styles. After that, the rest of the process is straightforward:

$ cd HDM-CartoCSS
$ mv project.yml project.mml # CartoCSS sometimes complains about file extensions
$ carto project.mml > style.xml
$ cd ..
$ python3 polytiles.py -b -180 -90 180 90 -s HDM-CartoCSS/style.xml -t output --threads 12 -z 0 5

The above example generates map tiles using 12 threads at zoom levels 0-5. When combining all of the concepts in this post, we can get a bottom layer that might look like this:

And a top layer that might look like this:

Combined, the view is pretty reasonable:

So, if a weather layer was between the two map layers, we would have the ability to see the severity of the weather as well as the major roads and labels on top of it. For less severe weather, where there would be fewer artifacts on the map, the bottom layer would be visible as well, providing additional context.

To render the map tiles, we can use Leaflet on a pretty simple HTML page:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/leaflet.css" integrity="sha512-Rksm5RenBEKSKFjgI3a41vrjkw4EVPlJ3+OiI65vTjIdo9brlAacEuKOiQ5OFh7cOI1bkDwLqdLw3Zg0cRJAAQ==" crossorigin=""/>
        <script src="https://unpkg.com/[email protected]/dist/leaflet.js" integrity="sha512-/Nsx9X4HebavoBvEBuyp3I7od5tA0UzAxs+j83KgC8PU0kgB4XiK4Lfe4y4cgBtaRJQEIFCW+oC506aPT2L1zw==" crossorigin=""></script>
        <style>
            html, body, #map {
                width: 100%;
                height: 100%;
                margin: 0;
                padding: 0;
            }
        </style>
    </head>
    <body>
        <div id="map"></div>
        <script>
            var map = L.map('map').setView([0, 0], 3);
            L.tileLayer('output/{z}/{x}/{y}.png', {
                minZoom: 0,
                maxNativeZoom: 5,
                maxZoom: 8,
                attribution: 'Map data &copy; <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors, <a href="https://creativecommons.org/licenses/by-sa/2.0/">CC-BY-SA</a>',
                id: 'base'
            }).addTo(map);
        </script>
    </body>
</html>

The call to L.tileLayer above is the most important part: it defines how our tiles are structured in the filesystem, which zoom levels are available, and controls other aspects of displaying them. Leaflet then takes care of loading the appropriate tiles, combining them into a cohesive view, and allowing users to pan and zoom around the map.

This was an overview of creating and displaying customized map tiles. As mentioned above, there are other, more automated, ways of doing this as well. A lot of online resources exist for various cartographic applications – more than for meteorology, it would seem. I’m not sure if there will be a part 5 of this blog post series, but if there is, it will likely go into detail on the integration of mapping and weather data.