Interested in joining the Silph Research group? Learn More »

Silph Study: #027

Published:
09.09.2019

OpenStreetMap and GO: What Needs to Change Before the Next Update


The following article is intended as a summary guide or “white paper” of sorts intended for interested travelers and also potentially Niantic engineers and future AR game developers. The author, WoodWose, is a Silph Road Research Group Scientist Emeritus who has had a long-term interest in researching the interactions between Pokémon GO and OpenStreetMap (OSM). This is a slight departure from regular Silph Research Group studies, but we hope that it is useful all the same.


About once a week, as if by clockwork, someone on The Silph Road subreddit invariably asks:

“When will OpenStreetMap data be updated in Pokémon GO?”

This is a fair question. Niantic has used OSM data to power various features of Pokémon GO since the very start, but the answer is simply something we cannot know. Niantic updates their maps when they need to. There is no published schedule. Over Pokémon GO’s three-year lifespan, there have been many months, even years, between updates of different map-related features.

As time has passed, however, the map data that Niantic uses has been investigated, analyzed, and most importantly experienced by trainers around the world. Through those observations, several flaws in the system have become increasingly apparent. The following issues by no means constitute a definitive list, but they do represent a small handful of problems that are either impacting a significant number of trainers or affecting specific Pokémon GO communities in a particularly severe way.

Key Points

    1. The OSM tag natural=bay completely blocks wild Pokémon appearing on some populated islands. There is no OSM standard to say that islands shouldn’t be covered by this tag.
      Update – October 16, 2019: The Saronic Gulf (the primary example used in this article) has regained wild Pokémon. Other locations, such as the Gulf of Cádiz remain without wild Pokémon. Despite the change, it appears that natural=bay has not been removed from the list of spawn blocking tags outside of the Saronic Gulf.
    2. Gyms located in parks that were mapped as “multipolygon relations” or surrounded by closed “highways” (which in OSM includes paths, footpaths, and dirt tracks) are unable to host EX raids.
      ✓ Update – October 10, 2019: Niantic has rolled out an EX raid update. Not only were both EX raid related recommendations in this article implemented, the hidden map data that controls EX raid eligibility was also updated to a newer version, possibly matching the map data that is currently used to determine nesting areas!
    3. Certain tags representing large geological features cause nests, namely natural=plateau and natural=valley. These map tags are able to eclipse entire urban areas, flooding them with a single species of Pokémon for weeks at a time and interfering with “increased appearance” events.
      ✓ Update – May 14, 2021: All of the major known nests associated with natural=plateau andnatural=valley appear to have been removed from that game. In particular, the removal of the plateau nests in Berlin was acknowledged in the release notes for version 0.207. At the time of writing, it is unclear if natural=plateau andnatural=valley no longer cause nests globally, or if the fixes were targeted at certain locations.
    4. The map data used for different game features has been sourced at different times. There are currently gaps of over a year between different game-map features.
    5. Certain important tags associated with nests and biomes are not displayed on the visual game map at all.

Bays Blocking Spawns

Certain OSM tags have the effect of limiting or even eliminating Pokémon spawns within their mapped areas. One such example is natural=bay. Like many other water-related tags, it blocks wild Pokémon from appearing inside any area on which it is present – with the exception of Pokémon spawns from items such as incense and lures.

Every time blocked areas have been updated using newer map data, the tag natural=bay has consistently caused issues for various coastal trainer communities across the globe: time, and time, and time again.

While similar issues have occurred with other tags (such as university campuses being incorrectly tagged), the unfortunate issue with blocking natural=bay is that it effectively cuts off already isolated island communities from a significant portion of the Pokémon GO experience and leaves trainers (who have often invested time and money in the game) unable to participate in fundamental gameplay elements for months at a time.

Image: The Saronic Gulf, west of Athens, Greece was covered by a spawn-blocking natural=bay tag, shown above as red. In this case, 3146.86km square kilometers encompassing several populated islands have been left without wild Pokémon since March 2019. The problem was quickly “fixed” on OSM but hasn’t been updated in Pokémon GO. Map © OpenStreetMap contributors.

To compound the issue further, the OSM wiki page for the natural=bay tag states:

“There is currently no definitive rule if small islands within a bay should be mapped as part of the bay or if they should be excluded of [sic] the bay area as inner rings of the corresponding multipolygon relation.”

This means that some OSM mappers do not include populated islands as “inners” of natural=bay areas and can’t necessarily be faulted for mapping in this way. As a result, many populated islands in bays are covered by tagging that results in wild Pokémon being completely absent, regardless of the level of urban development or historic GPS/cellular activity. Unless this is changed, more islands will almost certainly be impacted by the same problem in future updates as additional bay features are inevitably added to OSM.

Lack of EX Eligibility In Park Areas

Putting aside the aging map data used for EX Raids, there are additional outstanding issues that continue to impact EX eligibility.

Two such issues are:

1) Gyms located in parks that are mapped as “multipolygon relations” (complex map areas usually drawn using more than one line) have never been eligible unless another factor such as sponsorship is at play. This has been the case since the EX Raid feature was officially released. These same parks are otherwise completely capable of becoming nests in-game and often represent the largest, most socially/culturally significant parks in urban areas (the Sydney Royal Botanic Gardens is one such example).

2) Gyms surrounded by simple closed highway loops cannot be EX Eligible. In OSM, the key highway= indicates anything from a narrow walking track (highway=path) to a multi-lane freeway (highway=motorway). Blocking EX eligibility on gyms surrounded by closed highway loops may have been intended to prevent crowds of trainers gathering around POIs in the middle of roundabouts and other similar roads, but it also has the effect of blocking many gyms from EX eligibility if they are surrounded by any closed highway= loop.

If the intention was specifically to block EX Raids in roundabouts and similar features, it may be more effective to apply the exclusion to gyms surrounded by closed ways tagged with the key junction=, or more specifically tagged with junction=roundabout as opposed to highway=.

Large Geological Features Causing Nests

Nests are one of the oldest features in Pokémon GO and were one of the earliest community-building elements as trainers shared the locations of sought-after species with each migration.

Nesting tags can generally be allocated into several basic categories:

  1. park and recreation features (e.g. parks, gardens, playgrounds, and recreation grounds)
  2. “open” natural features (e.g. meadows, heaths, and grasslands)
  3. agricultural features (e.g. vineyards, farmland, and orchards)

Typically, these tags represent open green spaces that are not significantly covered by built-up urban development and large residential populations.

Two tags have come to stand out as exceptions to the above descriptions:

  1. natural=plateau (multiple cases across Berlin, Germany)
  2. natural=valley (evident in Považské Podolie, Slovakia)

Entire valleys and plateaus becoming nests is problematic in that they do not represent open green “park-like” spaces and are clearly not the intended functionality of the nest feature. The aforementioned tags only occur a small number of times globally. In several cases, they intersect built-up towns and cities, impacting significant areas sometimes consisting of hundreds of square kilometers.

Image: A significant portion of the city of Berlin, Germany, is covered by nesting natural=plateau areas, shown as dark green. Map © OpenStreetMap contributors.

While some may appreciate the novelty of these “meganests,” there are a number of issues associated with them, including:

  • The pool of available nesting species is dominated by older, common, and non-meta relevant Pokémon (a different topic in its own right), and this results in these locations being swamped by a single kind of undesirable Pokémon for weeks at a time.
  • Nests have frequently been observed to limit or even block the effects of particular events where select rare Pokémon appear more frequently in the wild.  Trainers who are located deep within these areas are all but excluded unless they travel significant distances.

A similar issue occurred with the landmass of Long Island, New York. Fortunately, in the case of Long Island, the tag that resulted in the nest was added in error. After several months and a lot of noise from the Long Island trainer community, a nest update resolved the problem. Conversely, the tags associated with plateaus and valleys are unconventional but still accurate, meaning that they cannot simply be removed from OSM without making changes that equate to tagging for the renderer.

So, if updating to newer “correct” map data isn’t the solution for these meganests, what is?

Some travelers hold the hypothesis that Niantic simply set all tags that contain the keys natural= and leisure= as nests and then took steps to remove those that may cause issues. Examples include the tags natural=wood and leisure=nature_reserve; both do not appear to have nesting properties despite still allowing spawns within their boundaries. If Niantic is capable of removing certain tags from their “nesting list,” this may be the appropriate course of action for tags such as natural=plateau and natural=valley.

The Game Map as a Visual Means of Communication

Because the map data used for Pokémon GO’s current visual game map was sourced during December of 2017, whereas EX eligibility was sourced July 2016 and nest map data was sourced February 2019, there is a disconnect between what trainers see on the game map and what they experience in gameplay. This disconnect is compounded further because Niantic has frequently pushed notifications or made statements including:

Different Pokémon may be appearing in parks around you. It’s a great time to explore your local parks!

The following Pokémon will be appearing more frequently in the wild around water…

These have frequently left travelers wondering – which parks? what water? The game map does not render the majority of nesting “park” features; many kinds of waterways do not display in-game despite having an obvious effect when it comes to increasing water types in their proximity; and the disconnect in time between certain map-related features further adds to the confusion.

Is it really ideal that trainers must use external map sources and database queries just to identify whether a park “counts” or if Pokémon GO recognizes that a particular body of water exists?

There are two distinct solutions to these problems:

Firstly, Niantic could synchronize the versions of all map-derived features so that game visuals, nests, and EX eligibility match, clearly communicating the environment as Pokémon GO understands it.

Secondly, Niantic could render more tags on the visual map. Currently, only 3 out of 20+ known OSM tags associated with nests¹ show as dark green, and OSM mapped waterways such as waterway=stream and waterway=river do not render at all despite their importance to particular varieties of water-type Pokémon.

What Niantic Has Done Right

It may seem easy to disparage Niantic over some of the issues discussed above. It is important to acknowledge, however, that as one of the first of its kind, Pokémon GO has achieved a lot as a global augmented reality game.

Travelers can move to a distant country, where the language, urban design, and culture might be completely distinct, and yet Pokémon GO’s gameplay so often manages to adapt, presenting a unique yet somehow familiar and deceptively detailed gameplay experience tying together the mapped urban fabric, biomes, weather and countless points of interest.

One could only imagine the kinds of discussions that occurred behind closed doors in Niantic’s offices as those responsible debated the wide variety of map features that ended up contributing to nests, biomes and other game features. It’s likely some difficult design choices had to be made.

Niantic has also shown that they do have the ability to listen and adapt, albeit at their own pace. The addition of more Pokémon to parks and some mapped paths with a focus on rural areas was a great start and made many previously unviable locations at least somewhat playable. Additionally, the update that removed the Long Island “meganest” in New York allowed thousands of trainers to participate in events where previously they were unintentionally restricted from doing so.

Parting Words

It would be to Niantic’s benefit to resolve these issues promptly – Niantic’s use of map data is a critical element when it comes to players’ interactions with their platform. OSM data plays an important role in Pokémon GO as well as other titles such as Harry Potter: Wizards Unite. Ignoring these problems or placing them on the backburner can leave impacted trainers in a situation where they might direct their energies and enthusiasm in less than desirable directions, such as seeking out exploits or map vandalism. Further, Niantic arguably has a responsibility as a trailblazer in this genre to ensure that they establish and maintain fair, high-quality ways for their users and customers to interact with the game worlds that they create, as they will set the standard for AR titles yet to come.

Until next time, see you on the highway=footway, travelers!

-Scientist (Emeritus) WoodWose

With special thanks to Scientists Cham1nade and Skyeofthetyger for always putting up with my inconsistent use of the Oxford comma, and for being amazing, patient copy editors.

Discussion regarding OpenStreetMap and Pokémon GO almost inevitably leads to a spike in vandalism of the platform that dedicated mappers have to spend hours tracking and cleaning up when they could instead be working on improving the map for everyone. That being said, the Meowth is well and truly out of the bag when it comes to OSM and Pokémon GO, and many of the above issues result in many illegitimate edits in their own right. If you are looking at adding something to OSM, maybe start by checking out the OSM wiki, remembering to keep your contributions fair and honest!

Footnotes

¹ The following tags cause nests and EX eligibility, and are known to render as Dark Green on the game map:
leisure=golf_course, leisure=park and landuse=recreation_ground.

The following tags have been observed to cause nests and EX eligibility (assuming all tags are shared), but do not appear on the game map:

leisure=garden, leisure=pitch, leisure=playground, leisure=recreation_ground, landuse=farmland, landuse=farmyard, landuse=grass, landuse=greenfield, landuse=meadow, landuse=orchard, landuse=vineyard, natural=grassland, natural=heath, natural=moor and natural=scrub, as well as the discussed nesting oddities natural=plateau and natural=valley.

Appendix: Timeline of known OSM to Pokémon GO Updates

Date

In-Game Map Visuals

Nests & Blocks

EX Eligibility

Path Spawns

7-16 Global Google Maps visuals OSM active at launch OSM data sourced OSM data sourced?
8-16
9-16
10-16
11-16
12-16 New OSM data sourced New OSM data sourced Implemented
01-17 South Korea only OSM visuals Updated
02-17
03-17
04-17
05-17
06-17
07-17
08-17 New OSM data sourced
09-17
10-17
11-17 Global OSM visuals Implemented
12-17 New OSM data sourced
01-18 Updated
02-18
03-18
04-18 New OSM data sourced + updated
05-18
06-18
07-18
08-18
09-18
10-18
11-18
12-18
01-19
02-19 New OSM data sourced OSM data sourced? (Unconfirmed) New OSM data sourced
03-19 Updated Updated
04-19
05-19
06-19
07-19
08-19
09-19
10-19 Updated

OpenStreetMap and the magnifying glass logo are registered trademarks of the OpenStreetMap Foundation. The Silph Road is not endorsed by or affiliated with the OpenStreetMap Foundation.