Extract coordinates from an ESRI ArcGIS web map¶

Sometimes you come across an ESRI/ArcGIS map on the web, and don't know how to download the data from it. We have two ways to take care of that!

This method is courtesy @mariepastora

The manual way¶

In Chrome or Chromium browser, go to View > Developer > Developer Tools

Once the Developer Tools menu is open, navigate to the Network tab (you might need to click the >> button and select it from a dropdown)

Refresh the page and watch all of the requests the page is making fly by!

Use the search bar to find pbf. Click the top result, and copy the URL displayed on the right hand side.

Take the f=pbf part of the URL and convert it to f=geojson.

There you go! Save the file as a .json or .geojson file and you're good to go.

The automatic way¶

The automatic method uses Playwright and Python (although you could write it in any other language).

We open the page with Playwright, inspect the network connections until we see the pbf one, then convert the URL to geojson and save it locally.

In [1]:

# The URL of the map
url = "https://mncommerce.maps.arcgis.com/apps/webappviewer/index.html?id=17cc12ed0e204a89b435fcd128a92eb6"

# The URL of the map url = "https://mncommerce.maps.arcgis.com/apps/webappviewer/index.html?id=17cc12ed0e204a89b435fcd128a92eb6"

In [15]:

# The code
from playwright.async_api import async_playwright
import requests

# Launch a browser
playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless = False)
page = await browser.new_page()

# Visit the page, wait for the pbf file
print("Loading page", url)
async with page.expect_response("*?f=pbf*") as response_info:
    await page.goto(url)

# Adapt it into geojson
response = await response_info.value
print("Found PBF URL", response.url)
print("Converting to geojson")
geojson_url = response.url.replace("?f=pbf", "?f=geojson")

print("Saving as output.json")
with open("output.geojson", "wb") as f:
    f.write(requests.get(geojson_url).content)


# Close browser
await browser.close()

# The code from playwright.async_api import async_playwright import requests # Launch a browser playwright = await async_playwright().start() browser = await playwright.chromium.launch(headless = False) page = await browser.new_page() # Visit the page, wait for the pbf file print("Loading page", url) async with page.expect_response("*?f=pbf*") as response_info: await page.goto(url) # Adapt it into geojson response = await response_info.value print("Found PBF URL", response.url) print("Converting to geojson") geojson_url = response.url.replace("?f=pbf", "?f=geojson") print("Saving as output.json") with open("output.geojson", "wb") as f: f.write(requests.get(geojson_url).content) # Close browser await browser.close()

Loading page https://mncommerce.maps.arcgis.com/apps/webappviewer/index.html?id=17cc12ed0e204a89b435fcd128a92eb6
Found PBF URL https://services1.arcgis.com/yfalcTIlueMdl9fz/arcgis/rest/services/Permitted_Route/FeatureServer/0/query?f=pbf&where=1%3D1&returnGeometry=true&spatialRel=esriSpatialRelIntersects&maxAllowableOffset=5.291677250021158&outFields=*&maxRecordCountFactor=2&outSR=102100&resultOffset=0&resultRecordCount=4000&cacheHint=true&quantizationParameters=%7B%22mode%22%3A%22view%22%2C%22originPosition%22%3A%22upperLeft%22%2C%22tolerance%22%3A5.291677250021158%2C%22extent%22%3A%7B%22xmin%22%3A197290.8295%2C%22ymin%22%3A5160602.503900001%2C%22xmax%22%3A554254.7607000005%2C%22ymax%22%3A5402877.7697%2C%22spatialReference%22%3A%7B%22wkid%22%3A26915%2C%22latestWkid%22%3A26915%7D%7D%7D
Converting to geojson

In [ ]: