OpenStreetMap：将请求连接到循环中，迭代每个 3166 个国家/地区代码，使用 Python 将响应解析为 DF

问题描述 投票：0回答：1

我目前正在处理在 Overpass-Turbo 的 API 端运行的组合请求：目的是连接如下请求；

[out:csv(::id,::type,"name","addr:postcode","addr:city","addr:street","addr:housenumber","website"," contact:email=*")][timeout:600];
area["ISO3166-1"="NL"]->.a;
( node(area.a)[amenity=childcare];
  way(area.a)[amenity=childcare];
  rel(area.a)[amenity=childcare];);
out;

使用 ISO3166-1 的密钥 - 请参阅 https://de.wikipedia.org/wiki/ISO-3166-1-Kodierliste 注意 - 我想运行例如在 Python 中 - 用不同的国家/地区代码进行编码

Netherlands, Germany, Austria, Switzerland, France,

等等 - 如何对其进行编码 - 以便我们可以在单个请求中运行所有内容 - 在 python 中..以便所有内容都出现在数据帧中 - 以逗号分隔的值

我认为，要将多个请求与不同的 ISO3166-1 国家/地区代码连接起来并在 Python 中的单个请求中运行它们，我们需要构建一个循环来迭代上述国家/地区代码，相应地修改请求，然后合并结果生成一个完整且单一的 DataFrame。 运行这样的请求库来发出 HTTP 请求并使用 pandas 来处理数据将适合完成此任务：

import requests
import pandas as pd
from io import StringIO

# List of ISO3166-1 country codes
country_codes = ["NL", "DE", "AT", "CH", "FR"]  # Add more country codes as needed

# Base request template
base_request = """
[out:csv(::id,::type,"name","addr:postcode","addr:city","addr:street","addr:housenumber","website"," contact:email=*")][timeout:600];
area["ISO3166-1"="{}"]->.a;
( node(area.a)[amenity=childcare];
  way(area.a)[amenity=childcare];
  rel(area.a)[amenity=childcare];);
out;
"""

# List to store individual DataFrames
dfs = []

# Loop through each country code
for code in country_codes:
    # Construct the request for the current country
    request = base_request.format(code)
    
    # Send the request to the Overpass API
    response = requests.post("https://overpass-api.de/api/interpreter", data=request)
    
    # Check if the request was successful
    if response.status_code == 200:
        # Parse the response as CSV and convert it to DataFrame
        try:
            df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
        except pd.errors.ParserError as e:
            print(f"Error parsing CSV data for {code}: {e}")
            continue
        
        # Add country code as a new column
        df['country_code'] = code
        
        # Append the DataFrame to the list
        dfs.append(df)
    else:
        print(f"Error retrieving data for {code}")

# Merge all DataFrames into a single DataFrame
result_df = pd.concat(dfs, ignore_index=True)

# Save the DataFrame to a CSV file or perform further processing
result_df.to_csv("merged_childcare_data.csv", index=False)

我在 Google-Colab 上运行这个：

嗯 - 我想实现这个代码：

a.获取 Country_codes - 即它包含我们要查询的国家/地区的 ISO3166-1 国家/地区代码。 b.基本请求；这应该是 Overpass API 请求的基本模板，并带有对应国家/地区代码的占位符 {}。

循环：循环应该迭代每个国家/地区代码，使用当前国家/地区代码修改基本请求，然后发送请求，最后将响应解析为 DataFrame，并将其附加到 dfs 列表。

最后我希望大家做一件事：dfs 中的所有 DataFrame 应该连接成一个 DataFrame result_df，然后我们可以将其保存到 CSV 文件或根据需要进一步处理。

但是好吧 - 目前我遇到了一些错误 - 我在 google-colab 上得到了这些错误。看这里

<ipython-input-3-67ee61d1e734>:33: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.


  df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
Skipping line 337: expected 1 fields, saw 2
Skipping line 827: expected 1 fields, saw 2

<ipython-input-3-67ee61d1e734>:33: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.


  df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
Skipping line 27: expected 1 fields, saw 2
Skipping line 132: expected 1 fields, saw 2
Skipping line 366: expected 1 fields, saw 2
Skipping line 539: expected 1 fields, saw 2
Skipping line 633: expected 1 fields, saw 2
Skipping line 881: expected 1 fields, saw 2
Skipping line 1394: expected 1 fields, saw 2
Skipping line 1472: expected 1 fields, saw 2
Skipping line 1555: expected 1 fields, saw 4
Skipping line 1580: expected 1 fields, saw 2
Skipping line 1630: expected 1 fields, saw 2
Skipping line 1649: expected 1 fields, saw 2
Skipping line 1766: expected 1 fields, saw 2
Skipping line 1843: expected 1 fields, saw 2
Skipping line 2067: expected 1 fields, saw 2
Skipping line 2208: expected 1 fields, saw 2
Skipping line 2349: expected 1 fields, saw 3
Skipping line 2414: expected 1 fields, saw 2
Skipping line 2419: expected 1 fields, saw 2
Skipping line 2423: expected 1 fields, saw 2
Skipping line 2464: expected 1 fields, saw 2
Skipping line 2515: expected 1 fields, saw 2
Skipping line 2581: expected 1 fields, saw 2
Skipping line 2855: expected 1 fields, saw 2
Skipping line 2899: expected 1 fields, saw 2
Skipping line 2950: expected 1 fields, saw 2

<ipython-input-3-67ee61d1e734>:33: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.


  df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
<ipython-input-3-67ee61d1e734>:33: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.


  df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
Skipping line 114: expected 1 fields, saw 2
Skipping line 212: expected 1 fields, saw 2
Skipping line 339: expected 1 fields, saw 2
Skipping line 340: expected 1 fields, saw 4
Skipping line 351: expected 1 fields, saw 3
Skipping line 357: expected 1 fields, saw 2
Skipping line 359: expected 1 fields, saw 3
Skipping line 510: expected 1 fields, saw 6
Skipping line 535: expected 1 fields, saw 2
Skipping line 546: expected 1 fields, saw 3
Skipping line 590: expected 1 fields, saw 4
Skipping line 596: expected 1 fields, saw 4
Skipping line 602: expected 1 fields, saw 3
Skipping line 659: expected 1 fields, saw 3
Skipping line 764: expected 1 fields, saw 2
Skipping line 836: expected 1 fields, saw 2
Skipping line 838: expected 1 fields, saw 2

<ipython-input-3-67ee61d1e734>:33: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.


  df = pd.read_csv(StringIO(response.text), error_bad_lines=False)
Skipping line 50: expected 1 fields, saw 3
Skipping line 302: expected 1 fields, saw 2
Skipping line 303: expected 1 fields, saw 2
Skipping line 740: expected 1 fields, saw 2
Skipping line 758: expected 1 fields, saw 2
Skipping line 1440: expected 1 fields, saw 2
Skipping line 1476: expected 1 fields, saw 3
Skipping line 1680: expected 1 fields, saw 3
Skipping line 1687: expected 1 fields, saw 2
Skipping line 1954: expected 1 fields, saw 3
python pandas dataframe request openstreetmap
1个回答
0
投票

嗨亲爱的ouroboros1

由于您的帮助，我被鼓励继续下去

import requests
import pandas as pd
from io import StringIO

# List of ISO3166-1 country codes
country_codes = ["NL", "DE", "AT", "CH", "FR"]  # Add more country codes as needed

# Base request template
base_request = """
[out:csv(::id,::type,"name","addr:postcode","addr:city","addr:street","addr:housenumber","website"," contact:email=*")][timeout:600];
area["ISO3166-1"="{}"]->.a;
( node(area.a)[amenity=childcare];
  way(area.a)[amenity=childcare];
  rel(area.a)[amenity=childcare];);
out;
"""

# List to store individual DataFrames
dfs = []

# Loop through each country code
for code in country_codes:
    # Construct the request for the current country
    request = base_request.format(code)
    
    # Send the request to the Overpass API
    response = requests.post("https://overpass-api.de/api/interpreter", data=request)
    
    # Check if the request was successful
    if response.status_code == 200:
        # Parse the response as CSV and convert it to DataFrame
        try:
            df = pd.read_csv(StringIO(response.text), sep='\t')
        except pd.errors.ParserError as e:
            print(f"Error parsing CSV data for {code}: {e}")
            continue
        
        # Add country code as a new column
        df['country_code'] = code
        
        # Append the DataFrame to the list
        dfs.append(df)
    else:
        print(f"Error retrieving data for {code}")

# Merge all DataFrames into a single DataFrame
result_df = pd.concat(dfs, ignore_index=True)

# Save the DataFrame to a CSV file or perform further processing
result_df.to_csv("merged_childcare_data.csv", index=False)

解决了问题 取回 570 kb 数据 -

非常感谢

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.