我有两个数据帧,df1
是场所的位置,df2
是站的位置。我正在尝试找到一种更有效的方法来应用距离函数,以查找在特定范围内的站点并返回站点的名称。如果距离函数是+/- 1
的纬度差,这是我的预期结果:
# df1
Lat Long
0 30 31
1 37 48
2 54 62
3 67 63
# df2
Station_Lat Station_Long Station
0 30 32 ABC
1 43 48 DEF
2 84 87 GHI
3 67 62 JKL
# ....Some Code that compares df1 and df2....
# result
Lat Long Station_Lat Station_Long Station
30 31 30 32 ABC
67 63 67 62 JKL
我有一个使用cartesian product /交叉联接的功能,可以在单个DataFrame上应用函数。此解决方案有效,但是我在一个真实的数据集中有数百万行,这使笛卡尔积非常慢。
import pandas as pd
df1 = pd.DataFrame({'Lat' : [30, 37, 54, 67],
'Long' : [31, 48, 62, 63]})
df2 = pd.DataFrame({'Station_Lat' : [30, 43, 84, 67],
'Station_Long' : [32, 48, 87, 62],
'Station':['ABC', 'DEF','GHI','JKL']})
# creating a 'key' for a cartesian product
df1['key'] = 1
df2['key'] = 1
# Creating the cartesian Join
df3 = pd.merge(df1, df2, on='key')
# some distance function that returns True or False
# assuming the distance function I want is +/- 1 of two values
def some_distance_func(x,y):
return x-y >= -1 and x-y <= 1
# applying the function to a column using vectorized approach
# https://stackoverflow.com/questions/52673285/performance-of-pandas-apply-vs-np-vectorize-to-create-new-column-from-existing-c
df3['t_or_f'] = list(map(some_distance_func,df3['Lat'],df3['Station_Lat']))
# result
print(df3.loc[df3['t_or_f']][['Lat','Long','Station_Lat','Station_Long','Station']].reset_index(drop=True))
我也尝试过使用iterrows()
的循环方法,但是比交叉联接方法慢。有没有更pythonic /更有效的方法来实现我想要的?
也许更快一些:
iterrows()
排序后,您可以使用'searchsorted“:
df2= df2.sort_values("Station_Lat")
“ idx”是“最近”电台lat。索引或idx + 1就是这个。也许您需要复制df2的最后一行(请参阅“搜索后的文档”),以避免对其过度索引。通过此自定义功能使用“应用”:
df1["idx"]=df2.Station_Lat.searchsorted(df1.Lat)
lambda怎么样?
def dist(row):
if abs(row.Lat-df2.loc[row.idx].Station_Lat)<=1:
return df2.loc[row.idx].Station
elif abs(row.Lat-df2.loc[row.idx+1].Station_Lat)<=1:
return df2.loc[row.idx+1].Station
return False
df1.apply(dist,axis=1)
0 ABC
1 False
2 False
3 JKL
dtype: object
输出:
df3[df3.apply(lambda x, col1='Lat', col2='Station_Lat': x[col1]-x[col2] >= -1 and x[col1]-x[col2] <= 1, axis=1)]['Station']