我正在尝试在Impala(SQL)中创建一个从镶木地板表中获取行的表。数据代表城市中的自行车骑行次数。如果起始代码(字符串,例如:“ 6100”)在第一个表中显示超过100次,则行将被导入到新表中。这是我到目前为止的内容:
#I am using Apache Impala via the Hue Editor
invalidate metadata;
set compression_codec=none;
invalidate metadata;
Set compression_codec=gzip;
create table bixirides_parquet (
start_date string, start_station_code string,
end_date string, end_station_code string,
duration_sec int, is_member int)
stored as parquet;
Insert overwrite table bixirides_parquet select * from bixirides_avro;
invalidate metadata;
set compression_codec=none;
create table impala_out stored as textfile as select start_date, start_station_code, end_date, end_station_code, duration_sec, is_member, count(start_station_code) as count
from bixirides_parquet
having count(start_station_code)>100;
出于某种原因,该语句将运行,但是新表中没有插入任何行。如果该行的起始代码在原始表中显示超过100次,则应将行导入到新表中。我想我的选择语句用词不正确,但我不确定准确度如何。
我认为您想要的最终查询是:
select start_date, start_station_code, end_date,
end_station_code, duration_sec, is_member, cnt
from (select bp.*,
count(*) over (partition by start_station_code) as cnt
from bixirides_parquet bp
) bp
where cnt > 100;