impala 中的一个 HDFS 扫描严重倾斜

问题描述 投票:0回答:0

其中一个 HDFS (01) 扫描严重倾斜,impala 只分配了 9 台主机。我能以任何方式解决这个问题吗?

Operator          #Hosts   Avg Time   Max Time   #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                                                                                   
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE            1  329.041us  329.041us  11.59K          -1    4.75 MB              0  UNPARTITIONED                                                                            
02:HASH JOIN          19   11.944ms   65.503ms  11.59K          -1   34.10 MB        2.00 GB  RIGHT OUTER JOIN, PARTITIONED                                                            
|--04:EXCHANGE        19  303.506us  467.216us  34.21K          -1  432.00 KB              0  HASH(state_vectors_data4.lat,state_vectors_data4.lastcontact,state_vectors_data4.icao24) 
|  01:SCAN HDFS        9  615.482ms    1s916ms  34.21K          -1  126.49 MB        1.38 GB  opensky.state_vectors_data4                                                              
03:EXCHANGE           19  277.767us  406.210us  65.33K          -1    1.14 MB              0  HASH(position_data4.lat,position_data4.maxtime,position_data4.icao24)                    
00:SCAN HDFS          19    1s358ms    1s822ms  65.33K          -1  248.28 MB        1.80 GB  opensky.position_data4
impala
© www.soinside.com 2019 - 2024. All rights reserved.