我有一个带模式的表用户表
|Column 1 | USER ID |int|
|Column 2 |EMAIL|chararray|
|Column 3 |LANGUAGE |chararray|
|Column 4 |LOCATION |chararray|
和具有模式的事务表
|Column 1 | ID |int|
|Column 2 |PRODUCT|int|
|Column 3 |USER ID |int|
|Column 4 |PURCHASE AMOUNT |double|
|Coulmn 5 |DESCRIPTION |chararray|
是...在不同的位置查找每种产品的数量。
我写了如下的猪脚本:-
user = LOAD '/tmp/users.txt' USING PigStorage (' ')
AS (USER_ID:int, EMAIL:chararray, LANGUAGE:chararray, LOCATION:chararray);
transaction = LOAD '/tmp/transaction.txt' USING PigStorage (' ')
AS (ID:int, PRODUCT:int,USER_ID:int, PURCHASE_AMOUNT:double,DESCRIPTION:chararray);
u1 = JOIN user by USER_ID, transaction by USER_ID;
u2 = GROUP u1 by LOCATION;
Result = FOREACH u2 GENERATE COUNT(u2.PRODUCT);
DUMP Result;
错误---错误org.apache.pig.tools.grunt.Grunt-错误1200:Pig脚本无法解析:标量投影无效:u2
这就是我要得到的。
在您的脚本中Result = FOREACH u2 GENERATE COUNT(u2.PRODUCT);
是错误的。分组操作后,数据结构发生变化。您可以用describe u2
找到它。请尝试以下操作(我假设字段在txt文件中用逗号分隔):
user = LOAD 'user.txt' USING PigStorage (',') AS (USER_ID:int, EMAIL:chararray, LANGUAGE:chararray, LOCATION:chararray);
transaction = LOAD 'transaction.txt' USING PigStorage (',') AS (ID:int, PRODUCT:int,USER_ID:int, PURCHASE_AMOUNT:double,DESCRIPTION:chararray);
u1 = JOIN user by USER_ID, transaction by USER_ID;
u2 = GROUP u1 by (LOCATION,PRODUCT);
Result = FOREACH u2 GENERATE FLATTEN(group) as (LOCATION,PRODUCT), COUNT($1);
DUMP Result;