Generating all fields from an alias after a JOIN in Pig

duckworthd picture duckworthd · May 31, 2012 · Viewed 12k times · Source

I would like to perform the equivalent of "keep all a in A where a.field == b.field for some b in B" in Apache Pig. I am implementing it like so,

AB_joined = JOIN A by field, B by field;
A2 = FOREACH AB_joined GENERATE A::field as field, A::field2 as field2, A::field3 as field3;

Enumerating all of A's entries is quite silly, and I would rather do something like,

A2 = FOREACH AB_joined GENERATE flatten(A);

However, this doesn't seem to work. Is there some other way I can do something equivalent without enumerating A's fields?

Answer

Sateesh picture Sateesh · Aug 29, 2012

This should work:

A2 = FOREACH AB_joined GENERATE $0..