Big Data module 2: Question on PIG operator FOREACH...GENERATE

odesh — Wed, 21 Aug 2019 15:56:50 GMT

Hello,

Please see the attachment on PIG relating to an example on the operator. FOREACH...GENERATE:

The syntax says: "alias = FOREACH{block|nested block} ..

I do not see the combination of block and nested block in the example in the attachment..

There are just 2 fields in the example : county and population. Why of these is the nested block ?

Thanks.

ODesh.

Re: Big Data module 2: Question on PIG operator FOREACH...GENERATE

Cynthia_sas — Thu, 22 Aug 2019 13:31:41 GMT

Hi:
Answer from instructors:

=== beginnning of comment ===

The screen shot is showing just the accepted syntax that is appropriate for the FOREACH operator .. so you can use a block with columns as shown in the example OR you can use a nested block, which the example does NOT show. Here is an example of a next FOREACH block. You will notice that is leveraging additional PIG operators and keywords to generate the desired columns.

The nested blocks is similar to writing nested subqueries in SQL.

result = foreach grpd_cstr{

total_orders = SUM(grpd_cstr_itm_cnt.itmCnt);

srtd_orders = order grpd_cstr_itm_cnt by itmCnt desc;

higest_bought = limit srtd_orders 1;

generate FLATTEN(higest_bought),total_orders as totalCnt;

};

=== end of comment ===

=== a useful link ===

https://pig.apache.org/docs/r0.17.0/basic.html Pig syntax guide

=== === ===

Hope this clarifies your question,

Cynthia

topic Big Data module 2: Question on PIG operator FOREACH...GENERATE in SAS Academy for Data Science

Big Data module 2: Question on PIG operator FOREACH...GENERATE

Re: Big Data module 2: Question on PIG operator FOREACH...GENERATE