Filter a string on the basis of a word

hadoop apache-pig
learner picture learner · Sep 16, 2011 · Viewed 49.8k times · Source

I have a pig job where in I need to filter the data by finding a word in it,

Here is the snippet

A = LOAD '/home/user/filename' USING PigStorage(',');
B = FOREACH A GENERATE $27,$38;
C = FILTER B BY ( $1 ==  '*Word*');
STORE C INTO '/home/user/out1' USING PigStorage();

The error is in the 3rd line while finding C, I have also tried using

C = FILTER B BY $1 MATCHES '*WORD*'

Also

C = FILTER B BY $1 MATCHES '\\w+WORD\\w+'

Answer

Donald Miner picture Donald Miner · Sep 17, 2011

MATCHES uses regular expressions. You should do ... MATCHES '.*WORD.*' instead.

These is an example here finding the word 'apache'.

Related questions

Difference between Pig and Hive? Why have both?

My background - 4 weeks old in the Hadoop world. Dabbled a bit in Hive, Pig and Hadoop using Cloudera's Hadoop VM. Have read Google's paper on Map-Reduce and GFS (PDF link). I understand that- Pig's language Pig Latin is a …

Read More
When to use Hadoop, HBase, Hive and Pig?

What are the benefits of using either Hadoop or HBase or Hive ? From my understanding, HBase avoids using map-reduce and has a column oriented storage on top of HDFS. Hive is a sql-like interface for Hadoop and HBase. I would …

Read More
PIG how to count a number of rows in alias

I did something like this to count the number of rows in an alias in PIG: logs = LOAD 'log' logs_w_one = foreach logs generate 1 as one; logs_group = group logs_w_one all; logs_count = foreach logs_group generate …

Read More