hadoop構築チュートリアルの5(Pigインストール)

1、解凍とインストール
解圧pig-0.10.0.tar.gzから/opt/ディレクトリの下:

tar -zxvf pig-0.10.0.tar.gz -C /opt/

mv /opt/pig-0.10.0 /opt/pig

chown -R hadoop:hadoop /opt/pig

su hadoop

2、構成/opt/pig/bin/pigファイル
pigファイルを開く
# # The Pig command script # # Environment Variables # #     JAVA_HOME                The java implementation to use.    Overrides JAVA_HOME. # #     PIG_CLASSPATH Extra Java CLASSPATH entries. # #     HADOOP_HOME/HADOOP_PREFIX     Environment HADOOP_HOME/HADOOP_PREFIX(0.20.205) # #     HADOOP_CONF_DIR     Hadoop conf dir # #     PIG_HEAPSIZE    The maximum amount of heap to use, in MB. #                                        Default is 1000. # #     PIG_OPTS            Extra Java runtime options. # #     PIG_CONF_DIR    Alternate conf dir. Default is ${PIG_HOME}/conf. # #     HBASE_CONF_DIR-Optionally,the HBAse configuration to run against#上のコードの下で構成を開始します.

export JAVA_HOME=/usr/java/jdk/

export PIG_INSTALL=/opt/pig

export HADOOP_INSTALL=/opt/hadoop

export PATH=$PIG_INSTALL/bin:%HADOOP_INSTALL/bin:$PATH

export PIG_CLASSPATH=$HADOOP_INSTALL/conf

3、実行
次のコマンドを使用して実行します.

cd /opt/hadoop/bin

./haoop fs -copyFromLocal /opt/data/test.txt /opt/data/test.txt

cd /opt/pig/bin

./pig

例:ユーザ名を取り出し、dist.txtに存在する

A = LOAD '/opt/data/test.txt' USING PigStorage('\t') AS (id,name);

dump A;

B = FOREACH A GENERATE name;

STORE B INTO '/opt/data/dist.txt' USING PigStorage(); 

cd /opt/hadoop/bin

./hadoop fs -ls /opt/data

./hadoop fs -ls /opt/data/dist.txt

./hadoop fs -cat /opt/data/dist/txt/part-m-00000

Pig Latin共通コマンド:

LOAD ...... USING PigStorage('') ...... AS ......;

FOREACH ...... GENERATE ......;

FILTER ...... BY ......;

DUMP;

STORE ...... INTO;

GROUP ...... BY;

AND OR