単一ノード配置Hadoopチュートリアル

HDFSの構築

ホスト名を追加

マスターホスト名を追加しただけです

[root@10 /xinghl/hadoop/bin]$ cat /etc/hosts
127.0.0.1   localhost 10.0.67.101
::1         localhost 10.0.67.101
10.0.67.101 master    10.0.67.101

リモートクラスタを構成する場合は、SSHを構成する必要があります。

私のところは単一ノードテストで遊んでいるので、この手順は先に省略しました.

hadoopを/usrディレクトリに解凍

必要なディレクトリをいくつか作成

mkdir /dfs
mkdir /dfs/name
mkdir /dfs/data
mkdir /tmp

プロファイルを変更し、$HADOOP_HOME/etc/hadoop下

変更sh

export JAVA_HOME=/usr/java

slavesの変更

ここにいるよ

localhost

core-siteを修正します。xml

       
                fs.defaultFS
                hdfs://master:8020
       
       
                io.file.buffer.size
                131072
        
       
               hadoop.tmp.dir
               file:/usr/hadoop/tmp
               Abase for other temporary   directories.
       
        
               hadoop.proxyuser.u0.hosts
               *
       
       
               hadoop.proxyuser.u0.groups
               *
       

hdfs-siteを変更します。xml

       
                dfs.namenode.secondary.http-address
               master:9001
       
     
             dfs.namenode.name.dir
             file:/usr/hadoop/dfs/name
       
      
              dfs.datanode.data.dir
              file:/usr/hadoop/dfs/data
       
       
               dfs.replication
               3
        
        
                 dfs.webhdfs.enabled
                  true
         

hadoopの起動

hadoop namenode -format # $HADOOP_HOME/bin 
start-all.sh # $HADOOP_HOME/sbin 

運転状態の確認

1 jpsコマンドを使用して2ログインを表示するhttp://10.0.67.101:8088/cluster

SSHを設定し、パスワードなしのログインをサポートする

[root@localhost sbin]# cd ~/.ssh/
[root@localhost .ssh]# ll
total 0
[root@localhost .ssh]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
75:ea:4f:ec:df:55:06:3c:af:91:12:50:69:a5:86:27 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|          ...o.  |
|           ooo   |
|          E.* +  |
|         . * . = |
|        S . . o +|
|         . . . +.|
|          . o . .|
|           +   ..|
|            o.. .|
+-----------------+
[root@localhost .ssh]# 
[root@localhost .ssh]# ll
total 8
-rw-------. 1 root root 1675 Aug 16 10:55 id_rsa
-rw-r--r--. 1 root root  408 Aug 16 10:55 id_rsa.pub
[root@localhost .ssh]# cat id_rsa.pub >> ~/.ssh/authorized_keys
[root@localhost .ssh]# ll
total 12
-rw-r--r--. 1 root root  408 Aug 16 10:55 authorized_keys
-rw-------. 1 root root 1675 Aug 16 10:55 id_rsa
-rw-r--r--. 1 root root  408 Aug 16 10:55 id_rsa.pub
[root@localhost .ssh]# ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 9e:7b:cb:c5:06:e9:81:e5:db:57:9c:f9:79:a2:c4:ce.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
[root@localhost ~]# exit
logout
Connection to localhost closed.
[root@localhost .ssh]# ssh localhost
Last login: Tue Aug 16 10:55:30 2016 from localhost
[root@localhost ~]# exit
logout
Connection to localhost closed.
[root@localhost .ssh]# 

posted @
2016-07-28 09:49 xingoo読書(
...) コメント(
...) コレクションの編集