読者です 読者をやめる 読者になる 読者になる

【mesos:3】mesosのクラスタ化

はじめに

殆ど、情報がなかった。多分、rpmでインストールしたせいと思う。
巷にあふれてる、mesos.confは、間違いなので、きをつけること。
尚、まだまだ、変わる気がする。。。

一応、動いていると思うけど、まだ、検証不足な状態。

環境

物理サーバ 3台 (x86_64)

Centos 6.4 (6.2から、yum updateで、バージョンアップ)

mesos_0.14

  • 1台 master + slave
  • 2台 slave

Open JDK 6

CDH4.2.1

  • 1台 namenode + second namenode + JobTracker + datanode
  • 2台 datanode
  • replica 1

Client 端末から、サーバの名前解決可能。*1

インストール

  1. master のIPを設定する。*2 (2台にコピー)
  2. slave のIP設定をする。(2台にコピー)
  3. slave,masterの環境設定をする。(2台にコピー)

インストールログ

vi /usr/local/var/mesos/deploy/masters
[root@uldata14 ~]# cat /usr/local/var/mesos/deploy/masters
10.29.254.14
vi /usr/local/var/mesos/deploy/slaves
[root@uldata14 ~]# cat /usr/local/var/mesos/deploy/slaves 
10.29.254.14
10.29.254.15
10.29.254.16
[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/masters root@10.29.254.15:/usr/local/var/mesos/deploy/masters
masters                                                                                       100%   13     0.0KB/s   00:00    
[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/slaves root@10.29.254.15:/usr/local/var/mesos/deploy/slaves
slaves                                                                                        100%   26     0.0KB/s   00:00    
[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/masters root@10.29.254.16:/usr/local/var/mesos/deploy/masters
masters                                                                                       100%   13     0.0KB/s   00:00    
[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/slaves root@10.29.254.16:/usr/local/var/mesos/deploy/slaves
slaves                                                                                        100%   26     0.0KB/s   00:00    
[root@uldata14 ~]# cp /usr/local/var/mesos/deploy/mesos-slave-env.sh.template /usr/local/var/mesos/deploy/mesos-slave-env.sh
[root@uldata14 ~]# cat /usr/local/var/mesos/deploy/mesos-slave-env.sh
# This file contains environment variables that are passed to mesos-slave.
# To get a description of all options run mesos-slave --help; any option
# supported as a command-line option is also supported as an environment
# variable.

# You must at least set MESOS_master.

# The mesos master URL to contact. Should be host:port for
# non-ZooKeeper based masters, otherwise a zk:// or file:// URL.
export MESOS_master=10.29.254.14:5050

export MESOS_hadoop_home=/usr/lib/hadoop

# Other options you're likely to want to set:
export MESOS_log_dir=/var/log/mesos
export MESOS_work_dir=/var/run/mesos
# export MESOS_isolation=cgroups

[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/mesos-slave-env.sh root@10.29.254.15:/usr/local/var/mesos/deploy/mesos-slave-env.sh
[root@uldata14 ~]# scp /usr/local/var/mesos/deploy/mesos-slave-env.sh root@10.29.254.16:/usr/local/var/mesos/deploy/mesos-slave-env.sh
[root@uldata14 ~]# vi /usr/local/var/mesos/deploy/mesos-master-env.sh
[root@uldata14 ~]# cat /usr/local/var/mesos/deploy/mesos-master-env.sh# This file contains environment variables that are passed to mesos-master.
# To get a description of all options run mesos-master --help; any option
# supported as a command-line option is also supported as an environment
# variable.

# Some options you're likely to want to set:
export MESOS_log_dir=/var/log/mesos

試し

mesos cluster start

[root@uldata14 ~]# mesos-start-cluster.sh

[root@uldata14 ~]# ps aux | grep mesos-master
root     17594  0.0  0.0 1393800 13764 ?       Sl   Oct03   0:39 /usr/local/sbin/mesos-master
root     22204  0.0  0.0 107456   936 pts/1    S+   14:25   0:00 grep mesos-master
[root@uldata14 ~]# ps aux | grep mesos-slave
root     17633  0.0  0.0 1393632 11620 ?       Sl   Oct03   0:15 /usr/local/sbin/mesos-slave
root     22206  0.0  0.0 107456   940 pts/1    S+   14:25   0:00 grep mesos-slave

[root@uldata15 ~]# ps auxw | grep /usr/local/sbin/mesos-slave
root      9425  0.0  0.0 1393632 11648 ?       Sl   Oct03   0:13 /usr/local/sbin/mesos-slave
root     10715  0.0  0.0 107456   944 pts/0    S+   14:25   0:00 grep /usr/local/sbin/mesos-slave

[root@psdata16 ~]# ps aux | grep /usr/local/sbin/mesos-slave
root     10645  0.0  0.0 1393632 11636 ?       Sl   Oct03   0:13 /usr/local/sbin/mesos-slave
root     12168  0.0  0.0 107456   944 pts/0    S+   14:26   0:00 grep /usr/local/sbin/mesos-slave

Hadoop を、開始 *3

[root@uldata14 ~]# su - hadoop
[hadoop@uldata14 ~]$ /usr/lib/hadoop/bin/start-hadoop-pssh.sh 
/usr/lib/hadoop/bin/start-hadoop-pssh.sh: line 4: /usr/lib/hadoop/bin/hadoop-config.sh: そのようなファイルやディレクトリはありません
Starting Hadoop namenode:                                  [  OK  ]
starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-uldata14.hadoop.local.out
Connection to uldata14.hadoop.local closed.
Starting Hadoop secondarynamenode:                         [  OK  ]
starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-uldata14.hadoop.local.out
Connection to uldata14.hadoop.local closed.
Starting Hadoop jobtracker daemon (hadoop-jobtracker): starting jobtracker, logging to /var/log/hadoop-0.20-mapreduce/hadoop-hadoop-jobtracker-uldata14.hadoop.local.out
Connection to uldata14.hadoop.local closed.                [  OK  ]
[1] 14:35:13 [SUCCESS] uldata14.hadoop.local
Starting Hadoop datanode:[  OK  ]
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-uldata14.hadoop.local.out
[2] 14:35:13 [SUCCESS] uldata15.hadoop.local
Starting Hadoop datanode:[  OK  ]
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-uldata15.hadoop.local.out
[3] 14:35:13 [SUCCESS] psdata16.hadoop.local
Starting Hadoop datanode:[  OK  ]
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-psdata16.hadoop.local.out
[1] 14:35:15 [SUCCESS] uldata14.hadoop.local
Starting Hadoop tasktracker daemon (hadoop-tasktracker): starting tasktracker, logging to /var/log/hadoop-0.20-mapreduce/hadoop-hadoop-tasktracker-uldata14.hadoop.local.out
[2] 14:35:15 [SUCCESS] uldata15.hadoop.local
Starting Hadoop tasktracker daemon (hadoop-tasktracker): starting tasktracker, logging to /var/log/hadoop-0.20-mapreduce/hadoop-hadoop-tasktracker-uldata15.hadoop.local.out
[3] 14:35:15 [SUCCESS] psdata16.hadoop.local
Starting Hadoop tasktracker daemon (hadoop-tasktracker): starting tasktracker, logging to /var/log/hadoop-0.20-mapreduce/hadoop-hadoop-tasktracker-psdata16.hadoop.local.out


Hadoop のJob をスタート

[hadoop@uldata14 ~]$ hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 3 100

WebUI の説明は、ここ。Mesos - Cluster & Framework Mgmt

*1:hostsなり、dnsなりで、各サーバの名前解決可能にしておかないと、WebUIがうまく動かない。

*2:ZKは、使わない

*3:自前のスクリプト