Hadoop 2.4.1 搭建Ha遇到问题记录
15/06/19 17:12:44 ERROR namenode.FSNamesystem: FSNamesystem initialization failed. java.io.IOException: Invalid configuration: a shared edits dir must not be specified if HA is not enabled. // 明明我配置了HA，却说没有,下面的配置写错了 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem.java:710) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem. (FSNamesystem.java:654) at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424) 15/06/19 17:12:44 INFO namenode.FSNamesystem: Stopping services started for active state 15/06/19 17:12:44 FATAL namenode.NameNode: Exception in namenode join java.lang.NullPointerException at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.stopActiveServices(FSNamesystem.java:1118) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.close(FSNamesystem.java:1446) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem. (FSNamesystem.java:781) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem. (FSNamesystem.java:654) at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424) 15/06/19 17:12:44 INFO util.ExitUtil: Exiting with status 1 15/06/19 17:12:44 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at v1/192.168.32.91 ************************************************************/
.写成了-正确如下： dfs.ha.automatic-failover-enabled true dfs.ha.automatic-failover.enabled true
2.journalnode的端口配置错误，导致不能格式化namenode,journalnode监听的端口是：8485，而不是8445.
3.格式化standby namnenode时，有两种方式
1.将active namenode格式化完成后，将整个hadoop.tmp.dir路径全部拷贝到standby机器上。
2.在active namenode格式化完成后，并且格式化zkfc(只需要在active上格式化一次，standby上的zkfc不需要再次格式化)启动好了namenode之后。 在standby namenode上执行:hdfs namenode -bootstrapStandby 也可以.如果active namenode没有启动的话，这里会报错。因为standby
namenode是通过active namenode的9000端口通讯的。若active namenode没有启动，则9000没有程序监听提供服务，standby尝试连接9次后会爆异常并且退出。
namenode是通过active namenode的9000端口通讯的。若active namenode没有启动，则9000没有程序监听提供服务，standby尝试连接9次后会爆异常并且退出。
没有启动active namenode时，报错如下：
[root@v2 hadoop]# hdfs namenode -bootstrapStandby 15/06/19 19:28:45 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = v2/192.168.32.92 STARTUP_MSG: args = [-bootstrapStandby] STARTUP_MSG: version = 2.4.1 STARTUP_MSG: classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z STARTUP_MSG: java = 1.7.0_15 ************************************************************/ 15/06/19 19:28:45 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 15/06/19 19:28:45 INFO namenode.NameNode: createNameNode [-bootstrapStandby] 15/06/19 19:28:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 15/06/19 19:28:49 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:50 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:51 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:52 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:53 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:54 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:55 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:56 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:57 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:58 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 15/06/19 19:28:58 FATAL ha.BootstrapStandby: Unable to fetch namespace information from active NN at v1/192.168.32.91:9000: Call From v2/192.168.32.92 to v1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused 15/06/19 19:28:58 INFO util.ExitUtil: Exiting with status 2 15/06/19 19:28:58 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92 ************************************************************/
启动了active如下：
[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:31:39 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = v2/192.168.32.92
STARTUP_MSG: args = [-bootstrapStandby]
STARTUP_MSG: version = 2.4.1
STARTUP_MSG: classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z
STARTUP_MSG: java = 1.7.0_15
************************************************************/
15/06/19 19:31:39 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/06/19 19:31:39 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
15/06/19 19:31:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
=====================================================
About to bootstrap Standby ID bnn2 from:
Nameservice ID: bns1
Other Namenode ID: bnn1
Other NN's HTTP address: http://v1:50070
Other NN's IPC address: v1/192.168.32.91:9000
Namespace ID: 32897138
Block pool ID: BP-134496651-192.168.32.91-1434751114027
Cluster ID: CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
Layout version: -56
=====================================================
15/06/19 19:31:44 INFO common.Storage: Storage directory /root/hadoop/tmp/dfs/name has been successfully formatted.
15/06/19 19:31:45 INFO namenode.TransferFsImage: Opening connection to http://v1:50070/imagetransfer?getimage=1&txid=0&storageInfo=-56:32897138:0:CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
15/06/19 19:31:46 INFO namenode.TransferFsImage: Image Transfer timeout configured to 60000 milliseconds
15/06/19 19:31:46 INFO namenode.TransferFsImage: Transfer took 0.01s at 0.00 KB/s
15/06/19 19:31:46 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 350 bytes.
15/06/19 19:31:46 INFO util.ExitUtil: Exiting with status 0
15/06/19 19:31:46 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92
************************************************************/
4.hbase启动时，如果不同节点时间不同的话，启动只有HMaster会起来，别的HRegionServer都会挂掉。同步时间即可，HBase需要用到hadoop的{core-site.xml/hdfs-site.xml}，否则会将配置好的ha hdfsname（hdfs://bns1/hbase）当成hostname去解析，把hdfs相关文件拷贝到hbase/conf下就能正常运行了。
5.hive配置元数据快记法：
4个配置项前缀：javax.jdo.option.Connection[URL/UserName/Password/DriverName]
createDatabaseIfNotExist
配置mysql允许远程连接：
#(执行下面的语句 *.*:所有库下的所有表 %：任何IP地址或主机都可以连接)
GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘123’ WITH GRANT OPTION;
FLUSH PRIVILEGES;
如果这么配置完了之后，可以远程登录但是本机登录不了的时候。可以用mysqladmin给本地重新修改下密码
/usr/bin/mysqladmin -uroot -h currentHostName password ‘new-password’
版权声明：本文为博主原创文章，未经博主允许不得转载。