Cube buildでFileNotFoundExceptionが表示されます

問題の背景

筆者の最近の好奇心からApache Kylinを研究し始めたのは、環境のインストールの過程で多くの奇妙な問題に遭遇し、これはその中の1つであり、記録してビットに共有した.もしあなたも同じように出会ったら、あなたに役に立つことを望んでいます.

環境依存

Hadoop2.7.3

HBase1.2.3

Hive 2.1.1

Kylin 1.6.0-HBase1.x

問題の説明

各種環境変数を配置した後、kylinは正常に起動した.その後modelとcubeを構成し、データのbuildプロセスを開始します.プログラム実行後にFileNotFoundExceptionの異常が発生したことを発見

java.io.FileNotFoundException: File does not exist: hdfs://polaris-1:8020/tmp/hadoop-root/mapred/staging/root1659475561/.staging/job_local1659475561_0004/libjars/hive-metastore-2.1.1.jar
  at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
  at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
  at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
  at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
  at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
  at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
  at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:99)
  at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
  at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
  at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
  at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
  at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
  at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
  at java.security.AccessController.doPrivileged(Native Method)

このようなエラーが発生して、とても気がふさいで、インターネットを利用して多くの后で依然として问题の原因を探し当てていないで、そこでコードをダウンロードして自分で研究します;上記の異常の原因はhive-metastoreのパッケージが見つからないことです.依存するのはHadoop 2ですから.7.3、スタック表示コードに従って、対応する行が見つからないことを発見する.このとき、バージョン依存エラーによる疑いがあります.jinfoコマンドでkylin依存classpathを表示する

/usr/local/hbase/conf:/usr/local/java//lib/tools.jar:/usr/local/hbase:/usr/local/hbase/lib/activation-1.1.jar:/usr/local/hbase/lib/aopalliance-1.0.jar:/usr/local/hbase/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hbase/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hbase/lib/api-util-1.0.0-M20.jar:/usr/local/hbase/lib/asm-3.1.jar:/usr/local/hbase/lib/avro-1.7.4.jar:/usr/local/hbase/lib/commons-beanutils-1.7.0.jar:/usr/local/hbase/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hbase/lib/commons-cli-1.2.jar:/usr/local/hbase/lib/commons-codec-1.9.jar:/usr/local/hbase/lib/commons-collections-3.2.2.jar:/usr/local/hbase/lib/commons-compress-1.4.1.jar:/usr/local/hbase/lib/commons-configuration-1.6.jar:/usr/local/hbase/lib/commons-daemon-1.0.13.jar:/usr/local/hbase/lib/commons-digester-1.8.jar:/usr/local/hbase/lib/commons-el-1.0.jar:/usr/local/hbase/lib/commons-httpclient-3.1.jar:/usr/local/hbase/lib/commons-io-2.4.jar:/usr/local/hbase/lib/commons-lang-2.6.jar:/usr/local/hbase/lib/commons-logging-1.2.jar:/usr/local/hbase/lib/commons-math-2.2.jar:/usr/local/hbase/lib/commons-math3-3.1.1.jar:/usr/local/hbase/lib/commons-net-3.1.jar:/usr/local/hbase/lib/disruptor-3.3.0.jar:/usr/local/hbase/lib/findbugs-annotations-1.3.9-1.jar:/usr/local/hbase/lib/guava-12.0.1.jar:/usr/local/hbase/lib/guice-3.0.jar:/usr/local/hbase/lib/guice-servlet-3.0.jar:/usr/local/hbase/lib/hadoop-annotations-2.5.1.jar:/usr/local/hbase/lib/hadoop-auth-2.5.1.jar:/usr/local/hbase/lib/hadoop-client-2.5.1.jar:/usr/local/hbase/lib/hadoop-common-2.5.1.jar:/usr/local/hbase/lib/hadoop-hdfs-2.5.1.jar:/usr/local/hbase/lib/hadoop-mapreduce-client-app-2.5.1.jar:/usr/local/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/usr/local/hbase/lib/hadoop-mapreduce-client-core-2.5.1.jar:/usr/local/hbase/lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/usr/local/hbase/lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/usr/local/hbase/lib/hadoop-yarn-api-2.5.1.jar:/usr/local/hbase/lib/hadoop-yarn-client-2.5.1.jar:/usr/local/hbase/lib/hadoop-yarn-common-2.5.1.jar:/usr/local/hbase/lib/hadoop-yarn-server-common-2.5.1.jar:/usr/local/hbase/lib/hbase-annotations-1.2.3.jar:/usr/local/hbase/lib/hbase-annotations-1.2.3-tests.jar:/usr/local/hbase/lib/hbase-client-1.2.3.jar:/usr/local/hbase/lib/hbase-common-1.2.3.jar:/usr/local/hbase/lib/hbase-common-1.2.3-tests.jar:/usr/local/hbase/lib/hbase-examples-1.2.3.jar:/usr/local/hbase/lib/hbase-external-blockcache-1.2.3.jar:/usr/local/hbase/lib/hbase-hadoop2-compat-1.2.3.jar:/usr/local/hbase/lib/hbase-hadoop-compat-1.2.3.jar:/usr/local/hbase/lib/hbase-it-1.2.3.jar:/usr/local/hbase/lib/hbase-it-1.2.3-tests.jar:/usr/local/hbase/lib/hbase-prefix-tree-1.2.3.jar:/usr/local/hbase/lib/hbase-procedure-1.2.3.jar:/usr/local/hbase/lib/hbase-protocol-1.2.3.jar:/usr/local/hbase/lib/hbase-resource-bundle-1.2.3.jar:/usr/local/hbase/lib/hbase-rest-1.2.3.jar:/usr/local/hbase/lib/hbase-server-1.2.3.jar:/usr/local/hbase/lib/hbase-server-1.2.3-tests.jar:/usr/local/hbase/lib/hbase-shell-1.2.3.jar:/usr/local/hbase/lib/hbase-thrift-1.2.3.jar:/usr/local/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/local/hbase/lib/httpclient-4.2.5.jar:/usr/local/hbase/lib/httpcore-4.4.1.jar:/usr/local/hbase/lib/jackson-core-asl-1.9.13.jar:/usr/local/hbase/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hbase/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hbase/lib/jackson-xc-1.9.13.jar:/usr/local/hbase/lib/jamon-runtime-2.4.1.jar:/usr/local/hbase/lib/jasper-compiler-5.5.23.jar:/usr/local/hbase/lib/jasper-runtime-5.5.23.jar:/usr/local/hbase/lib/javax.inject-1.jar:/usr/local/hbase/lib/java-xmlbuilder-0.4.jar:/usr/local/hbase/lib/jaxb-api-2.2.2.jar:/usr/local/hbase/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hbase/lib/jcodings-1.0.8.jar:/usr/local/hbase/lib/jersey-client-1.9.jar:/usr/local/hbase/lib/jersey-core-1.9.jar:/usr/local/hbase/lib/jersey-guice-1.9.jar:/usr/local/hbase/lib/jersey-json-1.9.jar:/usr/local/hbase/lib/jersey-server-1.9.jar:/usr/local/hbase/lib/jets3t-0.9.0.jar:/usr/local/hbase/lib/jettison-1.3.3.jar:/usr/local/hbase/lib/jetty-6.1.26.jar:/usr/local/hbase/lib/jetty-sslengine-6.1.26.jar:/usr/local/hbase/lib/jetty-util-6.1.26.jar:/usr/local/hbase/lib/joni-2.1.2.jar:/usr/local/hbase/lib/jruby-complete-1.6.8.jar:/usr/local/hbase/lib/jsch-0.1.42.jar:/usr/local/hbase/lib/jsp-2.1-6.1.14.jar:/usr/local/hbase/lib/jsp-api-2.1-6.1.14.jar:/usr/local/hbase/lib/junit-4.12.jar:/usr/local/hbase/lib/leveldbjni-all-1.8.jar:/usr/local/hbase/lib/libthrift-0.9.3.jar:/usr/local/hbase/lib/log4j-1.2.17.jar:/usr/local/hbase/lib/metrics-core-2.2.0.jar:/usr/local/hbase/lib/netty-all-4.0.23.Final.jar:/usr/local/hbase/lib/paranamer-2.3.jar:/usr/local/hbase/lib/protobuf-java-2.5.0.jar:/usr/local/hbase/lib/servlet-api-2.5-6.1.14.jar:/usr/local/hbase/lib/servlet-api-2.5.jar:/usr/local/hbase/lib/slf4j-api-1.7.7.jar:/usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hbase/lib/snappy-java-1.0.4.1.jar:/usr/local/hbase/lib/spymemcached-2.11.6.jar:/usr/local/hbase/lib/xmlenc-0.52.jar:/usr/local/hbase/lib/xz-1.0.jar:/usr/local/hbase/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/contrib/capacity-scheduler/*.jar

やはり、kylinはhbase classpathの内容に依存しており、hadoop 2が含まれている.5.1の関連パッケージは、関連する依存を取り除く.問題が解決される.