开源日志收集系统Scribe学习笔记(二) 各种问题解决

     开始使用Scribe去收集日志,慢慢的会碰到很多问题，这里整理下.

  

    1.scribed需要将数据写到HDFS时所报错误

"[hdfs] ERROR: HDFS is not configured for file: hdfs://master:9000/scribe/default/namenote"
"[hdfs] ERROR: HDFS is not configured for file: hdfs://master:9000/scribe/default/namenote/default-2012-09-06_00000"
"[default] Failed to open file <hdfs://master:9000/scribe/default/namenote/default-2012-09-06_00000> for writing"

     这个问题一般是因为CLASSPATH没有设置所导致,只要在启动scribed之前设置好CLASSPATH即可.

     同时可以在scribed启动时看到如下报错:

     Environment variable CLASSPATH not set!

     Environment variable CLASSPATH not set!

     Environment variable CLASSPATH not set!

     ...

     这里在启动scribe前用如下命令设置下CLASSPATH即可:

     export CLASSPATH=`$HADOOP_HOME/bin/hadoop classpath`

 

       还有一种情况就是直接报

      default] Failed to open file <hdfs://master:9000/scribe/default/namenote/default-2012-09-06_00000> for writing

      没有之前的ERROR: HDFS is not configured for file: hdfs://master:9000/scribe/default/namenote.通常这个是因为权限导致.也就是你启动scirbe和启动hadoop的用户必须是同一个，否则scirbe将没有权限往HDFS里写东西,则会报这个错误

 

 

   2.启动Scribe时,所报错误:

scribed: error while loading shared libraries: libboost_filesystem.so.1.45.0:
cannot open shared object file: No such file or directory

     这个是不能正确加载boost所导致,我们需要去检查下/usr/lib或/usr/lib64/目录下有没有libboost_filesystem.so.1.45.0这个文件.如果你的操作系统是64位,可以根据命令getconf LONG_BIT查看,那就需要将libboost_filesystem.so.1.45.0放到/usr/lib6/目录下即可.至于libboost_filesystem.so.1.45.0文件在哪里?可以通过命令sudo find / -name libboost_filesystem.so.1.45.0查找下即可,找到后再通过软连接过去.

     引起这个原因大部分还是操作系统是64位,而boost经默认安装libboost_filesystem.so.1.45.0被放到/usr/local/lib/目录下,所以我们需要软连接过去即可,命令:sudo ln -s /usr/local/lib/libboost_system.so.1.45.0 /usr/lib64/libboost_system.so.1.45.0

     类似的错误还有,thrift,libhdfs等,都可以用这种办法解决,类似错误:

scribed: error while loading shared libraries:libboost_filesystem.so.1.45.0: cannot open shared object file: No such file or directory
scribed: error while loading shared libraries: libthrift.so.0: cannot open shared object file: No such file or directory
scribed: error while loading shared libraries: libthriftnb.so.0: cannot open shared object file: No such file or directory
scribed: error while loading shared libraries: libhdfs.so.0: cannot open shared object file: No such file or directory
scribed: error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory

 

    3.NonblockingServer.h:34:19: error: event.h: No such file or directory

     错误原因是,没有安装libevent-devel,通过命令:yum install libevent-devel安装.

    如果没有事先安装libevent-devel而安装thrift，则会有可能还会导致:

    /usr/bin/ld: cannot find -lthriftnb 错误.如此要安装好libevent-devel再安装thrift

 

  4.启动Scribe时报:terminate called after throwing an instance of 'std::runtime_error'

  what():  locale::facet::_S_create_c_locale name not valid 错误.

   错误原因，环境关系。通过执行以下命令再启动scirbe即可:

   export LC_ALL="en_US.utf8"

   或者将其写入配置文件，如.bash_profile

 

更多技术文章、感悟、分享、勾搭，请用微信扫描: