在HDFS中,提供了fsck命令,用于检查HDFS上文件和目录的健康状态、获取文件的block信息和位置信息等。fsck命令必须由HDFS超级用户来执行,普通用户无权限。
查看hdfs fsck命令参数
[root@master ~]# hdfs fsck
Usage: hdfs fsck [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-includeSnapshots] [-storagepolicies] [-blockId ]
start checking from this path
-move move corrupted files to /lost+found
-delete delete corrupted files
-files print out files being checked
-openforwrite print out files opened for write
-includeSnapshots include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it
-list-corruptfileblocks print out list of missing blocks and files they belong to
-blocks print out block report
-locations print out locations for every block
-racks print out network topology for data-node locations
-storagepolicies print out storage policy summary for the blocks
-blockId print out which file this blockId belongs to, locations (nodes, racks) of this block, and other diagnostics info (under replicated, corrupted or not, etc)
查看文件中损坏的块(-list-corruptfileblocks)
hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -list-corruptfileblocks
将损坏的文件移动至/lost+found目录(-move)
hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/part-00168 -move
删除损坏的文件(-delete)
hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/part-00168 -delete
检查并列出所有文件状态(-files)
hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -files
检查并打印正在被打开执行写操作的文件(-openforwrite)
hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -openforwrite
打印文件的Block报告(-blocks)
需要和-files一起使用
hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks
打印文件块的位置信息(-locations)
需要和-files -blocks一起使用
hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks -locations
打印文件块位置所在的机架信息(-racks)
hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks -locations -racks