hdfs fsck命令操作查看Block文件块信息

2019-10-08 16:02:23 | 编辑

在HDFS中,提供了fsck命令,用于检查HDFS上文件和目录的健康状态、获取文件的block信息和位置信息等。fsck命令必须由HDFS超级用户来执行,普通用户无权限。

查看hdfs fsck命令参数

[root@master ~]# hdfs fsck
Usage: hdfs fsck  [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-includeSnapshots] [-storagepolicies] [-blockId ]
		start checking from this path
	-move	move corrupted files to /lost+found
	-delete	delete corrupted files
	-files	print out files being checked
	-openforwrite	print out files opened for write
	-includeSnapshots	include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it
	-list-corruptfileblocks	print out list of missing blocks and files they belong to
	-blocks	print out block report
	-locations	print out locations for every block
	-racks	print out network topology for data-node locations
	-storagepolicies	print out storage policy summary for the blocks
	-blockId	print out which file this blockId belongs to, locations (nodes, racks) of this block, and other diagnostics info (under replicated, corrupted or not, etc)

 

查看文件中损坏的块(-list-corruptfileblocks)

 hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -list-corruptfileblocks

 

将损坏的文件移动至/lost+found目录(-move)

hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/part-00168 -move

 

删除损坏的文件(-delete)

hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/part-00168 -delete

 

检查并列出所有文件状态(-files)

hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -files

 

检查并打印正在被打开执行写操作的文件(-openforwrite)

hdfs fsck /hivedata/warehouse/liuxiaowen.db/lxw_product_names/ -openforwrite

 

打印文件的Block报告(-blocks)

需要和-files一起使用

hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks

 

打印文件块的位置信息(-locations)

需要和-files -blocks一起使用

hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks -locations

  

打印文件块位置所在的机架信息(-racks)

hdfs fsck /logs/site/2015-08-08/lxw1234.log -files -blocks -locations -racks

 

登录后即可回复 登录 | 注册
    
关注编程学问公众号