hadoop3.0中使用纠删码

参考 :如何在CDH6.0中使用纠删码

https://cloud.tencent.com/developer/article/1363393

.纠删码是以目录为单位的,一旦某个纠删码策略被启用后,你就可以为某个目录设置策略

  测试代码

hadoop fs -mkdir /ec_xor
hdfs ec -getPolicy -path /ec_xor
hdfs ec -enablePolicy -policy XOR-2-1-1024k
hdfs ec -setPolicy -path /ec_xor -policy XOR-2-1-1024k
hdfs ec -getPolicy -path /ec_xor
  hdfs ec -setPolicy -path /ec_xor -policy RS-10-4-1024k
hdfs ec -enablePolicy -policy RS-6-3-1024k
hdfs ec -setPolicy -path /ec_xor -policy RS-6-3-1024k
ll
vi ec_test.txt
hadoop fs -put ec_test.txt /ec_xor
hdfs ec -setPolicy -path /ec_xor_s -policy XOR-2-1-1024k
hdfs ec -setPolicy -path /ec_xor_m -policy XOR-2-1-1024k
hdfs ec -setPolicy -path /ec_xor_l -policy XOR-2-1-1024k
ll
hadoop fs -put mysql-connector-java-8.0.12-1.el7.noarch.rpm /ec_xor
hadoop fs -put mysql-connector-java-8.0.12-1.el7.noarch.rpm /ec_xor_s
hadoop fs -put mysql-connector-java-8.0.12-1.el7.noarch.rpm /ec_xor_m
hadoop fs -put mysql-connector-java-8.0.12-1.el7.noarch.rpm /ec_xor_l
hadoop fs -put Downloads.zip /ec_xor_s
hadoop fs -put  /ec_xor_s
hadoop fs -put Downloads.zip /ec_xor_l
hdfs fsck /ec_xor_s -files -blocks -locations
rz -E
hadoop fs -rmr /ec_xor_s/
hadoop fs -mkdir /ec_xor_s/
hdfs ec -setPolicy -path /ec_xor_s -policy XOR-2-1-1024k
hadoop fs -ls /ec_xor_m/D
hadoop fs -ls /ec_xor_m/
hadoop fs -rm -r /ec_xor_m/Downloads.zip
hadoop fs -rm -r /ec_xor_l/mysql-connector-java-8.0.12-1.el7.noarch.rpm
ll
hadoop fs -put MTracer.zip /ec_xor_s
hdfs fsck /ec_xor_s -files -blocks -locations
hdfs fsck /ec_xor_m -files -blocks -locations
hdfs fsck /ec_xor_l -files -blocks -locations
hdfs ec -getPolicy -path /tmp
hdfs ec -getPolicy -path /
hdfs ec -getPolicy -path /ec_xor
hadoop fs -ls /
hadoop fs -ls /user
hdfs ec -setPolicy -path /user/zhangsan
hdfs ec -getPolicy -path /user/zhangsan
hdfs ec -getPolicy -path /user/yarn
hdfs ec -getPolicy -path /user/zhangsan -policy XOR-2-1-1024k
hdfs ec -setPolicy -path /user/zhangsan -policy XOR-2-1-102
hdfs ec -getPolicy -path /user/yarn
hdfs ec -getPolicy -path /user/zhangsan
hadoop fs -ls  /user/zhangsan
hadoop fs -ls  /ec_xor_l
hadoop fs -du -h   /ec_xor_l

您可以选择一种方式赞助本站

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

图片 表情