Understanding HDFS Recovery Processes (Part 1) - Cloudera Blog
blog.cloudera.com/understanding-hdfs-recovery-processes-part-1/
1 Users
0 Comments
29 Highlights
0 Notes
Top Highlights
UNDER_CONSTRUCTION state when the corresponding client’s lease expires, then it will be changed to UNDER_RECOVERY state when block recovery starts
a lease, which is essentially a lock
HBase’s Region Server (RS) writes to its WAL (Write Ahead Log), which is an HDFS file that helps to prevent data loss
In HDFS, files are divided into blocks, and file access follows multi-reader, single-writer semantics
To meet the fault-tolerance requirement, multiple replicas of a block are stored on different DataNodes
replication factor
This ensures the single-writer semantics
lease recovery
Before lease recovery causes the file to be closed, it’s necessary to ensure that all replicas of the last block have the same length
block recovery
FINALIZED
RWR
RUR
TEMPORARY
DataNode context
NameNode context
UNDER_CONSTRUCTION
UNDER_RECOVERY
COMMITTED
COMPLETE
Ready to highlight and find good content?
Glasp is a social web highlighter that people can highlight and organize quotes and thoughts from the web, and access other like-minded people’s learning.