GeistHaus
log in · sign up

Testing Lucene's index durability after crash or power loss

blog.mikemccandless.com

One of Lucene's useful transactional features is index durability which ensures that, once you successfully call IndexWriter.commit , even...

1 page links to this URL
Corruption detection in Lucene and Elasticsearch

100% reliable data storage is fundamentally impossible, but we can get pretty close with layers of protection against something going wrong at the physical level. Databases are typically agnostic to the specific protections that any given installation is using and mostly just assume that the data they read from disk is the data they wrote there previously. The protections might be in the filesystem itself but it’s more usual to push them down to the lower layers of RAID controllers and drive firmware. However it is impossible to truly guarantee protection against silent corruption and this happens often enough that many databases add their own mechanisms for detecting and correcting those rare cases where the data that they read isn’t the data that they previously wrote.

0 inbound links article en CC BY-SA 4.0