This page collects some information about Derby's interaction with various system file systems, and specifically about the assumptions that Derby makes about the services that the file system provides. Much of this information was posted to the derby-dev mailing list in November, 2010 by Mike Matrigali and Kathey Marsden.

We have always told users they have to have their databases on a local disk to ensure data integrity and that a system crash for an NFS mounted database could cause fatal corruption, but it would be better if we could explain exactly why.

We see the problem as one of documentation and implementation of NFS. There isn't just one "NFS" out there. And there are definitely all sorts of other remote mounting options.

Some of the problems that can arise, that are avoided in local disk configurations, include:

  1. We may not be able to prevent dual booting and thus db may get corrupted. All of our algorithms for preventing dual booting rely on the JVMs that are accessing the database to be on the same machine. Once 2 machines can access the same file we have no way to prevent corruption.
  2. Derby depends on synchonous write behavior when requested. Basically at certain times Derby asks the JVM to guarantee that data to a table or recovery log file has been written and forced to disk before returning. If this syncing is not correct a number of database problems can happen such as:
    1. we tell user a transaction was commited because we believe the log was forced, but the NFS was caching the result and crashes. Now the committed xact is not there.
    2. we want to remove some recovery log so we force data to disk, wait for it to hit disk and the delete the log file for those disk updates. But data is actually cached and lost and now we have old data in the db and no log files to recover it from.

When Derby was first written, we didn't believe any JVM implementation on top of NFS could guarantee a completed synchronous write. It may be the case that certain remote file system implementations now can guarantee this, and it may be the case that the JVM implementations make the right calls to the NFS file system to do this - but we believe it is a support nightmare to try and support this.

A quick google of NFS topics seems to indicate that there may be some versions of nfs that do support write sync. We notice that most of the hits were descriptions of how to disable the syncing to get better performance, indicating that many of NFS that might support write sync actually have it disabled. We have not been able to uncover a way that a Java program could find out if the required syncing was being enforced.

Note that we also can not guarantee recovery on disks with write cache enabled, which we think many users have set. Many may not even know it as it appears to be the default for some disk installations.

  • No labels