Recovery is the process of restoring a federated database to a consistent state after a transaction fails to commit. Depending on the nature of the failure, recovery is performed by the application that started the transaction or through one of several automatic mechanisms that you enable. There is also a manual recovery utility - oocleanup.
Whenever a transaction starts, it records update information in one or more journal files, which are used to return the federated database to its previously committed state if the transaction is aborted or terminated abnormally. Journal files enable Infinite Graph to roll back changes made by incomplete transactions; they differ from relational-database transaction logs, which usually contain before and after images of the data and allow forward recovery up to the last commit.
Journal files (.JNL extension) are written in a federated database’s journal directory. Every single-threaded application that updates the federated database creates a journal file in the journal directory. This journal file is automatically reinitialized at the end of each committed transaction and is reused by the application’s next transaction. A multithreaded application normally creates multiple journal files in the journal directory - one for each thread that executes a separate series of update transactions. As an administrator, you may change a federated database’s journal directory via the ooinstallfd command line tool.
Infinite Graph deletes journal files automatically when a process completes or when recovery is performed after a process or system failure. Never delete a journal file, because data corruption may result.
You can enable lock server to perform automatic recovery of failed transactions. Its recovery monitor is a thread within a standard lock server that identifies and recovers incomplete transactions belonging either to:
- Processes that ran on client hosts that are now no longer operational.
- Processes that have been disconnected from the lock server, although their client hosts are still operational.
The recovery monitor checks whether client hosts are operational by attempting to make a network connection to each client host and then waiting for a response within a specified timeout period. If a client host does not respond for any reason (for example, due to machine or network problems), it is assumed to have failed, and all transactions started by its processes are automatically recovered.
The recovery monitor is enabled by starting the lockserver process in the following manner:
In rare circumstances, Infinite Graph may not be able to recover a federated database automatically. In these situations, you can recover a federated database manually using the oocleanup utility. There are several options that you may specify, however the most common option is the -local flag.
The oocleanup tool uses journal files to roll back the uncommitted changes made by incomplete transactions. If the lock server is still running, oocleanup also releases the locks held by the incomplete transactions.
Further information on the oocleanup options can be found here.