RTG transaction support

FairCom RTG supports complete transaction processing for your existing COBOL applications. Of the three levels of transaction processing supported by the underlying FairCom DB technology, FairCom RTG creates files using our middle level of transaction processing known as "Prelmage". This means it is transaction-ready and supports ROLLBACK out-of-the-box without recovery. This is compatible with most existing COBOL platforms.

This concept guide explains FairCom RTG transaction handling and how you can fine tune it for the best possible performance.

Transaction Control - Why you should want it

Transaction control provides two extraordinarily powerful tools for your applications - atomicity of operations, and recovery of data. There are four key components of any complete transactional system, collectively referred to as ACID:

• Atomicity - Grouping of database update operations into all-or-nothing commits;
• Consistency - The database remains in a valid state due to any transaction;
• Isolation - Concurrent transactions do not interfere with each other outside of well-defined behaviors;
• Durability - Committed transactions are guaranteed to be recoverable, even if the data and index files are inconsistent.

FairCom RTG is fully ACID compliant and as such, FairCom RTG provides the highest levels of transaction integrity.

Some differences exist in the transaction handling support provided by ACUCOBOL, Micro Focus, isCOBOL, and RM/COBOL:

*

In ACUCOBOL a transaction is explicitly started using a START TRANSACTION directive.

**

An ACUCOBOL program allows rollback of updates only if a file has been defined as WITH ROLLBACK.

The FairCom RTG files are created PREIMG by default therefore they are transaction ready and support ROLLBACK out-of-the-box. There is no need to enable <transaction logging> to use ROLLBACK.

Enabling RTG transaction control

By default, all tables created by FairCom RTG are “transaction ready”, meaning they support default COBOL rollback capability. However, this model of transaction control does not provide recoverability in the event of a system failure. For example, if hardware power is cut to the computer running the FairCom RTG server process, data can be left in an inconsistent state on a disk, requiring time consuming and costly table rebuilds.

Easily enable full transaction control with FairCom DB from your existing ctree.conf configuration file. This can be applied to single selected tables, or to all tables using wildcards.

<transaction logging=”yes”>yes</transaction>

https://docs.faircom.com/doc/ctcobol/_transaction_.htm

For more info, see <transaction>. 

We recommend also including deferautocommit="yes" for enhanced performance.

Switching transaction modes

It is convenient to occasionally opt not to use full transaction control for performance reasons. For example, during bulk loading data and large batch processes that have well defined rollback states such as a complete data backup before starting.

It is easy to switch FairCom RTG tables into non-transaction mode and back. There are two utilities that can be considered:.

ctutil -tron

cttrnmod

Always remember: if you switch a table to a non-transaction mode, your table is now vulnerable to corruption in the event of a failed system. You should always ensure the table is restored back to full transaction control as soon as possible after usage. ctutil -info and the cttrnmod utility can both report back the current transaction state of a table.

Automatic Recovery

Transaction control is intended primarily for protection of data. System failures can occur during processing, which can gravely corrupt valuable data. These include everything from a software or OS failure to hardware failure, including power loss. Frequently, COBOL applications must rebuild all tables after such an event to recover as much data as possible and to bring their indexes back to a consistent state with the data.

When FairCom DB is started, one of the very first tasks it processes is to detect and determine if it was properly shut down from the previous run. If not, it undergoes a process called automatic recovery.

During recovery, the FairCom RTG server examines the current state of the transaction log and begins scanning and rebuilding all transactions up to and including the last committed transaction to the database. Once complete, all data and index files are in a clean consistent state ready for immediate operations by the application. This process is usually very quick and results in very little downtime for an application and the organization.

When the application crashes while instructions are being processed, the application will detect that a problem has occurred the next time it starts up. The transaction management logic will then automatically reset the database back to the last, most complete state that it can using the transaction log. Transactions committed before the crash will be redone, while those not yet committed will be undone. Only files which include a full transaction mode with logging enabled are able to be recovered.

Occasionally, tables do not remain on disk when recovery runs. Recovery stops as it can’t recover the referenced tables in the transaction logs. If it is ok to do so,FairCom RTG can skip the recovery of those tables. Only use this option when you absolutely know you need it.

There are options to improve recovery performance for very large environments.

Transaction control and performance

Transaction control comes at a performance cost. Fortunately, FairCom RTG transaction performance is tuned and configured out of the box for the most general best case usage.

Transaction performance is a balance of I/O and memory. Every transaction commit must be secured to persistent storage. This is done with a synchronous write to disk, which is the most expensive I/O write as it bypasses the filesystem cache. However, it is an absolute hard requirement to guarantee data recovery.

This write performance is balanced with holding data in memory caching as long as possible to maintain immediate access to updated record contents, avoiding additional I/O to disk. Periodically,updated “dirty” data is written back to the data and index files in the background to maintain the highest performance possible by the FairCom DB server.

Several transaction processing options are used to tune this balancing act:

CHECKPOINT_INTERVAL

CHECKPOINT_FLUSH

COMPATIBILITY TINDEX_WRITETHRU

Tip: Disable COMMIT_DELAY (-1) for most modern SSD storage devices. This is a legacy configuration that provides additional performance for older “spinning” type drives. It will be deprecated in the near future.

But I still want better performance!

If you are willing to open a minimal window of vulnerability, delayed durability may be an option for you. It ensures that transaction commits remain atomic and write to the transaction log as usual. However, it only goes to the transaction log in filesystem cache. It is not an immediate synchronous write to persistent storage. In this mode of operation a background thread is utilized to regularly flush and sync the filesystem cache to disk, ensuring your updates are secured. This can improve performance up to 300%!

The default (and recommended) time interval of this flush is once per second. Other values have not shown better performance and open wider vulnerability windows. In a high volume transaction writes, or with large transactions, the log buffer is often flushed and synchronized to disk immediately regardless, further minimizing vulnerabilities.

Transaction log files and best practices

Transaction control depends on a set of “write ahead” logs persisted on reliable storage devices. These should be located on your fasted available storage mounts. Modern SAN SSD is ideal. Frequently, it is necessary to retain as much transaction log data as possible for extended rollback or roll forward. Replicated environments may not be available, requiring the server to retain more logs than defaults allow. Options are available to ensure as much transaction log data is retained as possible.

Ultimately, hands off! These files are critical to the operation of your environment and the integrity of your data. You should ensure that transaction logs are not accessed by any other process, including virus scanners, etc. They should never be backed up, or otherwise copied over.

In fact, if for any reason FairCom RTG cannot write to these critical transaction logs, it will immediately terminate as it can no longer ensure the integrity of your data. A common situation where this happens is an out of storage space condition. Ensure there is always plenty of space for the transaction logs on the storage mount.