ISAM for C Operational Models

This chapter describes the various FairCom DB operational models, how to move files between operational models, and how to change the operational model for your application. FairCom DB installation is described in Chapter 1 “Installation” of the Quick Start and Product Overview Guide, along with steps for integrating your application with FairCom DB and helpful hints for getting started.

FairCom DB can be configured to operate in a variety of ways, which are referred to as operational models. The features that vary from one model to another are single-user versus multi-user access, support of a database server, support of higher level functions such as transaction processing and file mirroring, and availability of source code. This section provides a brief overview of the available models.

All configurations have been designed with the developer in mind, allowing you to switch models by simply re-compiling with FairCom’s m-tree make system.

The supported operational models are:

1. Standard Client/Server Model - client/server.
2. Standalone Single-User Model.
3. Standalone Multi-User Model.
4. LOCLIB Model - client/server model with client side local library support.
5. Multi-Threaded Standalone Model - Standalone Multi-user multi-threading support.
6. CTUSER Server Model - add user functionality to the Server side.
7. Bound Server Model - bind the FairCom Server engine into your application
8. Custom Server Model - add your own logic to the FairCom Server.

Standard Client/Server Model

The standard client/server model supports the complete FairCom DB API in the form of inter-process communication requests made to the FairCom Server. Instead of linking with a FairCom DB engine performing direct file I/O, the application is linked with a FairCom DB communication library making network requests to the FairCom Server. Select the desired communication protocol for your application when building the FairCom DB client-side library.

A wide selection of protocols is available, including TCP/IP, IPX/SPX, Shared Memory, and a number of proprietary protocols. No network programming experience is required. In most cases, converting from stand-alone model to client/server is as simple as re-linking the FairCom DB application. Heterogeneous support allows clients of various architectures to talk to a FairCom Server at the same time, making this model a strong network database solution.

Benefits

• All disk I/O is done by the FairCom Server, taking full advantage of data/index caches and minimizing network traffic for maximum performance.
• Client libraries resolve the complete FairCom DB API, allowing standalone applications to move to client/server seamlessly.
• Transaction processing with full roll-forward, rollback, auto-recovery and unique transaction history feature.
• Heterogeneous support: For example, SPARC/Intel clients can share data across any combination of client/server platforms. Connect any client to any Server.
• Scalability. Large number of supported platforms.
• Complete client-side library source code included in the FairCom DB package. Easy platform migration.
• Heavily dynamic indexes are automatically maintained for optimal speed and size, eliminating the need for routine rebuilding.

Consideration

• Consider migrating applications using the FairCom DB low-level API to the FairCom DB ISAM API. The ISAM API requires less network traffic and includes batch operations to reduce network traffic, boosting performance.

Standalone Single-User Model

The Standalone Single-User Model is part of FairCom’s non-client/server architecture offering single-user capabilities, meaning only one process may open a given file.

With the Standalone Single-user Model, your application is linked with the FairCom DB database engine that supplies a Function API in the form of a static library, DLL, or shared library. All file related operations are done through the FairCom DB API, which ultimately utilizes your compiler’s runtime library routines, (i.e., open, close, read, write, etc.). The file I/O, or disk I/O is cached to improve performance.

Applications designed to have only one user accessing files at a time can use this non-server single user mode. All of the capabilities of FairCom DB are supported, with the exception of file security control and encryption. The FairCom DB code will be linked as a part of your application. The developer can specify the amount of memory to be used for file buffering, which can have a significant effect on the speed of the system. Full source code is included, which allows the developer to move the product to any operating environment that supports the C language. The caching offered by the single-user model provides better throughput than the multi-user model.

This model provides an easy migration path to the client/server model with features such as: robust data concurrency, multi-user data integrity through on-line Transaction Processing (OLTP), and all of the other features offered by the client/server model.

Benefits

• Self-contained (Stand Alone) executable.
• Accelerated performance with data and index cache.
• Transaction processing support.
• Flexible API with powerful fixed and variable-length record support.
• Complete source code included for maximum control and portability.
• Compile as a static library, DLL, or Unix Shared Library.
• Mac libraries support Carbon API

Considerations

• No support for multi-user file access.
• Extra network traffic when accessing a network file server, especially with large files.

Standalone Multi-User Model

The Standalone Multi-User Model is part of FairCom’s non-client/server architecture offering multi-user peer-to-peer capabilities, meaning multiple processes may open the same file.

With the Standalone Multi-User Model, your application is linked with the FairCom DB database engine that supplies a Function API in the form of a static library, DLL, or shared library. All file related operations are done through the FairCom DB API, which ultimately utilizes your compiler’s runtime library routines, (i.e., open, close, read, write, etc.). The file I/O, or disk I/O is “flushed and force read” to and from the hard disk to ensure data integrity between all users.

The Standalone Multi-User Model offers record-level locking using the compiler’s runtime library lock/unlock functions for multiple-user data concurrency, even on a network file system. The FairCom DB code will be linked as a part of your application. Full source code is included, which allows the user to move the product to any operating environment that supports the C language.

This model provides an easy migration path to the client/server model with features such as: more robust data concurrency, multi-user data integrity through on-line transaction processing (OLTP), and all of the other features offered by the client/server model.

Benefits

• Self-contained (Stand Alone) executable.
• Flexible API with powerful fixed and variable-length record support.
• Complete source code included for maximum control and portability.
• Compile as a static library, DLL, or Unix Shared Library.
• Mac libraries support Carbon API

Considerations

• No support for transaction processing.
• Extra network traffic when accessing a network file server, especially with large files.
• Locking may be limited by the underlying operating system or network software.
• No intelligence or centralized security on the file server machine.
• No multi-user data & index cache capabilities to enhance performance.

Note! The FairCom Server minimizes or eliminates ALL OF THE ABOVE!

LOCLIB Model

Applications using the FairCom DB Local Library (LOCLIB) model can access local data directly from the hard disk like the Stand Alone model and logon as a client to one or more FairCom Servers, as in the client/server model. This model is especially popular on PC-based systems where speed and efficiency is gained by using the local drive in lieu of taking the network hit to access a FairCom Server. Temporary work files, extracted FairCom Server data, local database information, and special “on the fly” index files are typically candidates for local disk access.

A LOCLIB library is a merger of the FairCom DB Stand Alone file I/O logic and FairCom DB client communication logic into the same library. Applications linked with the combined library have the best of both worlds, being able to access a local disk and one or more FairCom Servers. The FairCom DB database function API is the same for both channels of I/O. Applications simply invoke a “switch” function call to direct the focus of the next operation. An application may log on to any number of FairCom Servers at the same time.

Benefits

• Provides fast local data storage and retrieval without adding extra network traffic.
• Complete source code for LOCLIB Library included in your FairCom DB package.
• Configure local disk access as single-user, single-user transaction processing, or multi-user access. Allows an easy upgrade path to FairCom Server operation. Simply move the data under the FairCom Server and change connections to upgrade. See FairCom Server for migration information.
• Applications can access data locally and/or log on to any number of FairCom Servers at the same time. Both stand-alone and client/server models are supported for the same application.
• With heterogeneous support, this model provides a unique way to port data between different architectures. An application may read data from one platform and write it to another. The communication layer handles all necessary byte flipping.
• Applications may “register” as many “c-tree” access channels to different databases as needed.
• Applications may instigate “switch c-tree” calls to indicate which data source to direct the I/O.

Consideration

• On multi-tasking operating systems, consider running a FairCom Server on the local machine instead of using the LOCLIB model. Although the LOCLIB model supports single-user transaction control, it does not support multi-user transaction control over the local data. Multi-user transaction control is supported by running a local FairCom Server.

Multi-Threaded Standalone Model

FairCom DB provides complete thread-safe client libraries for your multi-threaded projects. While the FairCom DB threading API can be used on its own, it provides a strong complement to the multi-thread client-side database support. These two advanced technologies provide sophisticated support for your multi-thread client applications. Options within the make system provide the necessary prompts to create a thread-safe library with your specified communications protocol.

FairCom authored its own Thread Manager/Scheduler that has been used for years in the FairCom Server. We did this because many operating systems, including most Unix systems, did not support native threads. Only recently have Unix vendors begun to deliver thread support. In order to provide complete multi-threaded client support to our customers, we have included this Thread Manager/Scheduler in the FairCom DB package. You can now create multi-threaded applications on all supported FairCom platforms, regardless of whether the operating system supports threads.

Benefits

• Multi-threaded clients are available on all FairCom Server supported platforms. If no native threading is available, FairCom’s proprietary Thread Manager/ Scheduler, which is included with FairCom DB, is used.
• Invoke your application to perform multiple simultaneous tasks.
• FairCom’s portable Thread API makes creating and managing threads easy.
• Unique “inter-thread” communication support within FairCom’s threaded API allows for complex thread co-operation.

Considerations

• Availability of native threads on your targeted operating system. If native threads are available, they may provide a performance boost.

CTUSER Server Model

FairCom also offers an easy method for you to expand the functionality of the FairCom Server by loading your own custom user functions. By using DLLs in the Windows environment, code resources on the Mac platform, or shared libraries in the Unix world, you can load and instigate through the CTUSER function your own logic on the FairCom Server.

The CTUSER model accepts a text buffer as its argument and passes this buffer to the FairCom Server. The FairCom Server executes the CTUSER function and loads the functionality coded into the CTUSER function stub provided to you in source code form with FairCom DB. Examples of CTUSER uses include:

1. Determine machine specifics on the FairCom Server machine, i.e., amount of hard drive space free, etc.
2. Start a cron process.
3. Instigate virtually any type of operation through C function calls.

Benefits

• All the benefits of regular client/server plus:
• CTUSER source code for stubbing in your application logic and loading it into the FairCom Server.
• The perfect hook for additional FairCom Server side operations.

Considerations

• CTUSER must consider defer logic in non-native threaded environments to avoid tying up the FairCom Server.
• Be careful not to call a system function that may not return, or may take a while to return, since the FairCom Server will not process other threads until CTUSER returns in non-native or non-preemptive environments.

Bound Server Model

The FairCom Database supports several operational models in addition to the traditional client/server model. The Server DLL model essentially allows you to compile the server into your application. This is the latest generation of the Bound Server model in which your application calls the server as a DLL. In the latest Server DLL moel, it is linked as a static library.

FairCom is pleased to provide you the ability to link your own functionality directly into FairCom’s industry proven database server with the FairCom SDK. This unique level of control sets FairCom apart from other alternatives by offering you the true power and flexibility required by your most demanding database server applications. With additional source code availability, no other database server vendor provides this level of access and control.

The Server DLL (based on the Bound Server Model introduced in earlier releases) provides the advanced engineer a proven database engine in the form of a multi-thread safe library. FairCom’s multi-threaded API and available communication logic give the serious engineer the necessary tools to create custom application-specific Servers.

This model allows you to tightly bind the FairCom Database Engine with your own application to create a powerful multi-threaded package.

Note: This model requires significant initial design efforts on the part of an experienced C programmer. Your application code implements the fundamentals of a multi-threaded FairCom Database Engine, such as threads, security, and any other requirements of your special application.

The FairCom Server DLL Development Program requires a signed distribution license prior to deployment. This model “takes the lid off” FairCom’s advanced database server and gives the developer the opportunity to use FairCom’s sophisticated database engine for custom needs. For additional information, contact your nearest FairCom office to discuss your requirements.

Benefits

• Linkable library allows you to link your own needs into the multi-thread safe FairCom Database Engine.
• Great technology to use for your web application servers. Bind web applications directly with the FairCom Database Engine and eliminate the overhead of inter-process communications.
• Supports all FairCom API functions and offers exceptional performance.

Considerations

• Requires engineering expertise to design and implement the FairCom Database Engine code.
• Requires a distribution license for production use.

Server DLL sample makefile in drivers folder

The folder named ctree.srvdll in the drivers folders provides examples of how to link with the Server DLL. This folder contains a makefile for each platform. Include folders have been added for both SQL and ISAM capabilities. The makefiles demonstrate linking to the Server DLL provided in the bin/ace/sql folder. The proper header files are provided to ensure consistency.

Function to pass FairCom DB configuration file contents in buffer

The c-tree Server DLL (shared library on Unix) supports a way of passing c-tree Server configuration options to the database engine rather than requiring a configuration file. To use this feature, before calling cThrdInit() to initialize the database engine, call ctSetConfigurationOptions() (ctSetConfigurationOptions, ctSetConfigurationOptions).

For your convenience, we have implemented this functionality in ctixmg, a c-tree sample application.

Function pass FairCom DB license options in buffer

The c-tree Server DLL (shared library on Unix) supports a way of passing c-tree Server license options to the database engine rather than requiring a license file.To use this feature, before calling cThrdInit() to initialize the database engine, call ctSetLicenseOptions() (ctSetLicenseOptions, ctSetLicenseOptions).

For your convenience we have implemented this functionality in ctixmg, an example application built in the topic titled Bound Server example - ctree.srvdll.

Custom Server Model

FairCom is pleased to provide you the ability to link your own functionality directly into FairCom’s industry proven database server with the FairCom Server SDK, included with the FairCom Developer’s CD. This unique level of control sets FairCom apart from other alternatives by offering you the true power and flexibility required by your most demanding database server applications. With additional source code availability, no other database server vendor provides this level of access and control.

The Custom Server Model provides the advanced engineer a proven database engine in the form of a multi-thread safe library. FairCom’s multi-threaded API and available communication logic give the serious engineer the necessary tools to create custom application-specific Servers.

This model allows you to move data intensive processes from the client to the server to reduce network traffic. Data intensive calculations, special sorts, filters, and reports are just a few example processes that can benefit from this model.

Note: This model requires significant initial design efforts on the part of an experienced C programmer. Instead of using the default FairCom Server Main() function, the FairCom Server’s logic can be customized to your specifications. Your FairCom Server logic implements the fundamentals of your customized FairCom Server, such as threads, communications, and any other requirements of your special Server.

The FairCom Custom Server Development Program requires a signed distribution license prior to deployment. This model “takes the lid off” FairCom’s advanced database server and gives the developer the opportunity to use FairCom’s sophisticated database engine for custom needs. For additional information, contact your nearest FairCom office to discuss your requirements.

Benefits

• FairCom Server linkable library allows you to link your own needs into the multi-thread safe FairCom Server.
• Developers may use their own communications subsystem, such as RPC.
• Great technology to use for your web application servers. Bind web applications directly with the c-tree Custom Server engine and eliminate the overhead of inter-process communications.
• Supports all FairCom DB API functions and offers exceptional performance.

Considerations

• Requires engineering expertise to design and implement the FairCom Server Main() function.
• Requires a distribution license for production use.

Operational Model Considerations

Each operational model has its strengths and weaknesses. This section examines issues that need to be considered when using each operational model. Be sure to review these issues when you are choosing the operational model that is best suited to your application.

Migrating Data Between Platforms and Operational Models

FairCom created FairCom DB with the intent to provide cross-platform support with a standardized database, no matter which operational model is used or what mix of platforms are involved. In some cases, this capability is invisible to the developer, in others careful consideration is required to allow various applications to work together.

This section provides an overview of topics covered in more detail in later sections. It is intended to provide background information to help you choose the appropriate operational model for your present and future implementations.

The considerations are broken down by operational model: Single-user standalone, Standalone Multi-user, and client/server. Applications from different models should not share files concurrently, though files can be exchanged as described in File Migration.

Single-User Standalone

Single-user applications should not share FairCom DB files simultaneously. There is no mechanism in the single-user library to prevent multi-user interference. However, files can be exchanged between different instances of an application for sequential, not simultaneous, use. See File Migration for details.

Multi-User Standalone

The multi-user standalone model has the same issues with exchanging files described in File Migration in the FairCom DB Programmers Reference Guide.

However, sharing files between platforms simultaneously introduces additional complications with byte order and alignment, and adds the issue of record locking.

• Byte order is critical. Applications expecting different byte orders cannot seamlessly share files in this model.

Note: The FairCom Server does allow seamless file sharing!

However, the UNIFRMAT define allows HIGH_LOW applications to store and retrieve data in LOW_HIGH format instead of HIGH_LOW. This allows the files to be shared. If a record schema is present in the form of a DODA, data records will be adjusted automatically. See the FairCom DB Features chapter for more information on UNIFRMAT. Record Schemas in the FairCom DB Programmers Reference Guide describes the DODA.

• All applications sharing FairCom DB files must use the same alignment settings. If not, the application developer is responsible for ensuring record layouts are consistent.
• Locking between cross-platform applications introduces the complications of different locking standards and network file sharing mechanisms.
  • In multi-user standalone mode, FairCom DB uses three different locking systems to deal with the different locking standards used by various operating systems. Any combination of systems sharing files must use the same locking method with the same settings or they will not properly identify each other’s locks. See Multi-User Concepts in the FairCom DB Programmers Reference Guide for information on locking methods and their implementation.
  • Different network file sharing systems handle locks in different ways. Sometimes, this leads to incompatible locking systems between applications. When mixing applications from different environments, verify that locks are working. A simple test is to use a sample application, such as ctixmg, a FairCom DB example program. In one instance of the application, enable locking and add a record (which locks that record automatically). On the other platform, open ctixmg, enable locking and attempt to read the record added by the other application. If ctixmg returns an error DLOK_ERR (42), locking is functioning properly. Otherwise, there is a locking issue requiring troubleshooting.
  • Note, the FairCom Server relieves the programmer of many of the issues regarding heterogeneous networking and data integrity as described in the following sections.

Client/Server

The client/server model must also deal with byte order, but the FairCom Server handles this automatically when a record schema, in the form of a DODA, is present in the data file. Clients manipulate the data in their native format, and the FairCom Server manipulates the data in its native format. Translation takes place automatically during communication between the clients and the server.

This automatic translation allows a variety of clients using a variety of standards to cooperate with minimal effort on the part of the developer. For example, FairCom Servers running on Mac, Windows, and Unix systems can communicate simultaneously with any combination of clients on Mac, Windows, and Unix systems. This allows the customer flexibility in their selection of client and server systems, and allows clients in a mixed environment to share data.

See Record Schemas in the FairCom DB Programmers Reference Guide for more information about the DODA.

The client/server models also require the extended initialization functions, such as InitISAMXtd(), which pass logon information and trigger automatic target key transformation. See TransformKey in the FairCom DB Programmers Reference Guide for more information.

File Migration

Applications of any model can exchange files, sharing them sequentially but not simultaneously. Ensure the applications involved do not have the files open when they are copied or moved. If the application in question is the FairCom Server, this is especially important. See Copying Server Controlled Files in the FairCom DB Programmers Reference Guide for additional information.

Issues that affect file exchanging include byte ordering of numeric values and alignment of data structures. FairCom DB uses the same default for byte order as the hardware CPU and the default structure alignment dictated by the compiler.

• Byte Ordering is a processor-related issue. Intel-based processors store numeric values least-significant byte first (LOW_HIGH), while other processors store numeric values most-significant byte first (HIGH_LOW). So a four-byte integer containing the value “16” stored on an LOW_HIGH system as the hexadecimal value 10 00 00 00 would be stored on a HIGH_LOW system as the hexadecimal value 00 00 00 10. The order of the bits is consistent within the bytes, but the order of the bytes in the numeric value is reversed. By default, FairCom DB files are stored in the native format to the operating system, so files created on a HIGH_LOW machine cannot be exchanged directly with applications on a LOW_HIGH machine. See Portable Data Through UNIFRMAT Support, in the FairCom DB Programmers Reference Guide, which allows HIGH_LOW systems to work with LOW_HIGH files.
• Different compilers align data structures in different ways, but most have compile-time options allowing the alignment to be adjusted. Some compilers default to 1-byte alignment, which means structures are not adjusted in any way. With 2-byte or higher alignment, variables are aligned to the boundary matching the alignment or the variable’s size, whichever is smaller. There is a potential for the compiler to add dead space within a data structure. See Buffer Regions in the FairCom DB Programmers Reference Guide for more information on alignment.

Some of the common problems caused by alignment differences are:

1. Record lengths when different applications use different amounts of padding;
2. Indexing when padding shifts key segments to different locations than specified in the ISEG structure;
3. Data corruption when applications align the record buffer differently.

Use the FairCom DB ctunf1 reformat utility to adjust the alignment and byte ordering for a given file before exchanging it with another platform. See CTUNF1 - File Reformatting Utility in the FairCom DB Programmers Reference Guide for more information on this utility.

Migrating Your Application Between Operational Models

FairCom DB is designed to simplify the transition between different platforms or operational models. For the most part, changing models is simply a matter of compiling new libraries and re-linking your application.

However, some concepts are specific to one operational model. The transition can be made smoother by reading the entire section titled Operational Models (Operational Models, /doc/ctreeplus/30177.htm) in the FairCom DB Developer's Reference Guide.

Single-User to Multi-User Standalone or Client/Server

Changing from a single-user application to multi-user (either standalone or client/server) introduces the possibility of multi-user interference. This is prevented by proper locking techniques and/or transaction processing (available with the FairCom Server). See Multi-User Concepts and Data Integrity in the FairCom DB Programmers Reference Guide for more information.

A single-user standalone application can support transaction processing. If switching to multi-user standalone, transaction processing will not be available. Transaction processing is possible when migrating from single-user standalone to multi-user client/server, since both support transaction processing. In fact, this simplifies the locking issue since locking can easily be added to existing transaction processing calls. For example, replacing Begin(ctTRNLOG) with Begin(ctTRNLOG | ctENABLE). See Adding Transaction Processing and Data Integrity in the FairCom DB Programmers Reference Guide for more information.

Multi-user Standalone to Client/Server

A frustrating scenario, relating to users migrating from FairCom DB Multi-User FPUTFGET model to the FairCom Server, needs to be discussed. The dilemma relates to users of the FairCom Server receiving FCRP_ERR (14) error codes when opening server based files. The message text for the FCRP_ERR (14) error states “File corrupt at open”. Keep in mind that this scenario only exists due to a power loss, a machine being tuned off without stopping the server, or a miscoded program that does not close its files, only when the FairCom DB developer chooses NOT TO implement transaction control as suggested for the FairCom Server.

The FCRP_ERR (14) did not exist in the multi-user mode, so therefore a scenario was created in which customers, who have been running successfully for years, upgrade to the FairCom Server for stronger data integrity, start to receive “corrupt data errors”. The error is due to either the user shutting off the power to their server computer or an application not properly closing its files. Under FPUTFGET, the end-user (or the application) could get away with this. Now they get errors. Can you imagine the end-user frustration?

First, it is in order to discuss the FCRP_ERR (14) error. In FairCom DB single-user mode and under the FairCom Server, a file update flag is maintained within the header of each file. When a file is written to, in any manner, this flag is set to indicate the file has been updated. When the file is closed, this flag is re-set, indicating that the application has properly closed the file. Any attempt to open a file which has been marked update and which has not been properly closed, will result in a FCRP_ERR (14) error. The FairCom Server has no way of knowing the state of this file, and therefore must return this state back to the application as an integrity error/warning.

In FairCom DB’s multi-user FPUTFGET mode, the write operation is immediate secured to disk and therefore this flag is not maintained nor considered. An error 14 simply did not exist under FPUTFGET. An application could exit without properly closing a file, or a computer or file server (i.e.: NetWare Server) could be powered off (switched off by end-user) without causing any immediate catastrophic results for the customer. Perhaps the last “enter-key” entry at the time of power loss could be out of sync, yet the entire file would not be in question.

So now, take the scenario where one has a user, who has been running for years under FPUTFGET, who upgrades to the FairCom Server. As a reasonable half-step, the developer chooses to start with the application running in the same matter as it did with FPUTFGET, and intends to add the necessary code changes for transaction processing at a later date. The developer’s efforts are easy, simply re-link the application with the FairCom DB Client library instead of the FPUTFGET library, and it is ready to go. Now, the updated application and new FairCom Server are deployed at the customer’s site. The customer proceeds to operate in the same manner before, now complimented by the Client/Server architecture, experiencing less network traffic, and better performance due to the Server’s data caches. Now, as before, the customer “flips the switch” at the end of the day, just like they have done for years (without you knowing it). CATASTROPHIC situation. Now all data/index files open at the time are in question. Because of the FairCom Server’s caches, not just the last “enter-key”, but the entire file is now undefined. A FCRP_ERR (14) is properly returned the next time the file is opened. The user, who is used to coming in the next day, switching on the machine and going back to work, was now calling the developer’s tech support line reporting numerous error 14 messages and corrupt data.

The following solutions are available to solve this dilemma:

1. Of course, the ultimate solution is to secure the data using transaction control. This way all entries are 100% secure, up to the point of the last transaction commit. When migrating from FPUTFGET, this solution requires application code changes to implement the transaction control, as transaction processing does not exist in FPUTFGET.
2. When the FairCom Server was introduced, a file mode was added to FairCom DB called ctWRITETHRU. If a developer defined a file as ctWRITETHRU, all its write operations would be flushed to disk in the same manner of the FPUTFGET model. What is different about server ctWRITETHRU versus the FPUTFGET is that the server continues to maintain the update flag in the file’s header as described above. Therefore, if the machine is switched off, the data is in the same state as the FPUTFGET model. The exception is that the open request returns the 14 error, where under FPUTFGET it does not. The server keyword COMPATIBILITY WTHRU_UPDFLG exists for the FairCom Server to instructs the server not to maintain this update flag for ctWRITETHRU files. Therefore to completely “clone” the FPUTFGET results, a developer needs to add the ctWRITETHRU file mode to all file definitions within the program, and activate the COMPATIBILITY WTHRU_UPDFLG keyword in the server’s configuration file (ctsrvr.cfg). (Actually, this is better than a clone, for all read operations may safely remain cached under the FairCom Server.)
3. There is an additional feature which prevents the need to change the client application. Once the server configuration keyword COMPATIBILITY FORCE_WRITETHRU has been activated, all files opened in non-transaction processing mode (nonTRNLOG), are automatically set to ctWRITETHRU. In other words, this has the same effect as the developer changing all the file modes to include ctWRITETHRU, but because this is automatically done on the server side, the client side application does not need to be touched. This solution is the best and easiest for developers who want to take an easy step from FPUTFGET to Client/Server without changing the application.
   By activating the following keywords in your server’s configuration file (ctsrvr.cfg), the bases are covered:

COMPATIBILITY FORCE_WRITETHRU

COMPATIBILITY WTHRU_UPDFLG

In addition, a warning message helps the developer recognize this vulnerability. If a server does NOT have COMPATIBILITY WTHRU_UPDFLG in its configuration file, and if non-transaction files are opened without ctWRITETHRU in the file mode, then a warning will be issued in CTSTATUS.FCS concerning the vulnerability to FCRP_ERR (14) if a server is not shut down properly. The warning is only entered into CTSTATUS.FCS one time after each server startup when a vulnerable file is detected. The keyword STATUS_MASK WARNING_FCRP_ERR may be added to the configuration file to eliminate this warning message.

See the COMPATIBILITY PREV610A_FLUSH keyword for a good balance between performance and recoverability.

See also:

• COMPATIBILITY FORCE_WRITETHRU
• COMPATIBILITY WTHRU_UPDFLG
• COMPATIBILITY PREV610A_FLUSH

Adding Transaction Processing

Adding transaction processing to an application permits complex, atomic updates and automatic recovery in the case of software or hardware failures. Transaction processing can be added with the transaction processing sub-API or with SetOperationState().

To add efficient transaction processing, use the transaction processing sub-API described in Data Integrity in the FairCom DB Programmer's Reference Guide. If your application is already prepared for multi-user use, replacing LockISAM(ctENABLE) calls with Begin(ctTRNLOG | ctENABLE) and replacing LockISAM(ctFREE) calls with Commit(ctFREE) or Abort(ctFREE) calls, depending on the status of the transaction at that point, is a simple method of implementing transaction processing. See ctixmg.c for an example and Data Integrity for more details.

To add transaction processing quickly, though not in the most efficient manner, use SetOperationState() with the OPS_AUTOISAM_TRN mode. This performs a transaction around each ISAM update. See SetOperationState (SetOperationState, SetOperationState) for more details. This method does not effectively lock updates. You will still need to use some form of locking control. Consider the OPS_LOCKON_GET mode for SetOperationState() to minimize network traffic.

Once the application supports transaction processing, add either of the transaction processing file modes, ctTRNLOG or ctPREIMG, to the data files requiring transaction processing using the UpdateFileMode() function. Index files must be rebuilt to add or remove transaction processing. For additional information on Transaction Processing, please see Data Integrity.

Single-Threaded to Multi-Threaded

Switching to a multi-threaded, ctThrd, library is similar to adding transaction processing: superficially very easy, but with many implications.

As a minimum when using the ctThrd library, you must implement the ctThrd API. You must call ctThrdInit() before initializing FairCom DB, just as you must initialize c‑tree before calling any other FairCom DB API functions.

Any new threads that call FairCom DB API functions must be created with ctThrdCreate() or attached with ctThrdAttach().

In the Multi-threaded Standalone model, do not mix the FairCom DB Instance API (RegisterCtree(), etc.) with the ctThrd API. Each thread is its own instance of FairCom DB, and handles instance switching via the ctThrd functions. This is not an issue with the Multi-threaded Client Model.

See ctThrd Function Overview in the FairCom DB Programmers Reference Guide for more detail.

Lock Protocol Modification Prevents Spurious FPUTFGET Corruption Errors

Lock protocols used in the FairCom DB FPUTFGET operational model were modified to properly protect all data and index header updates correcting various file corruption errors, typically error 527 (BIDX_ERR) or 69 (BNOD_ERR). In particular, on Unix systems the default locking strategy has been changed from ctLOCK_DIRECT to ctLOCK_EXTENDED (see Data Record Lock Implementations for details).

Additionally, I/O locks were added to Unix library builds as already used by Windows FPUTFGET libraries. This logic can be disabled and existing Unix applications can revert to ctLOCK_DIRECT with NO_ctBEHAV_FPGIOLOK defined in ctopt2.h.

In V11, an extensive FPUTFGET Unix lock protocol review led to important changes. Several existing lockf() system calls were replaced with standard POSIX fcntl() calls allowing direct Unix read locks and avoiding overhead of then required fseek() system calls.

Note: It is strongly recommended to thoroughly test existing Unix FPUTFGET applications with this new library and confirm no unexpected lock compatibility issues have been introduced.

Prior Library Compatibility

To prevent errors and file corruption, it is important to not allow an old FPUTFGET library to operate on files that were created or opened with the new locking strategy.

The FPUTFGET library now turns on a bit in the header of data and index files that it creates on opens. This bit prevents an FPUTFGET library that uses the previous Unix locking strategy from accessing files that use the new strategy. The open of the file will fail with error 744 and the following message will be logged to CTSTATUS.FCS:

Thu Aug 20 16:01:27 2015

FREL_ERR(744) unsupported DEF_MASK bits: 08000000x for file...

Thu Aug 20 16:01:27 2015

myfile.dat

Note: A file that has this bit turned on (as a result of the file being created or opened by a new FPUTFGET library) will also cause single-user and client/server operational models prior to this revision to fail to open the file with error 744. The solution for those models is to update them to a version of the code that has this revision applied.

The cttest utility has been modified to show which lock options are in effect. When linked with an FPUTFGET library on Unix that uses the new Unix FPUTFGET locking, cttest shows these options:

ctLOCK_EXTENDED: Using extended offset locking mode.

ctBEHAV_FPGIOLOK: Using new Unix FPUTFGET locking strategy.
 

When linked with an FPUTFGET library on Unix that does not use the new Unix FPUTFGET locking, cttest shows this option:

ctLOCK_DIRECT: Using direct offset locking mode.