FairCom — Administrative Utilities

These utilities are intended for administrators. They are run from the client-side.

ctadmn - Server Administrator Utility

The FairCom Server Administrator Utility, ctadmn, is used by an Administrator to manage general server operations including configuring users, groups and files. It can also be used to monitor the users logged on to the FairCom Server and to disconnect any user from the c-FairCom Server.

To use this utility, do the following (which may vary between environments, depending on user-interface specifics):

Start running the program, ctadmn, as any other program in the environment.
Enter an ADMIN group User ID. Initially, only ADMIN will exist until you create others.
Supply the current password for the User ID given.
1. If this is the first time ctadmn has been run and the password for the User ID ADMIN is still ADMIN, change it before leaving ctadmn to ensure secure access to this program in the future. You can also change your password using ctpass (see Controlling FairCom Server Access).
You will be prompted for the (optional) file password for the file FAIRCOM.FCS. We recommend you do NOT give this file a password since you should already be the only one who can run this utility. To confirm the absence of a password, press the return key.
In response to the next prompt, supply the current name for the FairCom Server and press enter. If the FairCom Server name has not been changed (see Basic Keywords for details on SERVER_NAME), simply press Enter to use the default name.

Authentication File - This utility supports the use of an encoded password file. Encoded password files keep user IDs and passwords from plain view when the utility is used within a script file. They are created with the ctcmdset utility. The plain text form of the file should be:

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

Menus

After finishing these steps, the main menus for the FairCom Server Administrator Utility will be displayed, allowing access to the following groups of operations:

 1. User Operations
 2. Group Definitions
 3. File Security
 4. Monitor Clients
 5. Server Information (IOPERFORMANCE)
 6. Server Configuration (SystemConfiguration)
 7. Stop Server
 8. Quiesce the Server
 9. Monitor Server Activity
10. Change Server Settings

The following sections detail the areas controlled by the Administrator Utility. The steps necessary for each operation may vary slightly in different environments.

Note: While using the Administration Utility, press the question mark key (‘?’) at any prompt to access Help.

User Operations

User operations available in ctadmn are as follows:

Add New User by:
- Entering a new User ID. (The ID is limited to alphanumeric characters, hyphens, underscores, and periods. A hyphen cannot be the first character in a name.)
- (optional) Enter a long name (i.e., a text string) for use as a User Description (e.g., for screen displays, where the User ID may be too terse).
- (optional) Enter a User Memory Limit, which is a maximum memory allocation for this particular user that will override maximum memory allocations set at the server-level (for any user) or at the group level (for any particular group).
- (optional) Enter the User Memory Rule: Absolute, Guideline, or Default. See USR_MEM_RUL in Configuring the FairCom Server for details.
- (optional) Enter a user password.
- (optional) Assign user membership in from 1 to 128 Groups (i.e., Group IDs).
- (optional) Enter the first valid date for this User ID.
- (optional) Enter the last valid date for this User ID.
- (optional) Enter limit on consecutive logon failures if different from system default. See LOGON_FAIL_LIMIT in Miscellaneous Control for details.
Remove an existing User ID.
List authorized User IDs.
Change the Password for a given User ID.
Change which Group(s) a User ID is associated with by adding (up to 128) or removing groups from a list of a User’s association with Group IDs.
Change the User Description, i.e., change the long name identifying the User ID.
Change User Memory. Change the maximum amount of FairCom Server memory a given user can consume.
Change Extended Logon Validation, including start date, end date, and consecutive logon failures.
Change User Logon Limit limits users to a specified number of concurrent logons based on their user name. By default, the limit is zero, meaning no limit.

Group Definitions

Group definition operations available in ctadmn are as follows:

Add New Group by:
- Entering a new Group ID.
- (optional) Entering a long name (i.e., a text string) for use as a Group Description (e.g., to display, where the Group ID may be too terse).
- (optional) Entering a Group Memory Allocation, which is a maximum memory allocation for Users who are a member of this particular Group, and will override maximum memory allocations set at the server-level (for any user).
Remove an Existing Group ID.
List Groups: List all current Group IDs and descriptions.
Change Group Membership: Add or remove User IDs from a given Group.
Change Group Description: Change the long name for the Group ID.
Change Group Memory: Change the maximum amount of server memory user in a given group can consume.
Change Group Logon Limit: Limit users to a specified number of concurrent logons based on their group membership. By default, the limit is zero, meaning no limit.

File Security

File security operations that can be performed on a given file using ctadmn are as follows:

Change the File’s Password.
Change the File’s Permission Mask, controlling file operation permissions for three classes of users: World, Group, and File Owner.
Change the File’s Group.
Change the File’s Owner.

Note: Applications can be designed so separate data files and/or index files can be joined into a “superfile,” which is a single physical file from the point of view of the operating system. Separate “logical” files within a superfile are called superfile member files. From the point of view of the Administrator, superfiles, member files, and separate data or index files are all treated the same way.

Monitor Clients

The Administrator may want to know which users are currently attached to the FairCom Server or may want to force a user to disconnect from the FairCom Server. These functions are available:

List Attached Users.
Disconnect Users.

Note: Users IDs are associated with Task users (i.e., sessions). It is a task, or session, that is actually terminated. If a User is disconnected using ctadmn, the CTSTATUS.FCS entry is augmented by the terminated user ID and node name.

ctadmn Options for Monitoring Client Activity

The FairCom DB Administration Utility, ctadmn, can be used to monitor FairCom DB client activity by following these steps:

Start ctadmn. When prompted, enter the user ID of a member of the ADMIN group, the user’s password, the optional FAIRCOM.FCS file password, and name of FairCom DB.
Select option 4, “Monitor Clients”, from the ctadmn main menu.
Select option 1, “List Attached Clients”, from the Monitor Clients menu. ctadmn displays an entry such as the following for each connected client:

UserID: GUEST                             NodeName:
Task 11                                   Communications: F_TCPIP
         Memory:  25K       Open Files: 2      Logon Time:   0:02
         Tran Time:   0:01  Rqst Time:   0:00  InProcess  Rqst# 13  TRANEND

The meaning of each field is shown in the following table:

Field	Explanation
UserID	The user ID specified by this client when connecting to the server.
NodeName	The application-assigned node name for this client.
Task	The server-assigned task ID for the server thread servicing this client. Entries logged by this thread to the server status log include this task ID (for example, “- User 11”).
Communications	The communication protocol used by this client.
Memory	The current server memory usage attributed to this client thread.
Open Files	The current number of open files by this client.
Logon Time	The total logon time for this client connection.
Tran Time	The current elapsed transaction time for this client’s currently active transaction. Watch for unexpected high transaction times.
Rqst Time	If the text to the right of this value reads “InProcess”, this thread is currently executing a request on behalf of a client and this time is the current elapsed time for this client’s current request. In this case, a large “Rqst Time” value indicates that the server is taking a long time to complete this client’s request. If the text to the right of this value reads “NoRequest”, this thread is waiting for the next client request and this time is the current elapsed time since the completion of the previous client request. In this case, a large “Rqst Time” value indicates that it has been a long time since the server has received a request from this client.
Rqst#	If the text to the left of this value reads “InProcess”, the function number and function name refer to the FairCom DB API function currently being executed on behalf of the client. If the text to the left of this value reads “NoRequest”, the function number and function name refer to the FairCom DB API function that were most recently executed on behalf of the client.

Server Information

This prompt provides performance information since the last startup of the FairCom Server.

The following is an example of the server performance statistics:

==============================================================
============  Server performance statistics  =================
============  Values are since last startup  =================
==============================================================

        data buffer requests   =          382
        data buffer hits       =          364
        data buffer hit rate   =         95 %

        index buffer requests  =           77
        index buffer hits      =           73
        index buffer hit rate  =         94 %

        # of read operations   =          113
        bytes read             =       226761
        # of write operations  =           20
        bytes written          =        91264

==============================================================

Press RETURN to continue...

Server Configuration

This prompt provides configuration information since the last startup of the FairCom Server.

Note that FairCom DB developers can retrieve much of this information using the SystemConfiguration() function.

The following is an example of current server resource values:

===============  Current Server Resource Values  ===============

        current system memory usage              =     229825491 (219.178 MB)
        highest system memory use                =     229825491 (219.178 MB)
        current system net allocations           =         51292
        c-tree Plus files opened by system       =            15
        physical c-tree Plus files open          =             6
        c-tree Plus file control blocks in use   =            18
        # message in delete node queue           =             0
        # message in checkpoint queue            =             0
        # messages in system monitor queue       =             0
        current # of logons                      =             1
        current # of pending locks (system wide) =             0
        maximum number of logons                 =            32
        limit for the maximum number of logons   =            32
        maximum number of client nodes           =      no limit
        maximum number of connections per node   =      no limit
        remote connections allowed               =           YES
        c-tree Server process ID                 =          3115
        c-tree Server version                    = V11.8.0.xxxxx(Build-190607)
================================================================

Press RETURN to continue...

Stop Server

This prompt allows the Administrator to stop the FairCom Server. ctadmn will ask for verification that the FairCom Server is to be stopped and ask for a shutdown delay in seconds.

Note: If the Replication Agent is running on a server, replication should be stopped before trying to stop the server. If replication is running when you attempt to stop the server, you will see error 792 (External server shutdown disabled).

Quiesce Server

The Quiesce Server option allows an administrator to immediately block all access from clients to the server while maintaining a consistent server state.

Successful Quiesce.

It is now safe to perform a system backup of FairCom DB Server's controlled files.
Press RETURN once the backup is completed to resume the FairCom DB Server.

Press RETURN to continue...

While in this state all data and index files are physically closed and can be safely copied, backed up, or moved. Simply press RETURN to bring the server back online to clients.

When you quiesce the server, as long as the connection that quiesced the server remains connected, all other connections are blocked. Only if that connection goes away do we allow the ADMIN user to logon again and undo the quiesce.

When the administrator selects the ctadmn utility's option to quiesce a FairCom Server, ctadmn checks for active transactions. If any transactions are active, ctadmn prompts the user for a maximum time to wait for transactions to complete. That value is passed to the ctQUIET() function, which waits up to the specified number of seconds before aborting active transactions and quiescing FairCom Server.

Monitor Server Activity

From this menu an administrator can quickly obtain lists of files, including by connection, as well as locks on a particular file. These options are useful in determining active file usage by applications.

The "Purge SYSLOG entries" option (available with V13), prompts for the number of days of activity to keep. "0 Days" results in a truncate, while larger values use the regular per record approach. See the SYSLOG TRUNCATE Server System Event Log Keyword.

        Monitor Server Activity:

        1. List all files open by the FairCom DB Server
        2. List all files open by a particular connection
        3. List connections that have a particular file open
        4. List locks held on a particular file
        5. Close a file that is open in KEEPOPEN mode
        6. Purge SYSLOG entries

        Enter your choice (1-6), or 'q' to return to previous menu>>

Change Server Settings

FairCom DB allows certain options to be enabled/disabled at run time. This ctadmn menu displays available options. As the list grows over time, you may find additional entries not specifically listed here.

Change Server Settings:
         1. Configure function monitor
         2. Configure checkpoint monitor
         3. Configure memory monitor
         4. Configure request time monitor
         5. Change dynamic dump sleep time
         6. Change dynamic dump sleep interval 
         7. Enable or disable a status log mask option
         8. Change advanced encryption master password
         9. Change a DIAGNOSTICS option
        10. Change the specified configuration option
Enter your choice (1-10), or 'q' to return to previous menu>>

Dynamic Data and Index Cache

Memory caching of data is one of the most important subsystems of a database server, including FairCom DB. Caching allows the database process to optimally serve clients with data directly from memory when possible. Separate independent memory caches are used for data and indexes (buffers). Proper cache sizing is a critical performance tuning task of the database administrator. Too little memory, and the database engine becomes bottlenecked with persisted storage I/O. Too much, and not enough memory is made available to the operating system and other applications running on the host machine.

Cache resizing is a powerful feature for performance tuning with increasing database loads. Adding additional client connections, thus increasing user concurrency, frequently benefits from larger caches. Another situation is a very large ad-hoc database load or query where temporarily having a larger memory cache improves overall performance. Changing a cache size requires restarting the FairCom DB server process which is not an easy task in the 24/7 enterprise environment.

FairCom DB "hard" allocates memory for caches at server startup directly from the OS memory heap. Once allocated, the memory is permanently associated with the database for the life of the process. Allocated memory is chunked into page size blocks within the memory cache subsystem, historically making it difficult to resize as needed. An additional complication is how to handle existing data residing in cache. Losing that "hot" data will likely result in performance loss for connected applications until a runtime steady state is again achieved over time.

FairCom has had increasing requests for ability to change cache sizes at runtime and this release introduces dynamic cache resizing. Now, you can fine tune data and index caches as performance demands based on database connections and load avoiding costly database down time.

How to Resize Your Current Cache

The server administrator utility, ctadmn, supports a new configuration option set_cache_options as a parameter. This is followed with a null-terminated JSON string with desired options. Supported cache options are as follows:

"timeout": <timeoutInSeconds> where <timeoutInSeconds> is the maximum number of seconds to wait for active transactions to complete when quiescing the server in order to change the cache sizes. The default timeout is 1 second.
"dat_memory": "<new_data_cache_size>" where <new_data_cache_size> is an integer cache size followed by optional suffix such as mb for megabytes or gb for gigabytes. For example, "dat_memory": "100 mb". If not specified, the data cache size is not changed.
"idx_memory": "<new_index_cache_size>" where <new_index_cache_size> is an integer cache size followed by optional suffix such as mb for megabytes or gb for gigabytes. For example, "idx_memory": "200 mb". If not specified, the index cache size is not changed.
"keep_cached_data": "<yes_or_no>" where <yes_or_no> is either yes, which means the currently-cached data is kept in the data cache, or no, which means the currently-cached data is removed from the data cache. The default is yes.
"keep_cached_nodes": "<yes_or_no>" where <yes_or_no> is either yes, which means the currently-cached nodes are kept in the index cache, or no, which means the currently-cached nodes are removed from the index cache. The default is yes.

Administrator Utility Usage

From ctadmn, choose option 10 "Change the specified configuration option"


                Change Server Settings:

                         1. Configure function monitor
                         2. Configure checkpoint monitor
                         3. Configure memory monitor
                         4. Configure request time monitor
                         5. Change dynamic dump sleep time
                         6. Change dynamic dump sleep interval
                         7. Enable or disable a status log mask option
                         8. Change advanced encryption master password
                         9. Change a DIAGNOSTICS option
                        10. Change the specified configuration option
                        11. Change Logon Block Settings

        Enter your choice (1-11), or 'q' to return to previous menu>> 10

Enter the configuration option and its value >> set_cache_options {"dat_memory": "200 mb"}

Successfully changed the configuration option.


Press RETURN to continue...

Expected Status Log Messages

The resize request triggers an internally generated server quiesce state with the following common messages.


Mon Jun 13 13:47:51 2022 - User# 00030 ctQUIET: attempt quiet transaction state with action: 1100a0x  timeout: 1 sec
Mon Jun 13 13:47:51 2022 - User# 00030 ctQUIET: blocking call with action: 641a2x
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 33: 817
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 40: 817
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 47: 817
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 52: 817
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 58: 817
Mon Jun 13 13:47:54 2022 - User# 00030 Transaction aborted at ctQTaborttran for user# 65: 817
Mon Jun 13 13:47:54 2022 - User# 00030 ctQUIET: end of blocking call
Mon Jun 13 13:47:54 2022 - User# 00030 Using 3 index node LRU lists with 3962 buffers per list
Mon Jun 13 13:47:54 2022 - User# 00030 ctQUIET: unblocking call with action: 809e00x
Mon Jun 13 13:47:55 2022 - User# 00030 ctQUIET: end of unblocking call

Examples

Set data cache size to 100 MB and keep existing data in cache (since "keep_cached_data" defaults to "yes"). No change to index cache. In this example, if data cache is already 100 MB, no action is taken:
1. set_cache_options {"dat_memory": "100 mb"}
Set data cache size to 100 MB and remove all data from data cache:
1. set_cache_options {"dat_memory": "100 mb", "keep_cached_data": "no"}
Keep current data cache size and remove all data from data cache:
1. set_cache_options {"keep_cached_data": "no"}
Set index cache size to 200 MB and keep existing nodes in cache:
1. set_cache_options {"idx_memory": "200 mb"}
Set index cache size to 200 MB and remove all nodes from index cache:
1. set_cache_options {"idx_memory": "200 mb", "keep_cached_nodes": "no"}
Keep current index cache size and remove all nodes from index cache:
1. set_cache_options {"keep_cached_nodes": "no"}

This example shows actions on both data and index cache:

set_cache_options {"dat_memory": "500 mb", "idx_memory": "400 mb", "keep_cached_nodes": "no", "timeout": 5}

Direct SDK API

An application that is logged in as the ADMIN user can change data and/or index caches by calling the ctSETCFG() API function with option of setcfgCONFIG_OPTION and value set to:

set_cache_options {<cache_options>}

where <cache_options> is the null-terminated string in JSON format.

This example showing how to change the cache sizes in C or C++ code by calling the ctSETCFG() API function:

NINT rc;

rc = ctSETCFG(setcfgCONFIG_OPTION, "set_cache_options {\"dat_memory\": \"100 mb\"}");

Diagnostic Logs

The configuration option DIAGNOSTICS RESIZE_CACHE, which can be specified in ctsrvr.cfg and changed at runtime, causes cache resize diagnostic messages to be logged to CTSTATUS.FCS.

See Also

ctSETCFG()

ctadmn user listing for rtexecute thread running report launched by RTSCRIPT

The Communications section of the user listing shows rtexecute for threads launched by RTSCRIPT calls. Below is an example listing:

UserID: ---                               NodeName:
Task 15                                   Communications: rtexecute
         Memory: 1512K     Open Files: 8     Logon Time:   --
         Tran Time:   --   Rqst Time:  0:30  InProcess  Rqst# 0   -unknown-

This is analogous to how the launched dynamic dump thread, idyndmp, is listed. The listing states which thread to kill if one is using ctadmn to stop a report. Killing the thread that shows RTSCRIPT as the last function called will not interrupt the report being compiled by rtexecute.

ctcmpcif - IFIL-based Compact Utility

This utility can be used to encrypt and decrypt c-tree data files.

Operational Model:

Client
Standalone
FairCom DB Standalone SQL Service

Standalone Usage

ctcmpcif <data file name> <options> <user> <pass> <server> <-sectors>

Where:

Note: The order of parameters must be be as specified in the above usage descirption.

<-sectors> - The sector size to use. Enter a dash followed by the size of the sectors. The sector parameter is especially useful if you need to adjust a file’s PAGE_SIZE to match the FairCom Server. This option must go after <server>. The sector size is 256 is in the example below:

ctcmpcif xx8.dat ADMIN ADMIN FAIRCOMSMS -256

<options> can be:

-callback - Enable notifications of compact progress.
-compress - Create the compacted data file with compression (defaults to zlib library if not specified in server configuration).
-encrypt=cipher - Create the compacted file using the specified cipher. See the encryption sections below.
Accepted values:
- AES: aes16, aes24, or aes32
- Blowfish: blf8, blf9, ..., or blf56
- DES: des8, des16, or des24
- none
-flexrec - (Supported in V11.5 and later) Add Hot Alter Table support to the file.
-idxseg=<M>@<S> - If creating an index, use M automatic segments of size S. Size is specified in megabytes, or you can specify MB or GB as a suffix. (Example: -idxseg=10@1GB)
-local - causes the utility to use a standalone (local side) connection instead of a client connection when linked with a LOCLIB library.
-nocompress - Create the compacted data file without compression.
-noflexrec - (Supported in V11.5 and later) Remove Hot Alter Table support from the file.
-oldsec - Use the utility's original security attribute assignment.
-online - (In V13 and later) Defragment the file in ctSHARED mode. Online compact is not allowed for changing encryption or compression settings.
-purge - Delete records that have illegal duplicate key values.
-redosrl - Reassign serial numbers to the records in the compacted file instead of copying the values from the original file.
-repairdata - Attempt repair if damaged data records are detected.
‑sortmem=<N> - Set the size of sort memory to N KB in non-client builds. This option can be used to increase the rebuild speed for large files.
-temppath=<temporary_path> - Use the specified directory for temporary files.
-updifil - Update IFIL resources in the data file.
-x8 - Use extended create blocks read from data and index files.

Client-Side Usage

# ctcmpcif DataFileName [-purge] [-updifil] [<UserId>]
[<UserPassword> [<ServerName>]]

Where:

-purge - purge duplicate records.
-updifil - update IFIL resources in the data file.

UserID, UserPassword, and ServerName are only needed for client versions of this utility. FairCom recommends building this utility as a Single-user Standalone application.

ctcmpcif reads the IFIL structure from DataFileName and calls CompactIFileXtd() and RebuildIFileXtd() to compact and rebuild DataFileName and its associated indexes. If ctcmpcif cannot extract the IFIL from the target file, it will ask for the name of another copy of the file from which to extract the IFIL information.

Prior to FairCom DB V9.3, several option values were defined such as updateIFIL, purgeIFIL, and badpartIFIL, that can be specified. These values are specified by adding them together. For example: myifil.tfilno = updateIFIL + purgeIFIL. A new approach simplifies checking of these options. The following option values are now specified in ctport.h:

#define updateIFILoption 0x0002
#define purgeIFILoption 0x0004
#define badpartIFILoption 0x0008
#define redosrlIFILoption 0x0010

These values can now be OR-ed together and the negative of the result stored in the tfilno field of the IFIL structure passed to the compact or rebuild function. For example, to indicate that you want to assign new serial numbers, do the following:

myifil.tfilno = -redosrlIFILoption;

CMPIFIL(&myifil);

If you want to also use more than one option when compacting or rebuilding a file, OR them in to the value and negate the result. For example,

/* assign new serial numbers and update IFIL resource. */

myifil.tfilno = -(redosrlIFILoption | updateIFILoption);

CMPIFIL(&myifil);

ctcmpcif will attempt to open files of any page size definition. In addition, in standalone mode, it will create the new file based on the provided page size.

Note: This tool does not compact partitioned files. If you attempt to compact a partitioned file, the tool will return PCMP_ERR (954, compacting partitioned file not supported).

Changing Encryption Attributes

In V11 and later, the compact utility can optionally change the encryption attributes. To use this option, OR in the setencryptIFILoption bit into the tfilno field of the IFIL structure whose address you pass to the compact API function. When using this and other options, remember to negate the tfilno value after you OR in the options. For example:

myifil.tfilno = -(redosrlIFILoption | setencryptIFILoption);
CMPIFIL(&myifil);

Specifying the Encryption Cipher

In V11 and later, the ctcmpcif utility supports an option to specify the encryption cipher for the data and index files created by the compact operation. Usage:

-encrypt=cipher - Create the compacted file using the specified cipher:

for AES, use aes16, aes24, or aes32
for no encryption, use none

Note: If an index file does not exist, the original data file's encryption attributes are used when creating that index file.

To change the encryption attributes of a file using this ctcmpcif compact utility or the compact API function from a client, you must add the option CHANGE_ENCRYPTION_ON_COMPACT YES to ctsrvr.cfg. Otherwise, the operation fails with error NSUP_ERR (454, not supported).

Environment Variable to Enable Advanced Encryption

In V11.5 and later, c-tree supports enabling advanced encryption at run time using an environment variable. Set the environment variable CTREE_ADVANCED_ENCRYPTION to YES to enable advanced encryption if it is supported. This environment variable can be used to allow c-tree utilities to enable advanced encryption even if they haven't been updated yet to automatically enable advanced encryption when needed. Examples include the rebuild and compact utilities, ctrbldif and ctcmpcif.

Note: If c-tree does not support advanced encryption and this environment variable is set, the c-tree initialization will fail.

Note: In order to use advanced encryption, a master key must be supplied. The utility will prompt for the master key, or a master key file can be created like this:

create master key file:
ctcpyf -k mykey -s mykey.fkf

set environment variable to the name of the master key file:

CTREE_MASTER_KEY_FILE=mykey.fkf

If using these steps, then the utility will read the master key from the file instead of prompting for the key.

Considerations

Even without the -x8 option, some extended create block settings such as the huge file, extended header, and 6-byte transaction number options, are always applied to new files.
The -x8 option requires all associated index files that are referenced in the data file's IFIL structure to exist, as the utilities use OpenIFile() to open the data file and all associated index files to read the extended create block values.

When an application calls the standalone version of the Xtd8 file compact or rebuild functions (for example, CMPIFILX8() or RBLIFILX8()), index files created by the compact or rebuild no longer have the 6-byte transaction number attribute enabled if the specified extended create block's x8mode field has the ctNO6BTRAN bit set.

A rebuild or compact will fail with the new error code TFLN_ERR (943) when a client library that supports the new rebuild or compact option format attempts to use this feature with a FairCom Server that does not support this new format.
In V11 and later, the compact and rebuild API functions preserve the original files' encryption attributes by default. If the index files don't exist when compact/rebuild are called, they will be created with the data file's encryption attributes.
In V11 and later, the compact utility always deletes and recreates the index files after saving the resources from the original index file. This provides the expected behavior of (possibly) reducing the size of the index file.

-oldsec - Updates in handling of security attributes

In V10.3 and later, the ctrbldif utility assigns data file owner/group/permissions to index files and ctcmpcif assigns original data file owner/group/permissions to compacted data/index files.

The rebuild and compact utilities now read the permission mask, owner, and group settings from the original data file and after the rebuild or compact is completed, the utilities assign these same attributes to the new files.

A command-line option has been added to the utilities to restore the previous security attribute behavior. Use the -oldsec option to cause the rebuild and compact utilities to set the security attributes as they did before this revision. For example:

ctrbldif mark.dat -oldsec ADMIN ADMIN FAIRCOMS

The -oldsec option can be used if rebuild and compact are failing with error 455 (user does not belong to group) if you delete an index then run the rebuild or compact utility on the data file using a user account that does not belong to the group that is assigned to the data file.

Note: These changes only apply to the client and server versions of these utilities. Setting the security attributes is not supported in standalone mode, so the standalone rebuild and compact utilities behave as follows:

1) When the standalone mode rebuild utility creates new index files instead of reusing the new index files (for example if the original index files have been deleted before running the rebuild utility), the new index files are assigned a permission mask of zero (no restrictions on permissions), and the owner and group are unassigned (empty).

2) The standalone mode compact utility always preserves the security attributes of the data file. If the index files do not exist, the newly-created index files are assigned the same values as the standalone rebuild does when it creates new index files.

ctdbdistinct - Key Count Utility

Operational Model

Client

ctdbdistinct is a utility for managing distinct key counts in SQL tables and databases.

Command Syntax

ctdbdistinct.exe -s server -c command [-d database][-t table][-u user][-p passwd]

Options

-c command where command is one of:
- enable - enables distinct key counts
- check - validates existing distinct counts
- repair - updates existing distinct counts
-d database target database (all tables)
-s server source c-tree Server name (may be "local" for direct access)
-t table target table (one table)
-u uid user name on source server
-p upw user password on source server

The -c enable command assigns the CTDBINDEX_DUPCNTFLAG to all non-unique indexes in a database or a specific table. These indexes will start tracking distinct key counts. This command is intended to upgrade existing tables or databases created without distinct key counts.

The -c check command compares the distinct key counts stored in indexes with the actual key counts, and lists tables that fail validation. It operates while the table(s) may be in use by other users. Tables that fail the validation are logged to stdout.

The -c repair command updates distinct-key-count indexes with the current number of distinct key counts. It can run when tables are in use by other users. It briefly acquires a table read lock to update the distinct key count.

If the distinct count exceeds the desired limit of 2 per leaf node, it logs "Distinct key discrepancy: 20 != 32". If it exceeds 1/3 of the total keys in the index, the count will likely lead to bad SQL optimizer plans. It logs "Extreme distinct key discrepancy: 5 != 435123". When a table’s indexes should be repaired, it logs "Incorrect Distinct counts for table myTable index i_myTable_names".

Example 1:

Check distinct key counts on all files in the ctreeSQL database.

ctdbdistinct -s FAIRCOMS -c check -d ctreeSQL -u admin -p ADMIN

Example 2:

Repair distinct key counts on one table ./ctreeSQL.dbs/mytable.dat

ctdbdistinct -s FAIRCOMS -c repair -t ./ctreeSQL.dbs/mytable.dat -u admin -p ADMIN

ctdump - Schedule Backup Utility

Note: For a complete discussion of dynamic dumps, see Backups and Data Integrity in the FairCom Server Administrator's Guide.

Operational Model:

Client

ctdump schedules a dynamic dump on the fly. The backup definition script may be located either on the server, or passed in from the client.

Command Syntax

ctdump [-s svn] [-u uid] [-p upw] [-t script] [-b bufsiz] [-n] [-c] [-o backup_filename] [-x]

Options

-s svn - c-tree Server name
-u uid - User name
-p upw - User password
-t script - Dump script name
-b bufsiz - Use buffer size of bufsiz bytes
-c - Send dump script from client
-m - Minimize progress notifications
-n - Send progress notifications to client
-o backup_filename - Write dump stream from server to file on client
-x - Write dump stream from server to stdout

For options available when scripting a dynamic dump, see Dynamic Dump Options.

The following demonstrates example usage of this utility:

ctdump ADMIN ADMIN thescript FAIRCOMS

Secure Authentication File

This utility supports the use of an encoded password file. Encoded password files keep user IDs and passwords from plain view when the utility is used within a script file. They are created with the ctcmdset utility. The plain text form of the file should be:

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

Dump and Restore Version Compatibility

The ctrdmp utility is used to restore a Dynamic Dump and the ctfdmp utility can be used to roll forward. Occasionally an update to the FairCom Database Engine may cause an incompatibility between versions. For this reason you must use the ctrdmp from the same release from which the dump was created. It is important to save a copy of the ctrdmp utility that is compatible with each dump file. ctfdmp, ctldmp, and ctrdmp utilities display the FairCom DB version used to compile them when they are run.

Compression and Encryption

The ctdump.exe utility can accept compression and encryption utility inputs.

Windows backup compressed with 7z compression utility:

ctdump -s FAIRCOMS -u admin -p ADMIN -t dump.txt -c -x | 7z a -si backup.7z

Unix backup compressed and AES encrypted with standard utilities:

./ctdump -s FAIRCOMS -u admin -p ADMIN -t dump.txt -c -x | gzip|openssl enc -aes-256-cbc -salt >backup.gz.aes

Tip! Unix/Linux Alternative

This method also works with server-side dynamic dump backups prior to this new support and can be implemented on Unix systems using named pipes (mkfifo), and sending the dump to the named pipe.

Create a named pipe:

mkfifo testpipe

The pipe blocks until both sides are in use. So we start our compression command:

cat testpipe | gzip > backup.gz &

Now we create our dynamic dump, with "testpipe" as the dump file. (This example uses the client output, however it should work the same from the server):

ctdump -s FAIRCOMQ -u admin -p ADMIN -t dump.txt -c -o testpipe

c-tree(tm) Version 11.8.0.61131(Build-180802) Dynamic Backup Utility

Reading dump stream from server with buffer size of 100000

Dynamic Dump has been successfully written to the file testpipe.

The backup.gz file contains the dump file.

The same approach can be used to compress and encrypt (and potentially other operations provided by system utilities):

cat testpipe|gzip|openssl enc -aes-256-cbc -salt >backup.gz.aes

To extract the dump:

cat backup.gz.aes | openssl enc -aes-256-cbc -d |gzip -d >backup.ctree

Use ctrdmp to extract the files from backup.ctree.

Error Codes

The following error codes are related to dynamic dump operations:

Error Name	Error Code	Explanation
FUNK_ERR	13	Cannot determine file type. Possibly a c-tree V4.3 file?
READ_ERR	36	Failed to read file, either a corrupted or non-tree file.
TCOL_ERR	537	Transaction log collision. Two sets of transaction logs in the same directory?
FCPY_ERR	796	Immediate dump restore file copy failed.
DRST_ERR	797	Immediate dump restore failed.

See Also:

ctrdmp - Dynamic Dump Recovery or System Rollback (ctrdmp - Backup Restore or System Rollback, Backup Restore Utility)
System Rollback
Rolling Forward from Backup

dyndmp and dyndumpsetopt

The Dynamic Dump can send a script to the server and receive a dump stream and/or status messages from the server. The following capabilities are available for scripting a dynamic dump:

When scheduling a dynamic dump, the client can send the dump script to the server.
When running a dynamic dump, the client can request that status messages be sent to it while the dump is performed and/or the dump stream file can also be sent to the client process.

To use these options, call the function dyndmpsetopt() before calling dyndmp():

extern NINT dyndmpsetopt(NINT option,pVOID value);

The following are the supported options. All options are disabled by default.

DDOPT_SENDSCRIPT - Send dump script to server. Set value to the script name, or set it to NULL to disable this option. Example:

dyndmpsetopt(DDOPT_SENDSCRIPT, "script.txt");

DDOPT_RECVSTREAM - Receive dump stream from server. Set value to YES to enable this option or NO to disable this option. Example:

dyndmpsetopt(DDOPT_RECVSTREAM, (pVOID) YES);

DDOPT_RECVSTATUS - Receive status messages from server. Set value to YES to enable this option or NO to disable this option. Example:

dyndmpsetopt(DDOPT_RECVSTATUS, (pVOID) YES);

DDOPT_SETCALLBK - Set callback function. Set value to the callback function pointer, or set it to NULL to disable the use of the callback function. Example:

extern ctCONV NINT mycallback(pVOID pctx,pVOID pdata,NINT datalen,NINT opcode);

dyndmpsetopt(DDOPT_SETCALLBK , &mycallback);

DDOPT_SETCONTEXT - Set callback function context pointer. Set value to the context pointer that will be passed to the callback function. Example:

mystruct mycontext;

dyndmpsetopt(DDOPT_SETCONTEXT, &mycontext);

DDOPT_SETBUFSIZ - Set communication buffer size. Set value to the buffer size to use. Example:

dyndmpsetopt(DDOPT_SETBUFSIZ, (pVOID) 100000);

Notes:

1) The dump options remain in effect for all dynamic dumps performed by the current connection until they are changed.

2) When the DDOPT_RECVSTREAM or DDOPT_RECVSTATUS options are used, the following dynamic dump script options are ignored:

COPY_NONCTREE - Non-ctree files cannot be copied.

DATE and TIME - No scheduling of dump for later time.

EXT_SIZE - Only one dump extent is created.

FREQ - No repeat of dump.

SEGMENT - No segmenting of dump stream.

The ctdump utility supports these features through command-line options:

usage: ctdump [-s svn] [-u uid] [-p upw] [-t script] [-b bufsiz] [-n] [-c] [-o backup]

Options:

-s svn - c-tree Server name
-u uid - User name
-p upw - User password
-t - Dump script name
-b bufsiz - Use buffer size of bufsiz bytes
-c - Send dump script from client
-m - Minimize progress notifications
-n - Send progress notifications to the client
-o backup_filename - Write dump stream from server to file on client

Example:

# ctdump -u ADMIN -p ADMIN -o backup.fcd -c -t script.txt -s FAIRCOMS -n

Results:

c-tree(tm) Version 11.1.0.46197(Build-150826) Dynamic Backup Utility

Reading dump stream from server with buffer size of 100000

Start dump. Estimated dump size: 2691072

FAIRCOM.FCS

86% 100% [ 2328576 of 2326528 bytes]

SYSLOGDT.FCS

89% 100% [ 86016 of 81920 bytes]

SYSLOGIX.FCS

99% 100% [ 266240 of 262144 bytes]

S0000000.FCS

100% 100% [ 4096 of 128 bytes]

S0000001.FCS

100% 100% [ 4096 of 128 bytes]

L0000001.FCS

100% 100% [ 2048 of 679 bytes]

End dump. Actual dump size: 2705408

Dynamic Dump has been successfully written to the file backup.fcd.

See Also:

Dynamic Dump Script File

ctdump stdout changes to the ctree Library

Changes to the ctree Library

To safely support directing the dump to stdout, the ctree library has been updated so it does not internally write to stdout, which would corrupt the dump stream. To accomplish this, references to stdout/stderr were replaced with macros CT_STDOUT, CT_STDERR, which are now controlled by internal functions:

extern ctCONV FILE * ctDECL getCtreeSTDOUT(void);

extern ctCONV FILE * ctDECL getCtreeSTDERR(void);

extern void ctDECL setCtreeSTDOUT(FILE * out);

extern void ctDECL setCtreeSTDERR(FILE * err);

We have updated references to the following to allow other c-tree utilities to make use of stdout redirection for greater conformance to Unix behavior standards:

printf
vprintf
stdout
stderr

ctfilblkif - File Block Utility

ctfilblkif

ctfilblkif will block, or unblock a specified FairCom DB file. The default behavior is to block access to the specified file. Pass the -u option to unblock a file.

Operational Model:

Client

Usage

ctfilblkif [-s server][-f filename][-u][-m] {-p password|-a authfile}

-s: Server (default: FAIRCOMS@localhost)
-f: File name, including extension and relative or absolute path (a relative path of "./" can be omitted)
-u: Unblock file
-m: Block file with manual reopen option (otherwise automatic reopen)
-a: Authentication file name
-p: Admin Password
-t: Allow active transactions time (seconds) to complete before blocking
-n: Block new transactions

Example:

Block new transactions and allow active transactions 3 seconds to complete before blocking the mark.dat file:

ctfilblkif -f mark.dat -n -t 3

Manual Reopen

The ctfilblkif utility now supports an option (-m) to block a file with the reopen option. This means that when the file is unblocked, those connections that had the file open when it was blocked must "manually" reopen the file (e.g., the application must explicitly reopen the file).

Example 1:

Block the file with the reopen option:

ctfilblkif -f mydatafile.dat -m -p ADMIN -s FAIRCOMS

Example 2:

Unblock the file. The connections that originally had the file open must reopen the file:

ctfilblkif -f mydatafile.dat -u -p ADMIN -s FAIRCOMS

Note: FairCom does not guarantee preserving the state of a transaction, locks, ISAM context when a file is blocked and then unblocked, even if using the auto open option. Because ISAM context information is lost when the file is blocked, an error 590 may be returned after the file has been blocked and unblocked using ctFBautopen or when using ctfilblkif utility, which uses this mode.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

ctfileid - Update File IDs

The Update File ID utility, ctfileid, provides a convenient and safe way to update the fileid parameter of the file header. See the section Copying Server Controlled Files in the FairCom DB Programmer's Reference Guide for details for when this may be necessary. The file is opened exclusively, ensuring that the server does not have it open.

Operational Model:

Client
Standalone

The syntax for this utility is shown below:

ctfileid file [-ioq] [-n <size>] [-t] [-s <svn>] [-u <uid>] [-p <upw>]

-i - Also update indexes related to data file.
-o - Force open of corrupted files (ctOPENCRPT).
-q - Quiet (do not output to stdout).
-t - Uses the PUTHDR() mode ctTIMEIDhdr to update only the time ID portion of the file ID.
-s <server name> - FairCom Server name.
-u <user ID> - User name.
-p <user password> - User password.

Note: ctfileid.c replaces the previous informal and undocumented utilities, updateid.c and newid.c.

Standalone Usage

An additional option is available to set the node size in standalone mode:

-n <size> - Set node size (standalone only).

See Also:

Copying Server Controlled Files

ctflush - File Flush Utility

ctflush is a c-tree utility that flushes updated buffers for c-tree files.

Operational Model:

Client

Usage

ctflush [-1 <passfile>] [-u <userid>] [-p <password>]

[-s <servername>] [-m 1|2|3 <filename>|4|5|6]

[-i <interval>] [-t <timeout>]

Where:

-1 <passfile> - encrypted password file (see below)
-u <userid> - user id for logon
-p <password> - user password for logon
-s <servername> - FairCom Server name including machine name or IP address
-i <interval> - for mode 4: interval in seconds between checkpoint calls for mode 5: interval in seconds between restore points
-t <timeout> - for modes 5 and 6: maximum time in seconds to wait for active transactions to complete
-m <mode> - 1, 2, 3 <filename>, 4, 5, or 6:

Mode 1 flushes updated buffers for all transaction-controlled files.

Mode 2 flushes updated buffers for all open files.

Mode 3 <filename> flushes updated buffers for the file <filename>.

Mode 4 performs a checkpoint.

Mode 5 creates a lightweight restore point.

Mode 6 creates a restore point with a checkpoint.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

ctpass - Password Utility

FairCom Server utility to allow users to change their password.

Operational Model:

Client

The following steps are required for a user to change the password associated with their own User ID.

For optional information on setting requirements for user passwords, see Setting password requirements.

Run the utility program ctpass as any other program in the environment.
Enter your current User ID.
Enter the current password for your User ID, if you have one. (Maximum 1024 characters. Maximum nine characters for V9 and prior).
Continue by entering the current name of the FairCom Server (i.e., the default name or another name, supplied in the FairCom Server configuration file).
Now change your password by entering the new password.
To be sure to enter the new password, you may be asked to enter it twice before it will be accepted. If the same name is not entered both times, try again.

Note: Whenever input is requested, the user may enter a question mark (?) to receive HELP.

After the new password is entered and confirmed, a message saying your User ID password has been successfully updated will be displayed. After being updated successfully, the new password must be used with the User ID to log on to the FairCom Server.

Note: All users can change their own passwords. In addition, users who are members of the ADMIN group can change the password of all accounts that are not members of the ADMIN group. Only the super ADMIN account (named ADMIN) can change a password for an account that is a member of the ADMIN group.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

ctpathmigr - Change Internal Path Separators

The ctpathmigr utility is used to change embedded path separators for FairCom DB SQL databases when migrating between Unix and Windows platforms. This functionality was previously available in the ctpath utility. This utility changes database and IFIL resource path separators from the native format of the system where the utility is run (i.e., \ for Windows; / for Unix) to the non-native format. It should be run before moving the files.

Operational Model:

Client

Syntax

ctpathmigr [-s <server>] [-u <user>] [-p <password>] [-d <database>]

Description

ctpathmigr with command line arguments returns 0 when the execution detected no errors, or a value different than 0 when errors were detected and the error messages are written to the screen.
If you omit the -d database switch, all databases in the session will be scanned.
Command line options should start with a '-' or '/' character. Use a lowercase character for the option (e.g., -s not -S).
Command line switches may have optional spaces between the switch and the argument. Example: -s FAIRCOMS or -sFAIRCOMS are the same.
Command line options may be entered in any order.

ctquiet - Quiesce FairCom DB Utility

The ctquiet utility allows an administrator to quiet the server from a script. An interactive option is available in the ctadmn utility.

Operational Model:

Client

Usage:

ctquiet [-c] [-f] [-i] [-m] {-p password|-a authfile} [-r] [-s server] [-t timeout] [-u] [-x] [-w]

Parameters:

-a: Authentication file name
-c: Wait for replication readers to finish processing
-f: Full consistency (default: crash consistency)
-i: Ignore inactive replication readers (used with -c option)
-m: Quiesces the server, changes the specified configuration options, and resumes normal server activity. Allows changing server configuration options that require the server to be quiesced, avoiding additional calls to the Server. Can be specified one or more times in a call to ctquiet.
-p: Admin Password
-r: Create an incremental backup restore point. Automatically unquiets server
-s: Server (default: FAIRCOMS@localhost)
-t: Set transaction timeout in seconds. (maximum/default: 60 seconds).
-u: Unquiet server
-x: Abandon Quiet request if timeout exceeded. (Default: Abort long transactions.) Applies ctQTfailAfterTimeout mode. This mode has been extended to apply to restore point checkpoint creation for incremental forward rolls.
-w - (Linux only) Execute command on successful quiet. Waits for SIGINT to unquiet server.

Note: The -c option requires ALL client utilities, replication readers, and servers to be installed from FairCom DB V11.3 or later. It is not compatible as a drop-in to current running environments.

Support for Incremental Backup Restore Points

Several changes have been made to ctquiet to support restore points necessary to carry out an incremental backup strategy:

The -r feature can be used to generate the restore point files necessary to carry out an incremental backup strategy.
The ctQTfailAfterTimeout mode has been extended to apply to restore point checkpoint creation for incremental forward rolls.
The -x option applies the ctQTfailAfterTimeout mode to the already existing options.
The -t timeout output has an effect only if -f or -r is specified.
The default timeout has been changed to match the server's maximum.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

Full Consistency

Note: This utility provides a -f option, which enables full consistency (also known as a "clean quiesce") in which files are flushed to disk from cache. When this option is used, the system does not require any files to be rebuilt or recovery from transaction logs to occur.

The default is the -f option is off, which results in "crash consistency" (also known as a "dirty quiesce"). The default means that transaction logs are required and, if present, the transaction logs are used to get back into a clean state once recovery completes.

If files are NOT under transaction control, the -f option is strongly recommended, otherwise you will have to do a rebuild to get the files back to a clean state.

Physical Copy of Files

A quiesced state allows a physical copy of files to be taken that can then be restored at a later time. For systems that provide hardware-based snapshot features, this allows extremely fast system backups to take place while maintaining full data integrity.

The -r option creates an incremental backup restore point. After the ctquiet call succeeds, this new restore point and all server transaction logs can be copied to a backup area and used for an incremental recovery.

Timeout

The -f and -r options require that no transactions are active for the quiet to complete. The quiet operation defaults to waiting 60 seconds for the outstanding transaction to complete before aborting these transactions. Operations by other users and connections wait during this time.

The -t and -x options modify the timeout behavior:

-t - Allows setting a shorter timeout. Times longer than 60 seconds are reduced to 60 seconds.
-x - Instead of aborting the transactions, the quiet operation will fail if the timeout expires without all transactions completing.

Example 1:

You want to create a server backup for your application that is not using transaction logging on all the files you want to backup. Only allow 10 seconds before aborting outstanding transactions.

ctquiet -f -t 10 -s FAIRCOMS -p <admin password>

Make your backup and then end the quiet:

ctquiet -u -s FAIRCOMS -p <admin password>

Example 2:

You want to create a fast server backup, and all your files are using transaction logging. Your backup will need to go through recovery before being used.

ctquiet -s FAIRCOMS -p <admin password>

Make your backup and then end the quiet:

ctquiet -u -s FAIRCOMS -p <admin password>

Example 3:

You want to create a restore point for an incremental backup and only allow a brief (1 second) delay. You don't want ctquiet to abort any long running transactions.

ctquiet -r -t 1 -x -s FAIRCOMS -p <admin password>

Notes

When you quiesce the server, as long as the connection that quiesced the server remains connected, all other connections are blocked. Only if that connection goes away do we allow the ADMIN user to logon again and undo the quiesce.
After the server is quiet and the ctquiet utility disconnects, one ADMIN connection is allowed to reconnect. There is no prevention of a separate process connecting as ADMIN while the server is in a quiet state and precluding the unquiet call.
There is a subtle distinction between a "quiet" state and a file blocked with the ctFILBLK() call. While in the quiet state, files are not physically closed and cannot be moved or replaced while in this mode. Compare this to a "blocked" state, where the file can be replaced, as the OS file handle has been released.
When the optional -w COMMAND switch is used (unavailable on Windows), the behavior of ctquiet is modified to perform as follows:
1. After successfully quieting the server, it makes a system call to execute COMMAND.
2. It remains connected and waits for SIGINT to unquiet the server. If ctquiet is killed before receiving SIGINT, the server will remain in a quiet state until a new connection unquiets the server.

ctrbldif - IFIL-based Rebuild Utility

A rebuild utility using the IFIL definitions stored in the header of a file:

Operational Model:

Client
Standalone

Usage:

ctrbldif DataFileName [<UserId> [<Password> [<ServerName>]]] <options>

Description

ctrbldif reads the IFIL structure from DataFileName and calls RebuildIFileXtd() to rebuild DataFileName and its associated indexes.

The following switches can be added in the location specified by <options> above:

-callback - In V11.5 and later: When used with a FairCom DB Server this will provide additional feedback on the rebuild progress.
-idxseg=<M>@<S> - If creating index, use M automatic segments of size S. Size is specified in megabytes, or you can specify MB or GB as a suffix. For example: -idxseg=10@1GB
-local - causes the utility to use a standalone (local side) connection instead of a client connection when linked with a LOCLIB library.
-oldsec - Use prev10.3 security attribute behavior.
-online - In V13 and later: Rebuild the index while it remains available to other users.
-purge - Purge duplicate records.
-repairdata - Attempt repair if damaged data records are detected.
-skipdatascan - In V11.5 and later, if you know your datafile is in a good state, you can use this option to skip the initial datafile validity scan for faster rebuilds.
-sortmem=<N> - Use N KB of memory for sorting. This option can be used to increase the rebuild speed for large files.
-updifil - Update the IFIL resource within the data file.
-x8 - Use extended create blocks read from data and index files.

FairCom DB Standalone Options

-temppath= - directory path, <temporary_path>, to create temporary sort and purged record files. (This standalone option is the equivalent of the TMP_PATH server configuration option.) The default location is the current directory.
-<sectors> - sector size, <sectors>. (in multiples of 128). The sector parameter is especially useful if you need to adjust a file’s PAGE_SIZE (index node size) to match the FairCom Server. FairCom DB standalone models default to 16 sectors (2048 byte node size) while the FairCom Server defaults to 64 sectors (8192 bytes).

FairCom Server Options (Must be passed before any optional switches)

<UserID> : client user name to logon to a FairCom Server.
<UserPassword> : client user password to authenticate to a FairCom Server.
<ServerName> : FairCom Server name for a client to connect.

Environment Variable to Enable Advanced Encryption

In V11.5 and later, c-tree supports enabling advanced encryption at run time using an environment variable. Set the environment variable CTREE_ADVANCED_ENCRYPTION to YES to enable advanced encryption if it is supported. This environment variable can be used to allow c-tree utilities to enable advanced encryption even if they haven't been updated yet to automatically enable advanced encryption when needed. Examples include the rebuild and compact utilities, ctrbldif and ctcmpcif.

Note: If c-tree does not support advanced encryption and this environment variable is set, the c-tree initialization will fail.

Considerations

Even without the -x8 option, some extended create block settings such as the huge file, extended header, and 6-byte transaction number options, are always applied to new files.
This option requires all associated index files that are referenced in the data file's IFIL structure to exist, as the utilities use OpenIFile() to open the data file and all associated index files to read the extended create block values.
When an application calls the standalone version of the Xtd8 file compact or rebuild functions (for example, CMPIFILX8() or RBLIFILX8()), index files created by the compact or rebuild no longer have the 6-byte transaction number attribute enabled if the specified extended create block's x8mode field has the ctNO6BTRAN bit set.
A rebuild or compact will fail with the new error code TFLN_ERR (943) when a client library that supports the new rebuild or compact option format attempts to use this feature with a FairCom Server that does not support this new format.

See Also:

Preventing Possible Data Loss with Compact & Rebuild Operations

Option to set index's automatic segment attributes

If a segmented index file was deleted and rebuilt using the ctrbldif utility or the rebuild API function, the new index file was not segmented. The index file's segment attributes, which are stored in the index file, were lost when the index file was deleted.

Beginning with V10.3, an option is provided with the rebuild and compact utilities that causes index files that are created by the utilities to be created using automatic segments of the specified size and maximum. The option is:

-idxseg=M@S

where M is the maximum number of segments and S is the segment size. S is interpreted as megabytes by default. Two suffixes, MB and GB, are also supported. MB indicates that the value is in megabytes and GB indicates that the value is in gigabytes.

Examples:

50 segments of 10 MB each:

ctrbldif test.dat -idxseg=50@10

100 segments of 300 MB each:

ctrbldif test.dat -idxseg=100@300MB

20 segments of 3 GB each:

ctrbldif test.dat -idxseg=20@3GB

Note: If the index files exist, the -idxseg option has no effect. It is only when the rebuild or compact utility creates new index files because the old index files do not exist that the -idxseg option affects the automatic segment properties of the index files.

-oldsec - Updates in handling of security attributes

In V10.3 and later, the ctrbldif utility assigns data file owner/group/permissions to index files and ctcmpcif assigns original data file owner/group/permissions to compacted data/index files.

ctrbldif mark.dat -oldsec ADMIN ADMIN FAIRCOMS

ctsmon - Server Heartbeat Monitor

ctsmon is used to check the current running status of a FairCom DB Server. Frequently, this is the "heartbeat" monitoring component in a clustering failover solution.

Usage

ctsmon [-u name] [-p pw] [-s s1,s2,...] [-t nn] [-c nn] [-d nn] [-o api_option]

where

-u name - User name to connect to the server with. Default is none.
-p pw - User password. Default is none.

It is assumed that all servers in the server list (-s) have the same user ID and password.

-s Server_name - Name of the Faircom server to log into.

The server name has the form of name[@host]. If no @host is given the local host is assumed. @host can also be given as using an I/P address Example: FAIRCOMS@192.168.1.1 Default is FAIRCOMS

-t nn - Time out value in seconds. Default is 30 seconds.
-c nn - Loop count for checking the server. This parameter has three possible values:
- +nn - Exit after nn times regardless if the server is up or down.
- 00 - Loop forever regardless if the server is up or down.
- -nn - Loop forever while the server is up, if unable to log into the server, exit after nn loops.
-d nn - Set debugging trace level to n (0 to 5).
-o api_option - An optional API call to retrieve additional information from server demonstrating the server is functional.

api_option is one of the following:

appname - First APP_NAME_LIST entry
curtime - Current server time in seconds since 1970
commits - Current number of database commits (ctGSMS.sct_trend)

Return status

-1 - Usage error, or internal error.

0 - Server responded.

+n - Error returned from InitISAMXtd call.

Description

This utility is intended for use with cluster management systems such as the Open Cluster Framework (OCF) and Pacemaker on Linux, and Solaris and Microsoft cluster managers. This is usually run in the background as a daemon process. It is a useful utility to use with any monitoring the current availability status of a FairCom DB server process.

The availability check is a basic connection/disconnection attempt. If the connection succeeds, it is assumed the server is active and available. If a further check such as the ability to retrieve data is required, the api_option can be used to trigger that request.

The source code of this utility is provided in the FairCom DB SDK and can be further customized for specific usages.

Example

ctsmon -u ADMIN -p ADMIN -s 10.6.1.1 -t 20 -c 3

ctsqlcdb - FairCom DB SQL Database Maintenance Utility

Operational Model:

Client

Usage

ctsqlcdb <command with args> [<servername>] [<user> <password>]]

Valid Commands:

-add <dbname>: Adds a reference to an existing database
-addctdb <dbname> <path>: Adds a reference to an existing CTDB database located at <path>
-create <dbname> [-casesensitive|-cs|-caseinsensitive|-ci]: Creates a new database
- -casesensitive|-cs: Sets database as case sensitive (default)
- -caseinsensitive|ci: Sets database as case insensitive
-create_preimage: Creates a preimage only database (supporting only atomicity; no transaction logs for durability.)
-drop <dbname>: Removes a reference to an existing database
-exist <dbname>: Returns 1 if the database exists, 0 if not, 2 on error
-list <servername>: Lists databases available from the server

Copy an existing database and add a reference to the new copy:

-copy <dbname> <newname> [<servername>]

-copy also includes non c-tree files when present in the database directory area.

where:

<dbname> is the database name, and
<servername> is the optional FairCom DB SQL name.

The ‑copy command supports virtual tables (called "Multi Record Tables" or "MRT tables").

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

ctsqlimp - SQL Import Utility

The FairCom DB SQL import utility, ctsqlimp, utility “registers” or links existing c-tree files with a FairCom DB SQL database without modifying the table structure such that the files remain accessible from the original application and also accessible from FairCom DB SQL.

For more information about this utility, see ctsqlimp - SQL Import Utility.

Operational Model:

Client

ctsqlimp is used as follows (after creating the database file):

ctsqlimp.exe <filename> [options]

Valid [options] values:

<filename> - data file name/path (relative paths to FairCom DB directory)
-h - display usage help
-d database - database name (default: ctreeSQL)
-s server - FairCom DB SQL Server name (default: FAIRCOMS)
-u userid - userid for logging into FairCom DB SQL
-a password - user password for authorization
-n symbolic - set SQL table name to symbolic
-o userid - set owner of table to userid
-i - non-interactive mode: ignore errors and continue
-r - remove table from dictionaries (file is not deleted)
-k - skip fields that don't comply with conventional identifiers rules
-c - allow table names not complying with conventional identifiers rules
-x - skip indexes
-p - promote unsigned integer to greater signed type
-P - promote unsigned types to greater signed type and set check for fitting value
-t - do not promote unsigned types to greater signed type
-m idxname - set index idxname as primary key
-z - allow indexes with missing string terminator in key segments
-l <size> - specify LONGVAR* field size threshold
-g - ignore existing index name in IFIL resource
-b - grant public access permissions to linked table
-B - grant read-only permissions on the table to the public

Note: If both -B and -b are specified, the read-only setting takes precedence. Notice that the owner of the table and the DBA have all the permissions.
This introduces a change in behavior for existing applications because this switch is now-case sensitive.

-j - non-interactive relink of existing table
-w script - write CREATE statements into script file script instead of importing the table
-e xmlfile - get DODA definitions from external XML file xmlfile
-f s(f) | z(f) | sz(f) - force string padding to (s)paces (z)eroes or (sz)spaces zero terminated. (f)orce presence of field delimiter
-l <size> - specify LONGVAR* field size threshold
-q prefix - prefix SQL table name with prefix (when the -q option is combined with the symbolic table name option, -n, the prefix is prepended to the symbolic name instead to the table name)
-y - keep existing permissions and synonym
--rowid_fld fldname - expose the ROWID value as field with name (table must have $ROWID$ field)
--rowid_idx idxname - expose index on the ROWID value with name (--rowid_fld option must be specified)
--tls - use a TLS connection with no certificate check (mutually exclusive with --tls_cert)
--tls_cert <cert> - use a TLS connection with certificate 'cert' (mutually exclusive with --tls)
--ignore_index_case - ignore case in segment modes when linking the index (this may cause incorrect query result)

Note: The parameters are case-sensitive. By default they are lowercase unless otherwise stated.

Example

To make existing c-tree files mydata.dat (containing proper IFIL and DODA resources) and mydata.idx accessible via FairCom DB SQL, follow these steps:

Create a database named ctreeSQL.
Copy your ISAM custmast.dat and custmast.idx files into the ctreeSQL.dbs subdirectory of the server's working directory.
Ensure FairCom DB SQL is running, as ctsqlimp is a client application.
Run the ctsqlimp utility found in the /tools/cmdline/admin/client/ directory of your FairCom DB installation:

ctsqlimp custmast.dat -u ADMIN -a ADMIN -s FAIRCOMS

Run the Interactive SQL, isql, utility and issue the command:

SELECT * FROM custmast;

Unicode

In V11.8 and later, the SQL import logic (affecting the ctsqlimp utility, the ctSqlImportTable() function, FairCom RTG sqllink and sqlize) for Unicode has been updated to take into consideration the string encoding set at the FairCom DB API level by the ctdbSetFieldStringEncoding() function. When the SQL import logic identifies a CHAR or a VARCHAR field and the string encoding has been set, the logic uses it to set the charset in the system table. If not present, the charset is not set (which was the behavior before this modification) and so SQL interprets the content as Latin‑1.

ctsqlutl - c-treeSQL Maintenance Utility

Operational Model:

Client

The FairCom DB SQL Maintenance Utility, ctsqlutl, is a general-purpose program to perform maintenance on the FairCom DB SQL Server.

The "rename column" (-rencol ) command allows you to change the name of a column. When the ctsqlutl utility is launched with the -rencol command, the program browses through the internal FairCom DB SQL dictionaries and updates the records with the new column name.

Syntax

The ctsqlutl utility syntax is as follows:

ctsqlutl [options] -rencol table_name column newcolumn

table_name: Name of the table
column: Current name of the column you are going to rename
newcolumn: Name of the column after renaming

Options

-o owner_name: Owner of table
-d database_name: Database name (default: ctreeSQL)
-s server_name: c-treeSQL Server name (default: FAIRCOMS)
-u userid: userid for logging onto the c-treeSQL Server
-a password: Password for authentication
-h: Display usage help

ctstat - Statistics Utility

The FairCom DB Statistics Utility, ctstat, is a client utility used to display statistics collected by FairCom DB. This utility provides valuable real-time monitoring of critical FairCom DB operations.

Operational Model:

Client

Usage

# ctstat report_type [-s svn] [-u uid] [-p upw]
[-i int [cnt]] [-h frq] [-d] [-m] [-t]

Reports:

-clf	Get the CHECKLOCK_FILE list entries
-deleted_files	Deleted Files Report
-file [csv]	File Activity Report
-filelist listname	List the entries on the specified file list
-filelocks file [N]	List all locks on a data file. Display Nth key
-fileops	File Operation Activity Report
-func	Function Timing Report
-funcfile	Function timing by File Report
-iotime on\|off	Turn disk I/O call timing on or off
-isam	ISAM Activity Report
-ma logfile	Log aggregate memory allocations to the specified file
-meminfo	Server Memory Info Report
-mf logfile	Log all memory allocations to the specified file
-ml	Get current memory allocation statistics
-mr min,max	Log only memory allocation statistics
-ms	Output memory allocation statistics
-mt options	Set memory allocation tracking options. Examples: -mt +ALL -mt +PI8TYP, -MBATYP
-mu	Unload module debug symbols
-sql	SQL Activity Report
-text	System Activity Report
-tranlog_flush	Transaction Log Flush Fixed Histogram Report
-userinfo	User Report with stats from USERINFO() function
-userinfox	User Report with stats from USERINFOX() function
-userlocks user	List all locks held by a user
-vaf file...	Admin-File Report
-vah handle...	Admin-User Report by User Handle
-var	Admin-Replication Reader Report
-vas	Admin-System Report
-vat	Admin-Transaction Report
-vau user...	Admin-User Report by User Name
-vtf file...	Tivoli-File Report
-vts	Tivoli-System Report
-wrktime on\|off\|reset	Turn function call timing on or off, or reset statistics

Options:

-d	Show cache stats as delta
-h frq	Print a description header every frq outputs
-i int [cnt]	Pause int seconds for optional cnt times
-I int [cnt]	Pause int seconds also before first row for optional cnt times
-m	Show memory file stats when using -vaf report
-p upw	User password
-s svn	c-tree Server name
-t	Output timestamp with header
-u uid	User name

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

-filelocks Notes

The -filelocks option lists all locks on a data file and, optionally, displays the Nth key. The lock offset and the associated keys are not read at the same time. Since we are reading records locked by other users to generate the key, there is no guaranteed relationship between the lock and the displayed key. The following are possible scenarios:

The displayed key is from before or after any changes made by the lock holder.
The locked offset no longer holds a valid record (it has been deleted, or updated and moved).
The locked offset could have been locked/modified/unlocked more than once between the time the lock offset was acquired and the time the record is read, so the offset could hold an entirely different record than what was originally locked.

The -filelocks file [key] command supports c-tree's standard wildcard filename matching for the specified file, allowing locks from multiple files to be displayed. The standard wildcards (used by ctsrvr.cfg keywords such as MEMORY_FILE and REPLICATE, etc) are:

* - Multi-character match

? - Single-character match

^ - Negation (must be first character)

-fileops Notes

Database resource consumption requires close monitoring of all file I/O activities. FairCom DB tracks values for additional physical file operations. The following values are tracked:

logical file opens
logical file closes
physical file opens
physical file closes
file creates
file renames
file deletes

Each value is a cumulative value since server startup, stored as an 8-byte integer field in the system snapshot structure.

The counters include every call, not just successful calls. Note that the physical file open count can be smaller than the physical file close count, because when a file is created it does not increment the physical file open count.

The ctstat utility supports a new option, -fileops, that displays these counters. Example:

ctstat -fileops -t -i 1 -h 10 -u ADMIN -p ADMIN -s FAIRCOMS

logopn/s logcls/s phyopn/s phycls/s filcre/s filren/s fildel/s total/s

0 0 0 0 0 0 0 0

154 158 23 26 13 0 3 377

959 926 70 63 25 0 4 2047

1016 989 4 0 0 1 1 2011

574 562 0 0 0 0 0 1136

481 486 2 0 0 0 0 969

425 417 0 0 0 0 0 842

376 380 0 0 0 0 0 756

The ctstat -fileops option requires a server that uses snapshot version 21 or later. If this option is used with a server that uses an earlier version of the utility, the utility fails with the error message:

Error: The -fileops option requires snapshot version 21, but this server is using snapshot version <version>.

where <version> is the older snapshot version that FairCom DB is using.

The snapshot file parsing utility, ctsnpr, has been updated to support the new SNAPSHOT.FCS file format.

-tranlog_flush notes:

The -tranlog_flush displays the transaction log flush histogram values. Use the -d option to display delta values. Otherwise, ctstat displays the cumulative histogram values. For example:

C:\> ctstat -tranlog_flush -h 1 -t -d -u ADMIN -p ADMIN -s FAIRCOMS -d

Fri Oct 4 16:24:30 2024

Since server startup:

tranlog flush count: 21761

tranlog flush time: 25.737986

tranlog flush average: 0.001183

Since last checkpoint:

tranlog flush count: 2

tranlog flush time: 0.007201

tranlog flush average: 0.003600

Since last sample:

tranlog flush count: 324

tranlog flush time: 0.495657

tranlog flush average: 0.001530

adaptive histogram adjustments: 3

time since most recent adjustment: 0

time since fixed histogram last cleared: 18

fixed histogram adaptive histogram

count time count time

132 < 0.001000 0 < 0.000031

0 < 0.002000 0 < 0.000062

192 < 0.003000 0 < 0.000093

0 < 0.004000 0 < 0.000124

1 < 0.005000 0 < 0.000155

0 < 0.006000 0 < 0.000186

0 < 0.007000 0 < 0.000217

0 < 0.008000 0 < 0.000248

0 < 0.009000 0 < 0.000279

0 < 0.010000 0 < 0.000310

0 < 0.011000 0 < 0.000341

0 < 0.012000 0 < 0.000372

0 < 0.013000 0 < 0.000403

0 < 0.014000 0 < 0.000434

0 < 0.015000 0 < 0.000465

0 < 0.016000 0 < 0.000496

0 < 0.017000 0 < 0.000527

0 < 0.018000 0 < 0.000558

0 < 0.019000 0 < 0.000589

0 < 0.020000 0 < 0.000620

0 < 0.021000 1 < 0.000651

0 < 0.022000 0 < 0.000682

0 < 0.040000 [0 avg] 1 < 0.001240 [0 avg]

0 >= 0.040000 0 >= 0.001240

-userinfox Notes

In V11.6 and later, the -userinfox option displays extended connection information. For example:

ctstat -userinfox -u ADMIN -p ADMIN -s FAIRCOMS -h 1 -t -i 1

status lastrequest trntime mem fils time uid/tid/nodename commprotocol readops readbytes writeops writebytes datarqsts datahits indexrqsts indexhits

0s busy TRANEND 0s 96K 1 3s CTQA/21/(2) ctqa FSHAREMM 643 35831372 706 16111449 18569 17930 2535 2535

0s busy USERINFOX -- 76K 2 11s ADMIN/22/ctstat FSHAREMM 4 16544 0 0 12 11 0 0

0s busy TRANEND 0s 70K 1 3s CTQA/23/(3) ctqa FSHAREMM 566 31750812 507 14227936 23180 22614 1467 1467

0s busy FRSVREC 0s 75K 1 3s CTQA/24/(4) ctqa FSHAREMM 754 41842916 520 16605870 32476 31722 2484 2484

0s busy FRSVREC 0s 90K 1 3s CTQA/25/(5) ctqa FSHAREMM 833 46135384 649 19387611 25274 24441 3602 3602

0s busy TRANEND 0s 94K 1 3s CTQA/26/(6) ctqa FSHAREMM 760 43814980 548 16663966 28098 27339 2700 2700

2s busy RRDREC 2s 236K 5 3s CTQA/27/(13) ctqa FSHAREMM 560 11343722 644 3677007 853 793 2033 1898

0s busy FRSREC 0s 253K 2 3s CTQA/28/(8) ctqa FSHAREMM 819 52887780 1690 53585664 40324 39518 6342 6341

0s busy TRANABT 0s 260K 2 3s CTQA/29/(9) ctqa FSHAREMM 776 50069732 1456 45076587 38689 37925 5968 5968

0s busy TRANEND 0s 183K 2 3s CTQA/30/(10) ctqa FSHAREMM 0 0 2198 44113920 0 0 776 776

0s busy NXTREC 0s 2956K 1 2s CTQA/31/(18) ctqa FSHAREMM 455 29557760 361 15667386 10848 10397 2 2

2s busy BATSETX 3s 16950K 6 3s CTQA/32/(12) ctqa FSHAREMM 17139 142302504 45462 42391289 10232 8130 80823 80756

-userlocks Notes

For the -userlocks report:

If UserID is a number, it is interpreted as a task ID.
If UserID is a string, it is interpreted as a name, and information on locks held by each task ID with a matching name is returned.

Because the -userlocks report may generate a large number of server calls (for each task ID and file), the -userlocks report interval may be increased up to 60 seconds, depending on the number of matching users and files involved.

Filelocks and Userlocks Note: Dumping large quantities of locks in a very active system could affect performance.

List files on internal server lists

In V12 and later, ctstat provides a list of files on specific internal server lists, such as ctKEEPOPEN.

Example

C:\> ctstat ‑filelist keepopen_list ‑u ADMIN ‑p ADMIN ‑d ‑h 1 ‑t ‑s FAIRCOMS

Fri Dec 20 09:29:39 2019

value member scale filename

1 0 0 mark.dat

-isam header changed to avoid misunderstanding

The header of the ctstat -isam command displays a /s suffix to indicate that the statistics are per second. For example, the following syntax will produce the output below:

ctstat.exe -s FAIRCOMS -u ADMIN -p ADMIN -h 2 -isam

add/s del/s upd/s read/s total/s

0 0 0 0 0

add/s del/s upd/s read/s total/s

0 0 0 0 0

Note: This constitutes a Compatibility Change.

Admin-System Report -vas

The admin-system (-vas) report displays FairCom DB Server system-wide statistics in the areas of cache usage, disk I/O, open files, established client connections, file locks, and transactions.

Example

Below is a sample admin-system report produced by executing the command:

ctstat -vas -u ADMIN -p ADMIN -s FAIRCOMS -h 10 -i 2


	cache			disk i/o		files	connect		locks		transactions
d%h	%m	i%h	%m	r/s	w/s	cur/max	cur/max	cur	l%h	%m	dead	act	t/s	r/t	w/t
99	1	99	1	1	1	2581/10000	2/128	0	100	0	0	1	36	0	0
99	1	99	1	2	0	2581/10000	2/128	0	100	0	0	1	35	0	0
99	1	99	1	2	0	2581/10000	2/128	0	100	0	0	1	28	0	0
99	1	99	1	0	0	2581/10000	2/128	0	100	0	0	1	22	0	0

The columns shown in this report are described as follows:

d%h Data cache hit rate

%m Data cache miss rate [100 - Data cache hit rate]

i%h Index cache hit rate

%m Index cache miss rate [100 - Index cache hit rate]

r/s Disk reads per second

w/s Disk writes per second

cur Current number of open files

max Server limit on number of open files

cur Current number of client connections

max Server limit on number of client connections

cur Number of locks currently held

l%h Lock hit rate [(lock attempts - locks blocked - locks denied) / lock

attempts]

%m Lock miss rate [100 - Lock hit rate]

dead Number of dead locks detected

act Current number of active transactions

t/s Number of transactions per second

r/t Number of read operations per transaction

w/t Number of write operations per transaction

log save Avg log save time per interval (microseconds) when used with the delta (-d) option

Tivoli-System Report -vts

The Tivoli-system (-vts) report displays FairCom DB Server system-wide statistics in the areas of cache usage, disk I/O, open files, established client connections, file locks, and transactions. The Tivoli-system report displays much of the same statistics that the admin-system (-vas) report displays, but in a format appropriate for input to tools such as the Tivoli monitoring application.

Example

Below is a sample Tivoli-system report produced by executing the command:

ctstat -vts -u ADMIN -p ADMIN -s FAIRCOMS -h 10 -i 2

#%cachehit %cachemiss r/s w/s maxfiles openfiles totalconnections activetransactions numdeadlock trans-r/s trans-w/s %hashhit %hashmiss transactions/s

92 8 0 0 10000 18 1 0 0 0 0 100 0 0

92 8 0 9 10000 18 1 0 0 0 17 100 0 1

92 8 0 0 10000 18 1 0 0 0 0 100 0 1

92 8 0 1 10000 18 1 0 0 0 1 100 0 1

92 8 0 0 10000 18 1 0 0 0 0 100 0 1

Note: The header line shown in this example is written as a single output line although it may be shown on multiple lines here.

The columns shown in this report are described as follows:

%cachehit Data and index cache hit rate

%cachemiss Data and index cache miss rate

r/s Disk reads per second

w/s Disk writes per second

maxfiles Server limit on number of open files

openfiles Current number of open files

totalconnections Current number of client connections

activetransactions Current number of active transactions

numdeadlock Number of dead locks detected

trans-r/s Number of read operations per transaction

trans-w/s Number of write operations per transaction

%hashhit Lock hit rate

%hashmiss Lock miss rate

transactions/s Number of transactions per second

Admin-File Report -vaf

The admin-file (-vaf) report displays FairCom DB Server statistics for the specified file. Note that multiple data or index files can be specified on the command line. Below is a sample admin-file report produced by executing the command:

ctstat -vaf mark.dat mark.idx -u ADMIN -p ADMIN -s FAIRCOMS -h 10

r/s w/s entries locks l%h %m dlock recrd node t filename

2 4 11863 4 100 0 0 128 n/a F mark.dat

0 4 11863 2 96 4 0 n/a 32768 I mark.idx

1 3 12192 4 100 0 0 128 n/a F mark.dat

0 9 12192 3 97 3 0 n/a 32768 I mark.idx

2 4 12730 5 100 0 0 128 n/a F mark.dat

0 3 12730 1 97 3 0 n/a 32768 I mark.idx

2 4 13236 5 100 0 0 128 n/a F mark.dat

0 2 13236 0 97 3 0 n/a 32768 I mark.idx

The columns shown in this report are described as follows:

r/s Disk reads per second for the file

w/s Disk writes per second for the file

entries Number of data records or key values in file

locks Number of locks currently held on file

l%h Lock hit rate for the file

%m Lock miss rate for the file

dlock Number of dead locks detected for the file

recrd Record length if data file, otherwise n/a

node Node size if index, otherwise n/a

t File type (F=fixed-length data, V=variable-length data, I=index)

filename Name of the file

Tivoli-File Report -vtf

The Tivoli-file (-vtf) report displays FairCom DB Server statistics for the specified file in a format appropriate for input to tools such as the Tivoli monitoring application.

Below is a sample Tivoli-file report produced by executing the command:

ctstat -vtf mark.dat mark.idx -u ADMIN -p ADMIN -s FAIRCOMS -h 10

#r/s w/s currentlocks waitinglocks filename

0 0 5 0 mark.dat

0 0 1 1034 mark.idx

1 3 4 0 mark.dat

0 6 0 1120 mark.idx

3 5 5 0 mark.dat

0 0 0 1208 mark.idx

2 4 4 0 mark.dat

0 0 2 1324 mark.idx

2 4 5 0 mark.dat

0 3 2 1402 mark.idx

The columns shown in this report are described as follows:

r/s Disk reads per second for the file

w/s Disk writes per second for the file

currentlocks Number of locks currently held on file

waitinglocks Cumulative lock wait count

filename Name of the file

Admin-User Report -vau

The admin-user report, -vau user..., displays FairCom DB Server statistics for the specified users. All existing connections whose user ID match the specified user ID are displayed.

Example

Below is a sample admin-user report produced by executing the command:

ctstat -vau GUEST -u ADMIN -p ADMIN -s FAIRCOMS -h 10

log function sec fil lok l%h %m dlock tid/uid/nodename

7s TRANEND 0 2 1 98 2 0 GUEST/10/

7s ADDREC 0 2 2 98 2 0 GUEST/12/

7s ADDREC 0 2 1 98 2 0 GUEST/13/

7s ADDREC 0 2 0 98 2 0 GUEST/14/

0s ctSNAPSHOT 0 2 0 0 0 0 ADMIN/15/ctstat

7s ADDREC 0 2 0 98 2 0 GUEST/17/

The columns shown in this report are described as follows:

log Total logon time for client

function Function client is currently executing

sec Current function request time

fil Current number of files open by this client

lok Current number of locks held by this client

l%h Lock hit rate for this client

%m Lock miss rate for this client

dlock Number of deadlocks detected for this client

tid/uid/nodename Server thread ID/User ID/Node name for this client

Function Timing Report -func

The function timing report (-func) displays FairCom DB Server statistics for each c-tree function that a client has called at least once since function timing was enabled. See -wrktime.

Below is a sample function timing report produced by executing the command:

ctstat -func -u ADMIN -p ADMIN -s FAIRCOMS -h 10

function counter secs

FRSVSET 10 0.002

NXTVSET 20 0.001

GETDODAX 2 0.000

COMMBUF 1 0.000

ctSNAPSHOT 10 0.002

The columns shown in this report are described as follows:

function Function name

counter Cumulative number of times this function has been called

secs Cumulative elapsed time for this function

Text Report -text

The following command generates a report in text format:

ctstat -text -u ADMIN -p ADMIN -s FAIRCOMS -h 10

This command writes a FairCom DB Server system snapshot to the file SNAPSHOT.FCS (output to the data folder). See the file SNAPSHOT.FCS for the detailed server statistics.

Example Output (truncated for brevity)

Server Shutdown System Snapshot

Wed May 22 15:36:41 2019

server version of structure: 21

bit map of var len contents: 0x

snapshot time stamp (seconds since 1970): 1558557401

elapsed server operation time: 173106.444

high res timer ticks per sec: 10000000

compatibility flag #1: 00002000x

compatibility flag #2: 00020000x

compatibility flag #3: 00000000x

compatibility flag #4: 00000000x

diagnostic flag #1: 40000000x

diagnostic flag #2: 00000000x

diagnostic flag #3: 00000000x

ct_udefer yield threshold (usec): 0

ct_udefer 64 yield duration (usec): 0

delete node thread queue reads: 2908

delete node thread queue writes: 2908

delete node thread queue rewrites: 0

delete node thread queue abandons: 0

delete node thread queue removals: 0

delete node thread queue no action: 0

count of lock attempts: 21708

subcount of hdr lock attempts: 2135

count of locks freed: 21707

count of locks denied: 0

count of locks blocked: 0

subcount of header blocks: 0

count of blocks released: 0

count of dead locks: 0

count of killed locks: 0

current count of locks held: 1

current count of blocked requests: 0

maximum count of locks held: 161

COMMIT LOCK loop defer (millisecs): 10

COMMIT LOCK loopers: 0

COMMIT LOCK total loop count: 0

cumulative blk record lock count: 0

cumulative blk record lock time: 0.000 (seconds)

maximum blk record lock time: 0.0000

cumulative blk index lock count: 0

cumulative blk index lock time: 0.000 (seconds)

maximum blk index lock time: 0.0000

cumulative transaction count: 369

cumulative transaction time: 534.255 (seconds)

average transaction time: 1.44784

transaction time detail:

122 < 0.05000

8 < 0.10000

5 < 0.15000

3 < 0.20000

16 < 0.25000

63 < 0.30000

69 < 0.35000

20 < 0.40000

11 < 0.45000

12 < 0.50000

5 < 0.55000

2 < 0.60000

1 < 0.65000

0 < 0.70000

0 < 0.75000

0 < 0.80000

1 < 0.85000

0 < 0.90000

0 < 0.95000

0 < 1.00000

1 < 1.05000

0 < 1.10000

1 < 2.00000 [avg of 0 per unit width]

29 >= 2.00000

maximum transaction time: 102.2966

checkpoint delay (microseconds): 0

commit delay block to clear ration: 2

commit delay cohort size base: 50

commit delay nominal time (microseconds): 2000

I/O Time Statistics -iotime

The FairCom DB Server SNAPSHOT feature includes support for collecting disk read and write timings on a per-file basis when high-resolution timer support is activated. Use the ctstat utility’s -iotime option to toggle the collection of disk I/O timings.

Turn on disk I/O timings:

# ctstat -iotime on -u ADMIN -p ADMIN -s FAIRCOMS

Turn off disk I/O timings:

# ctstat -iotime off -u ADMIN -p ADMIN -s FAIRCOMS

The ctstat utility’s -vaf option also outputs differential I/O timings for each file when the FairCom DB Server returns version 2 (or higher) GFMS structure statistics.

Example

C:\> ctstat -vaf mark.dat mark.idx -h 1 -i 10

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

0 0 0 0 26239 1 100 0 0 128 n/a F mark.dat

0 0 0 0 26239 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

128 237 1 12 108309 2 100 0 0 128 n/a F mark.dat

0 2 0 0 108308 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

121 243 1 14 186164 2 100 0 0 128 n/a F mark.dat

2 27 0 4 186163 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

109 219 1 10 256356 2 100 0 0 128 n/a F mark.dat

3247 3296 39 77 256355 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

103 206 1 10 322381 4 100 0 0 128 n/a F mark.dat

5623 5640 67 124 322380 1 99 1 0 n/a 8192 I mark.idx

I/O Statistics per File -file

The FairCom DB Server SNAPSHOT feature supports a mode to write snapshot statistics for all files open by the FairCom DB Server to disk. Use the ctstat utility’s -file option. The snapshot statistics for all open files are then written to the SNAPFILE.FCS file in comma-delimited or human-readable format.

CSV Example

Write statistics for all open files to SNAPFILE.FCS in comma-delimited format using the ctstat utility:

# ctstat -file csv -i 1 1

Sample SNAPFILE.FCS Contents

On-Demand File Snapshot

Mon Jun 25 16:40:51 2007

physical file size,logical file size,serial number,active entries,tran high mark,update timestamp,max file size,read ops,bytes read,write ops,bytes written,memory file high bytes,read time (msec.),write time (msec.),index height,file id,server id,time id,node size,record length,permanent file mode,max leaf key bytes,max non-leaf key bytes,file type,key length,key member number,number of members, super file type,max leaf marks,wrthdr sequence number,total lock attempts,header lock attempts,total lock wait count,header lock wait count,deadlocks,total locks denied,total locks freed,total blocks released,current locks held,current blocked requests,max special cache pages,current special cache pages,number of buffer pages,number of data cache pages,number of channels,number of users with file open,current memory record count,highest memory record count,killed locks,max segments,active segments,update flag,file type,duplicate key flag,index delete type,key padding byte,flavor,alignment,pointer size,file name

16384,16384,0,15,0,0,0,0,0,5,16768,0,0,0,-1,0x00000000,0x00000000,0x00000000,8192,0,0x00000000,8148,8174,1,12,0,0,0,2048,3,44,23,0,0,0,0,44,0,0,0,0,0,0,0,1,1,0,0,0,1,0,0xff,0,0,0,32,2,8,4,I0000001.FCS

====================

Human Readable Example

Write statistics for all open files to SNAPFILE.FCS in human-readable format using the ctstat utility:

# ctstat -file -i 1 1

Sample SNAPFILE.FCS Contents

On-Demand File Snapshot

Tue Jun 26 09:58:28 2007

phyrec numrec sernum nument hghtrn tstamp mxfilz fredops fredbyt fwrtopts fwrtbyt mhghbyt fredtim fwrttim idxhgt fileid servid timeid nodsiz reclen logtyp maxkbl maxkbn filtyp keylen kmem nmem suptyp maxmrk hdrseq floktry flokhlk flokblk flokhbk flokdlk flokdny flokfre flokrel flokcur fblkcur datlmt datspl bufcnt datcnt numchn fusrcnt memcnt hghcnt flokkil segmax seglst updflg ktype autodup deltyp keypad flflvr flalgn flpntr filename

16384 16384 0 15 0 0 0 1 8192 10 57728 0 0 0 -1 0x00000000 0x00000000 0x00000000 8192 0 0x00000000 8148 8174 1 12 0 0 0 2048 3 68 35 0 0 0 0 68 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0xff 0 0 0 32 2 8 4 I0000001.FCS

====================

Connection Information -userinfo and -userinfox

Connection reporting options -userinfo and -userinfox are available to display additional statistics about existing user connections.

New information included in this alternative output:

The status and idle time of the connection.
The last FairCom DB Server request made.
Time spent in a transaction.
Amount of memory consumed by the client.
Number of files open by the client.
The time the user has been logged in.
User ID, Thread ID, and Nodename of the user.
User communication protocol used to connect to the FairCom DB Server. Because the column labeled uid/tid/nodename is not fixed in size, the space between nodename and commprotocol is a tab character. Even if the output is not properly aligned, the parsing will be easy by looking for a '\t' (tab).

-userinfo Example

ctstat.exe -userinfo -u ADMIN -p ADMIN -s FAIRCOMS -h 5

status lastrequest trntime mem fils time uid/tid/nodename commprotocol

0s busy ADDVREC 1s 596K 46 11s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m00s idle tmexec -- 7626K 7 58m00s ADMIN/23/SLCDESK SQL_SHAREMM

0s busy ADDVREC 1s 189K 46 11s ADMIN/25/cttctx(p26576,t9220) FSHAREMM

0s busy DELVREC 0s 185K 46 11s ADMIN/26/cttctx(p26576,t31684) FSHAREMM

0s busy DELVREC 0s 186K 46 11s ADMIN/27/cttctx(p26576,t28564) FSHAREMM

0s busy DELVREC 0s 187K 46 11s ADMIN/28/cttctx(p26576,t27780) FSHAREMM

0s busy ADDVREC 0s 592K 46 12s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m01s idle tmexec -- 7626K 7 58m01s ADMIN/23/SLCDESK SQL_SHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/25/cttctx(p26576,t9220) FSHAREMM

0s busy ADDVREC 0s 185K 46 12s ADMIN/26/cttctx(p26576,t31684) FSHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/27/cttctx(p26576,t28564) FSHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/28/cttctx(p26576,t27780) FSHAREMM

0s busy DELVREC 0s 593K 46 13s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m02s idle tmexec -- 7626K 7 58m02s ADMIN/23/SLCDESK SQL_SHAREMM

-userinfox Example