FairCom — FairCom DB Utility Programs

FairCom DB utility programs provide mechanisms to maintain, recreate and reorganize your data and index files. Two programs are also provided to facilitate porting c-tree V4.3 data and index files to the latest FairCom DB formats.

If a utility program is run without any command-line arguments, either it will display the appropriate command line syntax or it will prompt you for the necessary information. Entries on the command line surrounded by square braces, [ ], are optional. Entries surrounded by angle braces, < >, indicate that a user-supplied value, an unquoted string or number, is to be entered. Other entries are literals that should be entered as shown. However, the literals may be in lower or upper case.

ctclntrn Utility - Clean Transaction Mark

Operational Model:

Standalone

Usage:

ctclntrn [-<page size in bytes>] <file name> [<password>]

Description

ctclntrn “cleans” the high-water transactions marks within a FairCom DB index.

Suggested Steps for Successful Clean and Reset

In the event of an impending transaction number overflow, the server administrator should follow these steps to restart transaction numbering:

Perform a clean FairCom DB shutdown.
Delete the transaction logs and housekeeping files: files S0000000.FCS, S0000001.FCS, D*.FCS, I*.FCS, and L*.FCS.
Use the ctclntrn utility to clean all indices (which resets the transaction numbers in the leaf nodes and index header) used by your application, including your application index files, superfiles, variable-length data, and c-tree Server files including the following if present:
- FAIRCOM.FCS -- If you are not using any User IDs or passwords other than ADMIN and GUEST, you can simply delete this file. (If you do delete this file, tFairCom DB will re-create it with the single user ADMIN and password ADMIN.)
- CTSYSCAT.FCS -- This is only present if you are using c-tree ODBC Drivers.
- SEQUENCEIX.FCS
- SYSLOGIX.FCS
- SYSLOG*.FCS
- ALL FairCom DB SQL Database Dictionaries
  - <databasename>.dbs/SQL_SYS/<databasename>.fdd
  - ctdbdict.fsd (session dictionary)
Restart the c-tree Server.

The server will create new transaction logs and start transaction numbering from 1 again.

It is important to run ctclntrn on all files. If you miss any files and later open that file with a large transaction number in its header, the server will again increase its transaction number to that large value. You will need to repeat this procedure should that occur.

FYI: The ctclntrn utility uses the CleanIndexXtd() FairCom DB function. This function cleans any leaf nodes of exceptional transaction marks and resets the transaction high-water mark in the header to zero. This avoids rebuilding the entire index file.

You can use the c-tree High-water Mark Utility, cthghtrn, to verify that the transaction high-water marks in the files are back to zero, or other reasonably low number.

Tip: The AUTO_CLNIDXX YES configuration option can help automate and detect files which have been missed.

See also

cthghtrn - Displays the high-water mark for transactions (cthghtrn - Displays the high-water mark for transactions, cthghtrn utility)

ctclosefile - Close Files on KEEPOPEN List

It is easy to create memory files or set persisted files in a ctKEEPOPEN state. However, those files need to be closed at times. The ctclosefile utility provides this functionality and can be used in a scripted environment. ctclosefile will close a specified FairCom DB memory file or any file held open with a ctKEEPOPEN directive.

ctclosefile [-u user] [-p password] [-s server][-f filename] [-c command] [-w file password ] {-a authfile}

Parameters:

-u - User ID

-p - User Password

-s - Server (default: FAIRCOMS@localhost)

-f - File name

-c - Command

-w - File password

-a - Authentication file name

The ctclosefile utility also supports adding and removing ctKEEPOPEN list entries. To use this feature, specify -c <command> where <command> is either add (to add the specified file name to the keepopen list) or remove (to remove the specified file name from the keepopen list).

Example

Add mark.dat to keepopen file list:

C:\> ctclosefile -u admin -p ADMIN -f mark.dat -c add

Successfully added file name ""mark.dat"" to keepopen list

ctcmpcif - IFIL-based Compact Utility

This utility can be used to encrypt and decrypt c-tree data files.

Operational Model:

Client
Standalone
FairCom DB Standalone SQL Service

Standalone Usage

ctcmpcif <data file name> <options> <user> <pass> <server> <-sectors>

Where:

Note: The order of parameters must be be as specified in the above usage descirption.

<-sectors> - The sector size to use. Enter a dash followed by the size of the sectors. The sector parameter is especially useful if you need to adjust a file’s PAGE_SIZE to match the FairCom Server. This option must go after <server>. The sector size is 256 is in the example below:

ctcmpcif xx8.dat ADMIN ADMIN FAIRCOMSMS -256

<options> can be:

-callback - Enable notifications of compact progress.
-compress - Create the compacted data file with compression (defaults to zlib library if not specified in server configuration).
-encrypt=cipher - Create the compacted file using the specified cipher. See the encryption sections below.
Accepted values:
- AES: aes16, aes24, or aes32
- Blowfish: blf8, blf9, ..., or blf56
- DES: des8, des16, or des24
- none
-flexrec - (Supported in V11.5 and later) Add Hot Alter Table support to the file.
-idxseg=<M>@<S> - If creating an index, use M automatic segments of size S. Size is specified in megabytes, or you can specify MB or GB as a suffix. (Example: -idxseg=10@1GB)
-local - causes the utility to use a standalone (local side) connection instead of a client connection when linked with a LOCLIB library.
-nocompress - Create the compacted data file without compression.
-noflexrec - (Supported in V11.5 and later) Remove Hot Alter Table support from the file.
-oldsec - Use the utility's original security attribute assignment.
-online - (In V13 and later) Defragment the file in ctSHARED mode. Online compact is not allowed for changing encryption or compression settings.
-purge - Delete records that have illegal duplicate key values.
-redosrl - Reassign serial numbers to the records in the compacted file instead of copying the values from the original file.
-repairdata - Attempt repair if damaged data records are detected.
‑sortmem=<N> - Set the size of sort memory to N KB in non-client builds. This option can be used to increase the rebuild speed for large files.
-temppath=<temporary_path> - Use the specified directory for temporary files.
-updifil - Update IFIL resources in the data file.
-x8 - Use extended create blocks read from data and index files.

Client-Side Usage

# ctcmpcif DataFileName [-purge] [-updifil] [<UserId>]
[<UserPassword> [<ServerName>]]

Where:

-purge - purge duplicate records.
-updifil - update IFIL resources in the data file.

UserID, UserPassword, and ServerName are only needed for client versions of this utility. FairCom recommends building this utility as a Single-user Standalone application.

ctcmpcif reads the IFIL structure from DataFileName and calls CompactIFileXtd() and RebuildIFileXtd() to compact and rebuild DataFileName and its associated indexes. If ctcmpcif cannot extract the IFIL from the target file, it will ask for the name of another copy of the file from which to extract the IFIL information.

Prior to FairCom DB V9.3, several option values were defined such as updateIFIL, purgeIFIL, and badpartIFIL, that can be specified. These values are specified by adding them together. For example: myifil.tfilno = updateIFIL + purgeIFIL. A new approach simplifies checking of these options. The following option values are now specified in ctport.h:

#define updateIFILoption 0x0002

#define purgeIFILoption 0x0004

#define badpartIFILoption 0x0008

#define redosrlIFILoption 0x0010

These values can now be OR-ed together and the negative of the result stored in the tfilno field of the IFIL structure passed to the compact or rebuild function. For example, to indicate that you want to assign new serial numbers, do the following:

myifil.tfilno = -redosrlIFILoption;

CMPIFIL(&myifil);

If you want to also use more than one option when compacting or rebuilding a file, OR them in to the value and negate the result. For example,

/* assign new serial numbers and update IFIL resource. */

myifil.tfilno = -(redosrlIFILoption | updateIFILoption);

CMPIFIL(&myifil);

ctcmpcif will attempt to open files of any page size definition. In addition, in standalone mode, it will create the new file based on the provided page size.

Note: This tool does not compact partitioned files. If you attempt to compact a partitioned file, the tool will return PCMP_ERR (954, compacting partitioned file not supported).

Changing Encryption Attributes

In V11 and later, the compact utility can optionally change the encryption attributes. To use this option, OR in the setencryptIFILoption bit into the tfilno field of the IFIL structure whose address you pass to the compact API function. When using this and other options, remember to negate the tfilno value after you OR in the options. For example:

myifil.tfilno = -(redosrlIFILoption | setencryptIFILoption);

CMPIFIL(&myifil);

Specifying the Encryption Cipher

In V11 and later, the ctcmpcif utility supports an option to specify the encryption cipher for the data and index files created by the compact operation. Usage:

-encrypt=cipher - Create the compacted file using the specified cipher:

for AES, use aes16, aes24, or aes32
for no encryption, use none

Note: If an index file does not exist, the original data file's encryption attributes are used when creating that index file.

To change the encryption attributes of a file using this ctcmpcif compact utility or the compact API function from a client, you must add the option CHANGE_ENCRYPTION_ON_COMPACT YES to ctsrvr.cfg. Otherwise, the operation fails with error NSUP_ERR (454, not supported).

Environment Variable to Enable Advanced Encryption

In V11.5 and later, c-tree supports enabling advanced encryption at run time using an environment variable. Set the environment variable CTREE_ADVANCED_ENCRYPTION to YES to enable advanced encryption if it is supported. This environment variable can be used to allow c-tree utilities to enable advanced encryption even if they haven't been updated yet to automatically enable advanced encryption when needed. Examples include the rebuild and compact utilities, ctrbldif and ctcmpcif.

Note: If c-tree does not support advanced encryption and this environment variable is set, the c-tree initialization will fail.

Note: In order to use advanced encryption, a master key must be supplied. The utility will prompt for the master key, or a master key file can be created like this:

1: create master key file:

ctcpyf -k mykey -s mykey.fkf

2: set environment variable to the name of the master key file:

CTREE_MASTER_KEY_FILE=mykey.fkf

If using these steps, then the utility will read the master key from the file instead of prompting for the key.

Considerations

Even without the -x8 option, some extended create block settings such as the huge file, extended header, and 6-byte transaction number options, are always applied to new files.
The -x8 option requires all associated index files that are referenced in the data file's IFIL structure to exist, as the utilities use OpenIFile() to open the data file and all associated index files to read the extended create block values.

When an application calls the standalone version of the Xtd8 file compact or rebuild functions (for example, CMPIFILX8() or RBLIFILX8()), index files created by the compact or rebuild no longer have the 6-byte transaction number attribute enabled if the specified extended create block's x8mode field has the ctNO6BTRAN bit set.

A rebuild or compact will fail with the new error code TFLN_ERR (943) when a client library that supports the new rebuild or compact option format attempts to use this feature with a FairCom Server that does not support this new format.
In V11 and later, the compact and rebuild API functions preserve the original files' encryption attributes by default. If the index files don't exist when compact/rebuild are called, they will be created with the data file's encryption attributes.
In V11 and later, the compact utility always deletes and recreates the index files after saving the resources from the original index file. This provides the expected behavior of (possibly) reducing the size of the index file.

See also

File Recovery in the FairCom DB Developer's Guide
Updates in handling of security attributes
Preventing Possible Data Loss with Compact & Rebuild Operations

-oldsec - Updates in handling of security attributes

In V10.3 and later, the ctrbldif utility assigns data file owner/group/permissions to index files and ctcmpcif assigns original data file owner/group/permissions to compacted data/index files.

The rebuild and compact utilities now read the permission mask, owner, and group settings from the original data file and after the rebuild or compact is completed, the utilities assign these same attributes to the new files.

A command-line option has been added to the utilities to restore the previous security attribute behavior. Use the -oldsec option to cause the rebuild and compact utilities to set the security attributes as they did before this revision. For example:

ctrbldif mark.dat -oldsec ADMIN ADMIN FAIRCOMS

The -oldsec option can be used if rebuild and compact are failing with error 455 (user does not belong to group) if you delete an index then run the rebuild or compact utility on the data file using a user account that does not belong to the group that is assigned to the data file.

Note: These changes only apply to the client and server versions of these utilities. Setting the security attributes is not supported in standalone mode, so the standalone rebuild and compact utilities behave as follows:

1) When the standalone mode rebuild utility creates new index files instead of reusing the new index files (for example if the original index files have been deleted before running the rebuild utility), the new index files are assigned a permission mask of zero (no restrictions on permissions), and the owner and group are unassigned (empty).

2) The standalone mode compact utility always preserves the security attributes of the data file. If the index files do not exist, the newly-created index files are assigned the same values as the standalone rebuild does when it creates new index files.

Preventing Possible Data Loss with Compact & Rebuild Operations

In most cases, compact and/or rebuild operations perform as expected: scanning your data file for potential invalid record headers, fixing those it finds invalid, and finally writing out a new compressed file and rebuilding indexes. Recently, a case was investigated where our default behavior should not be used due to potential data file corruption.

In a few rare and select cases, FairCom identified situations where compacting a variable-length and (typically) non-transaction enabled (non-TRNLOG) file, using the ctcmpcif compact utility or any of the CompactIFile functions, can fail, potentially, and unexpectedly, also altering your original file being compacted such that further attempts result in data loss.

Typically, it is only expected to see this situation occur with non-TRNLOG enabled files. However, in case #2 discussed below, a valid TRNLOG enabled file data file was found to exhibit this very unusual behavior. This raises an important point – if you encounter situations where you believe you must rebuild TRNLOG enabled files, please contact FairCom, as this is a highly unexpected event and we want to fully understand the context and nature of your data integrity situation. The recoverable nature of TRNLOG files should always maintain full data integrity through nearly any circumstance.

We have modified prior behavior to take a much more conservative approach protecting your original data state. Compact and rebuild (which can produce similar results) facilities listed below have been altered such that, when any data file integrity issue is detected, default behavior is now to stop further operations. In addition, no changes are made to existing original data records.

Note: This is a compatibility Change.

Full List of Functions Changed

The following utilities and function calls now default to stopping if an error is encountered:

ctcmpcif – Compact utility
ctrbldif – Rebuild utility
GUI tool - Dr. ctree has compact and rebuild functionality

Long function name	Short function name
CompactIFile()	CMPIFIL()
CompactIFileXtd()	CMPIFILX()
CompactIFileXtd8()	CMPIFILX8()
RebuildIFil()	RBLIFIL()
RebuildIFileXtd()	RBLIFILX()
RebuildIFileXtd8()	RBLIFILX8()

Default Behavior Changes the Original File

Prior to FairCom DB V11.5, the default behavior for compact and rebuild operations over variable-length record data files was to scan for invalid record headers before compacting a file. If the logic found invalid headers, it attempted to fix the headers in the original file. If the operation failed at this point, the original file could become corrupted. Two significant classes of situations are known to arise:

In the vast majority of cases, this can only occur if the original data file is already corrupted due to invalid and irrecoverable record headers. Because the original data file is corrupted, restoring a backup is recommended.
In some rare cases, it is possible the operation could exhibit vulnerability caused by mistaking old marks or binary record data as valid record header marks. In a very specific case, compact operations can treat space management nodes with inuse length 0, while logically a correct value, as invalid. This case could lead to data loss in a valid file, making it necessary to restore from a backup.

To ensure the second case does not cause data loss, observe our recommendations below and always back up all concerned data files before compacting or rebuilding.

Compact and Rebuild Best Practice

To avoid potential data loss, FairCom strongly recommends only using our latest FairCom DB V11.5 compact and rebuild logic that default to stopping if a corrupt data record header is encountered, and returns error DCPT_ERR (1107) rather than attempting to repair the data link in the record header. This allows an administrator to know with greater certainty that serious damage has occurred and they may prefer to recover the file from a backup.

If an administrator prefers allowing FairCom DB to attempt to recover a file, as done prior to V11.5, the following options are available to revert to prior behavior:

-repairdata command-line switch - Use this switch with the ctcmpcif and ctcmpcif compact utilities, e.g., ctcmpcif -rdata
-repairCorruptIFILoption option - The repairCorruptIFILoption option should be OR-ed into the tfilno of the rebuild (RebuildIFIL()) and compact (CompactIFile()) functions using the setIFILoptions() macro.

For example:
myIFIL.tfilno = setIFILoptions(repairCorruptIFILoption | updateIFILoption);

Best Practices:
1.) It is always important to back up concerned data files before you attempt to compact or rebuild them.
2.) Ensure sufficient disk space is available before starting a compact or rebuild. You will need up to at least 3X the size of the data file plus index files (original files + new files + copy) for a successful compact or rebuild.
3.) Consider keeping a copy of the data file for sufficient time and ensure all expected data is available before removing the backup.

ctcv43 - Convert V4.3 Data File to FairCom DB Format

ctcv43 <V4.3 data file name> <FairCom DB file name>

This utility converts a data file from c-tree V4.1F-V4.3C to FairCom DB format. It supports the conversion of files with up to 4 GB in size. Both fixed and variable-length files are supported.

If the new data file does not exist, it will be created. ctcv43 is designed to run in single user mode.

After converting the data files, you may use the function, RebuildIFile(), to construct the associated index files. If you have stand-alone indexes to convert, use ctin43 to first create a flat key file, and then use ctindx to build the new index file.

Note: ctin43, provided with FairCom DB, must be linked with your c-tree V4.3 application library.

ctcv67 - Extended File Conversion Utility

ctcv67 <old file name> <new path> [<option> <argument> ...]

The first two arguments are required. The new path is prepended to the old file name or the files in the parameter file. The options are listed in the table later in this topic.

ctcv67 performs various conversions, including:

Converting data and index files from Standard FairCom DB format to Extended c‑tree format (HUGE or non-HUGE) using Incremental ISAM (IFIL) or parameter files.
Adding huge file support to a non-huge file.
Adding, removing, or changing segment support.
Automatically compacting the output file.

Note: ctcv67 must be built by FairCom. An executable version is included in the FairCom Server installation for each platform, in the Utils folder.

If the original file has an extended header, the new file inherits the extended header attributes. This utility does not convert a huge file to a non-huge file. If a superfile host is specified, ctcv67 converts all member data files and their associated indexes. When a mirrored file is specified, the conversion does NOT produce the mirrors, but maintains the mirrored names in IFIL structures. Copy the converted master files to produce the mirror files.

Unless a parameter file is used, a data file must have an embedded IFIL definition for its associated indexes to be converted. A data file will have an embedded IFIL definition if it was created with one of the CreateIFile() routines or PutIFile() was called for the file.

If an index file is specified by the <old file name> parameter, then a new index file with an extended header will be created, all the resources will be transferred to the new index, its key length will be increased if HUGE is specified and the key supports duplicates, but no key values will be transferred to the new file. In essence, it will be an empty index file.

The following options are available:

Option	Arguments	Description
F	<yes \| no>	If yes, <old file name> is a parameter file with file modes set to ctREADFIL (8) and the second parameter (# of files) doubled. PP ignored. Default: no
T	<yes \| no>	Set to yes to output the names that will be assigned to the converted files along with pro forma segment specifications. This option is for information purposes only: No actual conversion takes place. Default: no
P	<page size>	Sets the page size in bytes used to create the new files. This setting must be compatible with the existing file. Default: 8192
PP	<yes \| no>	Set to yes to have the conversion run the Superfile prepass. Default: no
6	<yes \| no>	Set to yes to enable 6-byte transaction number support. This option is recommended for all files using FairCom Server Version 8 and later. Default: no
H	<yes \| no>	Set to yes to create a huge file. Default: yes
G	<new segment host file name> <host segment size in MB>	The new file name of a data or index file to be created as a segmented file. Derive this name from the new path and the file’s original name. (The T option can be used to check on the new name assigned to a file.) Use a G entry for each new file to be segmented. Note: Even if the old file is segmented, segments must be specified for the new file or it will no longer be segmented.
S	<segment name> <segment size in MB>	This option must follow immediately after the G option for the associated host file. Repeat this option as necessary to create all the additional segments of the file. The segment name will be used exactly as specified. The new path will NOT be prepended to the name.
A	<max # auto segments including host>	Use as many S <name> <size> entries as needed or OR one A <max> entry for each host. The S or A entries for each host must follow immediately after each host's G entry.
E	<encryption method>	Specify a valid ctSETENCRYPT() mod value (see SetEncryption). For example: ctAES32 for AES with a 32-byte key.
L	<File Extension size in Bytes>	Note: 2GB maximum.
TP	<Temp file directory>	Defaults to .\

Example:

ctcv67 vcusti new H no G new\vcusti.ndx 10 S e:\vc.s01 10 S f:\vc.s02 0

where:

vcusti is the existing file name.
new is the new path.
H no creates a normal file rather than a huge file.
G new\vcusti.ndx 10 sets the new segment host file name to new\vcusti.ndx and the host segment size to 10 MB.
S e:\vc.s01 10 S f:\vc.s02 0 creates the following segments:

e:\vc.s01 with a size of 10 MB.

f:\vc.s02 with a size of 0 MB.

Ordinarily, the new path name is simply prepended to the original file name. If both have absolute names, the new (absolute) path replaces the absolute portion of the original name. If only the original file is absolute, the new (relative) path is inserted immediately after the absolute portion of the original file name. Otherwise, the new path (relative or absolute) is prepended to the original (relative) file name.

For example, to convert the data file C:\DATA\SAMPLE.DAT and its associated index, C:\DATA\SAMPLE.IDX, into huge files with three segments of 2000 MB each, use the following command line:

ctcv67 C:\DATA\SAMPLE.DAT C:\HUGEDATA\ H yes

G C:\HUGEDATA\DATA\SAMPLE.DAT 2000 S D:\HUGEDATA\SAMPLE.SG1 2000 S E:\HUGEDATA\SAMPLE.SG2 2000 G C:\HUGEDATA\DATA\SAMPLE.IDX 2000 S D:\HUGEDATA\SAMPLE.IS1 2000 S E:\HUGEDATA\SAMPLE.IS2 2000

Even if the original file is segmented, the new file will not be segmented unless a new segment definition is specified using the ‘G’ and ‘S’ options.

The associated indexes are recreated based on the IFIL resource embedded in the data file or based on the parameter file information. If the associated data file becomes huge, index files allowing duplicate records will have their key lengths automatically increased by 4 bytes to accommodate the longer associated position information used to break the tie. If you recall, when allowing duplicate key values in an index, you must add 4 bytes to the key length. For huge files, you must add a total of 8 bytes. In addition, the index file names in the IFIL definition will reflect the new path name.

ctcv67 creates (or appends to) a text file named CV67.REP, which reports on the actions and progress of the utility program.

ctdbdistinct - Key Count Utility

Operational Model

Client

ctdbdistinct is a utility for managing distinct key counts in SQL tables and databases.

Command Syntax

ctdbdistinct.exe -s server -c command [-d database][-t table][-u user][-p passwd]

Options

-c command where command is one of:
- enable - enables distinct key counts
- check - validates existing distinct counts
- repair - updates existing distinct counts
-d database target database (all tables)
-s server source c-tree Server name (may be "local" for direct access)
-t table target table (one table)
-u uid user name on source server
-p upw user password on source server

The -c enable command assigns the CTDBINDEX_DUPCNTFLAG to all non-unique indexes in a database or a specific table. These indexes will start tracking distinct key counts. This command is intended to upgrade existing tables or databases created without distinct key counts.

The -c check command compares the distinct key counts stored in indexes with the actual key counts, and lists tables that fail validation. It operates while the table(s) may be in use by other users. Tables that fail the validation are logged to stdout.

The -c repair command updates distinct-key-count indexes with the current number of distinct key counts. It can run when tables are in use by other users. It briefly acquires a table read lock to update the distinct key count.

If the distinct count exceeds the desired limit of 2 per leaf node, it logs "Distinct key discrepancy: 20 != 32". If it exceeds 1/3 of the total keys in the index, the count will likely lead to bad SQL optimizer plans. It logs "Extreme distinct key discrepancy: 5 != 435123". When a table’s indexes should be repaired, it logs "Incorrect Distinct counts for table myTable index i_myTable_names".

Example 1:

Check distinct key counts on all files in the ctreeSQL database.

ctdbdistinct -s FAIRCOMS -c check -d ctreeSQL -u admin -p ADMIN

Example 2:

Repair distinct key counts on one table ./ctreeSQL.dbs/mytable.dat

ctdbdistinct -s FAIRCOMS -c repair -t ./ctreeSQL.dbs/mytable.dat -u admin -p ADMIN

ctdidx - Create flat key file

ctdidx <index file name> <index member #> <flat key file name>

This utility will run either in single user mode or with the FairCom Server. The FairCom Server or single-user transaction processing mode is necessary to dump the keys from an index which supports transaction processing. ctdidx creates a flat file containing each key value at full, uncompressed length followed by the associated record position in long integer, 4-byte, form. ctdidx reads each key in order from the index member designated, 0 for the host index, 1 for the first member, etc., and places the keys in the flat file it creates.

ctdmpidx - Index Dump Utility

ctdmpidx is available to dump an index. This utility is useful for viewing index header details, viewing nodes and key values. The syntax is shown below:

ctdmpidx [-<page size>] [-listkeys] [-full] <filename> <member #> [<rflg>]

The optional parameter -page size equals sector size * 128 (third parameter in InitCtree()). If page size is not entered, a default value of 16 will be used. filename specifies the index file targeted for analysis.
The member # refers to the index member number. A physical index file can contain one or more indexes. Each index has a member number (0, 1, 2, 3, etc.). For example, the sample index file custordr.idx provided with the FairCom ODBC Driver contains a total of two indexes. Depending on whether you specify 0 or 1 you will be looking at either the order number index or the customer number index.
rflg represents an optional recovery flag switch and is only applicable when compiled with TRANPROC. Any character will enable rflg, which will result in c-tree skipping automatic recovery.
If this utility detects the index file is using advanced encryption, it turns on c-tree's advanced encryption support and prompts for the master encryption password.
The -listkeys, option is available in V10.3 and later. It lists all the key values and their associated record offsets in an index.
The -full option causes ctdmpidx to display all keys when reading an index node. Otherwise, only the first few keys are displayed.
filename specifies the index file targeted for analysis.

As of V12, ctdmpidx also checks for a transaction mark whose position in the node precedes the first mark position recorded in the node header. It also checks if the number of marks in the node differs from the mark count in the node header. In these cases, it outputs these messages:

ERROR: Found mark at element <n> but expected first mark at element <m>

ERROR: Found <n> marks but expected <m> marks

Example

Below is an example of launching ctdmpidx along with output showing the header and prompt for an index node address.

# ctdmpidx -2048 custmast.idx 0

header info for file #0 [custmast.idx]:

configuration version: 81x index root ptr: 1000x

node size: 2048 superfile last member#: 0

data record length: 0 extend file size: 0

max bytes per leaf node: 1348 max bytes per non-leaf: 2030

file mode: 16392 key length: 4

file type: 1 # of additional members: 0

update flag: 0x index member number: 0

key type: 0 file id number: 0

duplicate flag: 0 server id number: 0

# of active entries: 4 current node ptr: 0x

delete stack ptr: 0x index high transaction #: 0

last byte used: 1800x serial number: 0x

last physical byte: 1800x delete type: 0

permanent file mode flags: 0 superfile type: 0

leaf anchor ptr: 1000x superfile member #: 0

file flavor: 2 file alignment: 8

key padding byte: 20x maximum name length: 255

header record ptr: 0x resource header ptr: 800x

Enter node address in hex: 1000

NODE: 1000x

trans active: no

predecessor node: 0x

successor node: 0x

# of entries: 4

# of bytes: 32

leaf flag: 1 (0 => non-leaf / 1 => leaf)

member #: 0

element offset key value

1 101fx 31303030

1 0 0 0

2 105ex 31303031

1 0 0 1

3 1099x 31303032

1 0 0 2

4 10d9x 31303033

1 0 0 3

ctexmc - File creation utility / example

Usage

ctexmc <ParameterFileName> [U<UserId> P<UserPassWord> F<FilePassword>]

Description

This utility can run with or without the FairCom Server. It creates the files specified in the named parameter file. If any of the files already exist, it terminates unsuccessfully. The UserId and password parameters are optional and only apply to FairCom Server environments.

ctfchk - File Checksum Utility

ctfchk is a standalone c-tree utility that calculates the checksum on all active records in a fixed-length c-tree data file. It reads active records from a fixed-length c-tree data file in physical order and outputs a checksum and active record count. The utility uses the CRC-32 algorithm to compute the checksum.

This utility can be used to compare the record contents of two fixed-length data files. Two c-tree data files that contain the same active record images in the same order will generate identical checksums. Two c-tree data files whose active record contents differ will generate different checksums (subject to the limitations of the CRC-32 algorithm).

Operational Model:

Standalone

Usage

ctfchk <filename> [-skpchk] [-logcrc] [-logbyt]

[-logrec <record number>] [-opencrpt]

Where:

-logcrc - Output the current CRC after each active record is read.
-logbyt - Output the offset of each active record.
-logrec <record_number> - Output the contents of the specified record. The first active record in the file in physical order is record 1.
-opencrpt - Permits the utility to open a file whose update flag is set.

When the file is already opened by another instance (Server, for instance), trying to open it to calculate the checksum will fail with error FCRP_ERR (14, File Corrupt at Open). If the -opencrpt option is added, the utility tries to open the file using the ctOPENCRPT filemode. If this second open attempt succeeds, the utility outputs the following message and proceeds to calculate the checksum of all active records in the file:

NOTE: This file's update flag is set. Opened file using ctOPENCRPT mode.

ctfilblkif - File Block Utility

ctfilblkif

ctfilblkif will block, or unblock a specified FairCom DB file. The default behavior is to block access to the specified file. Pass the -u option to unblock a file.

Operational Model:

Client

Usage

ctfilblkif [-s server][-f filename][-u][-m] {-p password|-a authfile}

-s: Server (default: FAIRCOMS@localhost)
-f: File name, including extension and relative or absolute path (a relative path of "./" can be omitted)
-u: Unblock file
-m: Block file with manual reopen option (otherwise automatic reopen)
-a: Authentication file name
-p: Admin Password
-t: Allow active transactions time (seconds) to complete before blocking
-n: Block new transactions

Example:

Block new transactions and allow active transactions 3 seconds to complete before blocking the mark.dat file:

ctfilblkif -f mark.dat -n -t 3

Manual Reopen

The ctfilblkif utility now supports an option (-m) to block a file with the reopen option. This means that when the file is unblocked, those connections that had the file open when it was blocked must "manually" reopen the file (e.g., the application must explicitly reopen the file).

Example 1:

Block the file with the reopen option:

ctfilblkif -f mydatafile.dat -m -p ADMIN -s FAIRCOMS

Example 2:

Unblock the file. The connections that originally had the file open must reopen the file:

ctfilblkif -f mydatafile.dat -u -p ADMIN -s FAIRCOMS

Note: FairCom does not guarantee preserving the state of a transaction, locks, ISAM context when a file is blocked and then unblocked, even if using the auto open option. Because ISAM context information is lost when the file is blocked, an error 590 may be returned after the file has been blocked and unblocked using ctFBautopen or when using ctfilblkif utility, which uses this mode.

Authentication File

This utility supports the use of an encoded password file. Encoded password files keep user IDs and passwords from plain view when the utility is used within a script file. They are created with the ctcmdset utility. The plain text form of the file should be:

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

See also

ctquiet - Quiesce FairCom DB Utility (ctquiet - Quiesce FairCom DB Utility, ctquiet Utility)

ctflush - File Flush Utility

ctflush is a c-tree utility that flushes updated buffers for c-tree files.

Operational Model:

Client

Usage

ctflush [-1 <passfile>] [-u <userid>] [-p <password>]

[-s <servername>] [-m 1|2|3 <filename>|4|5|6]

[-i <interval>] [-t <timeout>]

Where:

-1 <passfile> - encrypted password file (see below)
-u <userid> - user id for logon
-p <password> - user password for logon
-s <servername> - FairCom Server name including machine name or IP address
-i <interval> - for mode 4: interval in seconds between checkpoint calls for mode 5: interval in seconds between restore points
-t <timeout> - for modes 5 and 6: maximum time in seconds to wait for active transactions to complete
-m <mode> - 1, 2, 3 <filename>, 4, 5, or 6:

Mode 1 flushes updated buffers for all transaction-controlled files.

Mode 2 flushes updated buffers for all open files.

Mode 3 <filename> flushes updated buffers for the file <filename>.

Mode 4 performs a checkpoint.

Mode 5 creates a lightweight restore point.

Mode 6 creates a restore point with a checkpoint.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

ctfileid - Update File IDs

The Update File ID utility, ctfileid, provides a convenient and safe way to update the fileid parameter of the file header. See the section Copying Server Controlled Files in the FairCom DB Programmer's Reference Guide for details for when this may be necessary. The file is opened exclusively, ensuring that the server does not have it open.

Operational Model:

Client
Standalone

The syntax for this utility is shown below:

ctfileid file [-ioq] [-n <size>] [-t] [-s <svn>] [-u <uid>] [-p <upw>]

-i - Also update indexes related to data file.
-o - Force open of corrupted files (ctOPENCRPT).
-q - Quiet (do not output to stdout).
-t - Uses the PUTHDR() mode ctTIMEIDhdr to update only the time ID portion of the file ID.
-s <server name> - FairCom Server name.
-u <user ID> - User name.
-p <user password> - User password.

Note: ctfileid.c replaces the previous informal and undocumented utilities, updateid.c and newid.c.

Standalone Usage

An additional option is available to set the node size in standalone mode:

-n <size> - Set node size (standalone only).

ctflat - Create FairCom DB file from a flat file

Usage

ctflat <flat file> <new file> <record length> [<file mode> <ext size>]

Description

ctflat creates a FairCom DB fixed-length data file from an existing flat file with the specified record length. Optionally, you can specify the file mode to use when creating the file. For example, to enable transaction processing and the file size extension parameter.

ctflat assumes that the flat file begins at offset zero and has the record length indicated on the command line. You can modify the source to make it more appropriate for your specific circumstances.

ctflvrfy - Index Verify Utility

Usage

ctflvrfy <filename.idx>

Description

The Index Verify utility, ctflvrfy, takes the index name as the parameter and calls the ctVERIFY() and chkidx() functions to allow the user to verify the index, and optionally inspect it at a low-level.

>ctflvrfy custmaster.idx

Verifying file [custmaster.idx]

Retrying open with sect of [8]

Retrying open with sect of [12]

Retrying open with sect of [16]

Retrying open with sect of [20]

Retrying open with sect of [24]

Retrying open with sect of [28]

Retrying open with sect of [32]

Retrying open with sect of [36]

Retrying open with sect of [40]

Retrying open with sect of [44]

Retrying open with sect of [48]

Retrying open with sect of [52]

Retrying open with sect of [56]

Retrying open with sect of [60]

Retrying open with sect of [64]

Verifying the host index

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #1

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #2

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #3

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #4

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #5

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=38 header=38

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 38 KEYS FOUND

Verifying member index #6

Internal Index Verify: no tree structure

The return of ctVERIFY = 0, 0 KEYS FOUND

Verifying member index #7

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #8

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=16 header=16

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 16 KEYS FOUND

Verifying member index #9

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verifying member index #10

Verifying index delete stack...

Verifying index links...

Verifying index leaf nodes...

Index page scan finds entries=75 header=75

Index nodes per level of tree structure - [0: 1]

( 2 resource pages )

Internal Index Verify: SUCCESSFUL

The return of ctVERIFY = 0, 75 KEYS FOUND

Verify complete......

Do you wish to perform a low-level tree walk? (Y/N): n

cthghtrn - Displays the high-water mark for transactions

This utility displays the high-water transaction marks within an index file. This would be typically used when the FairCom Server transaction mark number gets too large.

Operational Model:

Standalone

Usage

cthghtrn [-<page size in bytes (e.g., -1024)>] <file name> [-nonzero]

Where:

-nonzero - outputs only the non-zero transaction high water mark values.

Examples

cthghtrn custmast.idx

cthghtrn -4096 custmast.idx

See also

ctclntrn Utility - Clean Transaction Mark in the FairCom DB Developer's Guide.

ctidmp - Examine Dump Files

Operational Model:

Standalone

Usage:

ctidmp [dump file name]

Description

The ctidmp utility lists the contents of a dynamic dump file or a specific extent of a dump broken into multiple files with the !EXT_SIZE keyword. Compile and link this utility with a FairCom DB Standalone Single-user TRANPROC library.

ctindx - Create index file from flat key file

Usage

ctindx <flat key file> <index file> <index member> [<transformation file>]

Description

This utility runs in single user mode. It supports indexes requiring transaction processing. The index file must already exist, and the index member (0 for the host index, 1 for the first additional index member, etc.) must be empty. The key length of the index member is used to deduce the key length of the flat file. You can optionally specify a transformation file specifying how to transform each key value before entry into the index file.

The optional transformation file is a text file similar to an ISAM parameter file formed as follows:

<# of segments>

... repeated for each segment ...

An offset of zero corresponds to the first byte of the key value stored in the flat key file. The length dictates how many bytes are affected by the transformation. The mode is interpreted as shown in Key Segment Modes in the FairCom DB Programmer's Reference Guide.

ctinfo - ISAM File Info

Operational Model:

Client
Standalone

Usage:

ctinfo FileName [-isam] [-rdkeys] [<UserId> [<UserPassword> [<ServerName>]]]

Description

ctinfo retrieves IFIL and DODA structures from a FairCom DB file as well as XCREblk extended header information if available. This utility is useful in both client-server and standalone mode.

-isam - Open the specified data file in ISAM mode (V10.3 and later). See below.
-rdkeys - If this option is enabled, ctinfo opens the data file and associated indexes at the ISAM level instead of performing a low-level open, which avoids FRSKEY error: 26. Without this option, ctinfo does not cause FairCom Server to log replication diagnostics; an ISAM level file open used by -rdkeys will cause the details to be logged.
ServerName defaults to FAIRCOMS if not provided.

Note: The ISAM open requires the index files to exist and to be accessible, unlike the low-level open which requires only the data file to exist.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

Index Compression Modes

The file information utility, ctinfo, displays index compression modes as included in the key type when displaying index information.

Example

IIDX #2 {

/* key length */ 4,

/* key type */ 2048, (0x0800 = KTYP_KEYCOMPSRLE)

/* duplicate flag */ 0,

/* null key flag */ 0,

/* empty character */ 0,

/* number of segments */ 1,

/* r-tree symbolic index */ cm_custnumb_idx,

/* alternate index name */ 0000000000000000,

/* alternate collating seq */ 0000000000000000,

/* alternate pad byte */ 0000000000000000,

};

Output

In V10.3 and later, the ctinfo -isam option causes the utility to open the specified data file in ISAM mode. This option is useful for checking if a REPLICATE keyword enables replication for the file.

In V11 and later, the ctinfo utility outputs the header record for fixed-length files or the header key count for each logical index. For example:

Header Count

4776 keys in index 0

4776 keys in index 1

4776 keys in index 2

ctinfo Sample Output

c-tree Plus(R) IFIL and DODA Extraction Utility.

This utility extracts file and field information from the file you

specify and displays that information. If the required IFIL and DODA

resources are present, the data file should work with the FairCom

Drivers. The ODBC interface requires detailed information about the

relationship between data and index files and the layout of records

within a data file. This information is stored in the IFIL and DODA.

..\admin_custmast.dat is a variable-length data file.

Data record length = 40

Logical file size = 19802

Physical file size = 32768

File mode = 0x34 = VLENGTH TRNLOG

Schema ID = 0

Conditional Index Resource Details:

File does not have a conditional index resource.

Extended File Mode Details:

ctFRC2HDRS : file has extended header

ctTRANDEP : file is transaction-dependent

ctFILEPOS8 : file uses 8-byte file offsets

ctFILCRECHKmd3 : file created and no subsequent checkpoint

Extended File Creation Block:

extended file mode : 0x40000052

segment file size : 0

max total file size : 0

1st file extent size : 0

file extent size : 0

max # of segments : 1

disk full threshold : 0

File Definition Mask Details:

DEF_MASK_IFIL2 : extended IFIL structures

DEF_MASK_SPCMGT : unambiguous superfile space management

DEF_MASK_PSCL24 : complete Pascal string def

DEF_MASK_XTDPRM : extended permission mask

DEF_MASK_UNIACS : UNIFRMAT alt col seq in IFIL

DEF_MASK_HUGEFL : ctHUGEFILE enabled

DEF_MASK_HDRSZ4 : V7 HDRSIZ: 4 * 128

DEF_MASK_PARTIN : partition files

IFIL {

/* data file name */ ..\admin_custmast,

/* data file number */ -1,

/* data record length */ 40,

/* data extension size */ 0,

/* data file mode */ 54 = VLENGTH TRNLOG EXCLUSIVE PERMANENT

/* number of indices */ 2,

/* index extension size */ 0,

/* index file mode */ 50 = ctFIXED TRNLOG EXCLUSIVE PERMANENT

/* first r-tree field */ 0000000000000000,

/* last r-tree field */ 0000000000000000,

/* temporary file number*/ -1

};

IIDX #0 {

/* key length */ 8,

/* key type */ 0,

/* duplicate flag */ 0,

/* null key flag */ 0,

/* empty character */ 0,

/* number of segments */ 1,

/* r-tree symbolic index */ $ROWID$,

/* alternate index name */ 0000000000000000,

/* alternate collating seq */ 0000000000000000,

/* alternate pad byte */ 0000000000000000,

};

ISEG [] {

{16, 8, 3},

};

IIDX #1 {

/* key length */ 4,

/* key type */ 0,

/* duplicate flag */ 4,

/* null key flag */ 0,

/* empty character */ 0,

/* number of segments */ 1,

/* r-tree symbolic index */ admin_custmast_sys_001,

/* alternate index name */ ..\admin_custmast_sys_001.idx,

/* alternate collating seq */ 0000000000000000,

/* alternate pad byte */ 0000000000000000,

};

ISEG [] {

{3, 4, 12},

};

SCHEMA MAP HEADER:

flavor: 2

align: 8

delimiter: '\0' (0 decimal)

padding: ' ' (32 decimal)

map length: 76

nbr fields: 10

nbr blocks: 10

DODA:

field offset field type length

----- ------ ---------- ------

$DELFLD$ 0 CT_ARRAY (128) 9

$NULFLD$ 9 CT_ARRAY (128) 1

$ROWID$ 16 CT_INT8 (231) 8

cm_custnumb 24 CT_FSTRING (144) 4

cm_custzipc 28 CT_FSTRING (144) 9

cm_custstat 37 CT_FSTRING (144) 2

cm_custrtng 39 CT_FSTRING (144) 1

cm_custname 40 CT_2STRING (162) 49

cm_custaddr 89 CT_2STRING (162) 49

cm_custcity 138 CT_2STRING (162) 49

Header Count

4776 keys in index 0

4776 keys in index 1

4776 keys in index 2

The required IFIL and DODA resources are present.

A special data types DLL may be required to access

this file with the FairCom(R) Drivers.

Contact your application vendor for details.

ctmtlk - Lock operation test utility

Usage

ctmtlk

Description

Utility to ensure proper locking operation with the multi-threaded stand-alone model.

ctmove - Moving Files without Stopping the FairCom Server

The ctmove utility can be used to move files while the FairCom Server is running. This is a useful tool to minimize downtime during a file recovery. The move operation is protected by opening the file. The syntax of this tool is shown below:

# ctmove [-o] file1 file2 [-n size] [-s svn] [-u uid] [-p upw]

-o: Force open of a corrupted file by using the ctOPENCRPT file mode.
-n: Defines the node size to be used (Standalone only).
-s: Specifies the server name for connecting to the server.
-u: Specifies the user id for connecting to the server.
-p: Specifies the user password for connecting to the server.

ctotoi - Add IFIL and DODA Resources

The ctotoi tool adds IFIL and DODA resources to a c-tree data file based on an OTP file.

Usage

ctotoi [<-sectors>] <otp_file_name>

ctpartadmin - Partitioned File Administration Utility

The generalized partitioned file administration utility is used to easily manage partitioned files.

ctpartadmin [-u user] [-p password] [-s server] [-f filename] [-w file password] [-c command] [-n partition number] [-d partition name] [-k partition key] {-a authfile}

Where:

-u: User ID
-p: User Password
-s: Server (default: FAIRCOMS@localhost)
-f: File name
-w: File password
-a: Authentication file name
-c: Partition Admin commands:

add - Add new partition

create - Create a partition index

purge - Delete existing partition

archive - Archive existing partition

activate - Activate archived partition

base - Set partition number low limit

status - Return partition status

rebuild - Rebuild partition

lowbound - Get active low partition number

highbound - Get active high partition number

The following options apply to the -c create command.

-r: Partition rule expression string. If the expression string contains space characters, the string should be specified within quotes.

ctpartadmin -c create -r "year(co_ordrdate) - 2000"

-i: The name used to reference the index. The index name is combined with the table name to make up the index file name. For example, the following creates an index file custordr_part.idx

ctpartadmin -c create -f custordr -i "part"

-I: (uppercase i) The same as -i but the partition index is created to allow only unique keys
-g: The name of the field used to make the partition key, sorted in ascending order
-G: The name of the field used to make the partition key, sorted in descending order

The -g and -G options can be combined and specified multiple times to create a multi-segment key

ctpartadmin -c create -f customer-orders -G customer_category -g order_date

The following options are mutually exclusive, and only one is required (not required for lowbound or highbound options). If more than one is entered, the last option entered prevails.

-n: Partition number (rawno)
-d: Partition file name
-k: Partition file key

ctpartadmin return code is used for the status command.

lowbound and highbound output a single value for capture. This value is a LONG.

When the -c create command option is successful the 'file partitioned' message is displayed, and the -c status command option displays 'partition active'

>ctpartadmin -c create -f custordr.dat -u admin -p ADMIN -r year(co_ordrdate) -i part -g co_ordrdate

file partitioned

>ctpartadmin -f custordr.dat -u admin -p ADMIN -c status -n 2002

partition active

ctpathmigr - Change Internal Path Separators

The ctpathmigr utility is used to change embedded path separators for FairCom DB SQL databases when migrating between Unix and Windows platforms. This functionality was previously available in the ctpath utility. This utility changes database and IFIL resource path separators from the native format of the system where the utility is run (i.e., \ for Windows; / for Unix) to the non-native format. It should be run before moving the files.

Operational Model:

Client

Syntax

ctpathmigr [-s <server>] [-u <user>] [-p <password>] [-d <database>]

Description

ctpathmigr with command line arguments returns 0 when the execution detected no errors, or a value different than 0 when errors were detected and the error messages are written to the screen.
If you omit the -d database switch, all databases in the session will be scanned.
Command line options should start with a '-' or '/' character. Use a lowercase character for the option (e.g., -s not -S).
Command line switches may have optional spaces between the switch and the argument. Example: -s FAIRCOMS or -sFAIRCOMS are the same.
Command line options may be entered in any order.

ctquiet - Quiesce FairCom DB Utility

The ctquiet utility allows an administrator to quiet the server from a script. An interactive option is available in the ctadmn utility.

Operational Model:

Client

Usage:

ctquiet [-c] [-f] [-i] [-m] {-p password|-a authfile} [-r] [-s server] [-t timeout] [-u] [-x] [-w]

Parameters:

-a: Authentication file name
-c: Wait for replication readers to finish processing
-f: Full consistency (default: crash consistency)
-i: Ignore inactive replication readers (used with -c option)
-m: Quiesces the server, changes the specified configuration options, and resumes normal server activity. Allows changing server configuration options that require the server to be quiesced, avoiding additional calls to the Server. Can be specified one or more times in a call to ctquiet.
-p: Admin Password
-r: Create an incremental backup restore point. Automatically unquiets server
-s: Server (default: FAIRCOMS@localhost)
-t: Set transaction timeout in seconds. (maximum/default: 60 seconds).
-u: Unquiet server
-x: Abandon Quiet request if timeout exceeded. (Default: Abort long transactions.) Applies ctQTfailAfterTimeout mode. This mode has been extended to apply to restore point checkpoint creation for incremental forward rolls.
-w - (Linux only) Execute command on successful quiet. Waits for SIGINT to unquiet server.

Note: The -c option requires ALL client utilities, replication readers, and servers to be installed from FairCom DB V11.3 or later. It is not compatible as a drop-in to current running environments.

Support for Incremental Backup Restore Points

Several changes have been made to ctquiet to support restore points necessary to carry out an incremental backup strategy:

The -r feature can be used to generate the restore point files necessary to carry out an incremental backup strategy.
The ctQTfailAfterTimeout mode has been extended to apply to restore point checkpoint creation for incremental forward rolls.
The -x option applies the ctQTfailAfterTimeout mode to the already existing options.
The -t timeout output has an effect only if -f or -r is specified.
The default timeout has been changed to match the server's maximum.

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

Full Consistency

Note: This utility provides a -f option, which enables full consistency (also known as a "clean quiesce") in which files are flushed to disk from cache. When this option is used, the system does not require any files to be rebuilt or recovery from transaction logs to occur.

The default is the -f option is off, which results in "crash consistency" (also known as a "dirty quiesce"). The default means that transaction logs are required and, if present, the transaction logs are used to get back into a clean state once recovery completes.

If files are NOT under transaction control, the -f option is strongly recommended, otherwise you will have to do a rebuild to get the files back to a clean state.

Physical Copy of Files

A quiesced state allows a physical copy of files to be taken that can then be restored at a later time. For systems that provide hardware-based snapshot features, this allows extremely fast system backups to take place while maintaining full data integrity.

The -r option creates an incremental backup restore point. After the ctquiet call succeeds, this new restore point and all server transaction logs can be copied to a backup area and used for an incremental recovery.

Timeout

The -f and -r options require that no transactions are active for the quiet to complete. The quiet operation defaults to waiting 60 seconds for the outstanding transaction to complete before aborting these transactions. Operations by other users and connections wait during this time.

The -t and -x options modify the timeout behavior:

-t - Allows setting a shorter timeout. Times longer than 60 seconds are reduced to 60 seconds.
-x - Instead of aborting the transactions, the quiet operation will fail if the timeout expires without all transactions completing.

Example 1:

You want to create a server backup for your application that is not using transaction logging on all the files you want to backup. Only allow 10 seconds before aborting outstanding transactions.

ctquiet -f -t 10 -s FAIRCOMS -p <admin password>

Make your backup and then end the quiet:

ctquiet -u -s FAIRCOMS -p <admin password>

Example 2:

You want to create a fast server backup, and all your files are using transaction logging. Your backup will need to go through recovery before being used.

ctquiet -s FAIRCOMS -p <admin password>

Make your backup and then end the quiet:

ctquiet -u -s FAIRCOMS -p <admin password>

Example 3:

You want to create a restore point for an incremental backup and only allow a brief (1 second) delay. You don't want ctquiet to abort any long running transactions.

ctquiet -r -t 1 -x -s FAIRCOMS -p <admin password>

Notes

When you quiesce the server, as long as the connection that quiesced the server remains connected, all other connections are blocked. Only if that connection goes away do we allow the ADMIN user to logon again and undo the quiesce.
After the server is quiet and the ctquiet utility disconnects, one ADMIN connection is allowed to reconnect. There is no prevention of a separate process connecting as ADMIN while the server is in a quiet state and precluding the unquiet call.
There is a subtle distinction between a "quiet" state and a file blocked with the ctFILBLK() call. While in the quiet state, files are not physically closed and cannot be moved or replaced while in this mode. Compare this to a "blocked" state, where the file can be replaced, as the OS file handle has been released.
When the optional -w COMMAND switch is used (unavailable on Windows), the behavior of ctquiet is modified to perform as follows:
1. After successfully quieting the server, it makes a system call to execute COMMAND.
2. It remains connected and waits for SIGINT to unquiet the server. If ctquiet is killed before receiving SIGINT, the server will remain in a quiet state until a new connection unquiets the server.

ctrbldif - IFIL-based Rebuild Utility

A rebuild utility using the IFIL definitions stored in the header of a file:

Operational Model:

Client
Standalone

Usage:

ctrbldif DataFileName [<UserId> [<Password> [<ServerName>]]] <options>

Description

ctrbldif reads the IFIL structure from DataFileName and calls RebuildIFileXtd() to rebuild DataFileName and its associated indexes.

The following switches can be added in the location specified by <options> above:

-callback - In V11.5 and later: When used with a FairCom DB Server this will provide additional feedback on the rebuild progress.
-idxseg=<M>@<S> - If creating index, use M automatic segments of size S. Size is specified in megabytes, or you can specify MB or GB as a suffix. For example: -idxseg=10@1GB
-local - causes the utility to use a standalone (local side) connection instead of a client connection when linked with a LOCLIB library.
-oldsec - Use prev10.3 security attribute behavior.
-online - In V13 and later: Rebuild the index while it remains available to other users.
-purge - Purge duplicate records.
-repairdata - Attempt repair if damaged data records are detected.
-skipdatascan - In V11.5 and later, if you know your datafile is in a good state, you can use this option to skip the initial datafile validity scan for faster rebuilds.
-sortmem=<N> - Use N KB of memory for sorting. This option can be used to increase the rebuild speed for large files.
-updifil - Update the IFIL resource within the data file.
-x8 - Use extended create blocks read from data and index files.

FairCom DB Standalone Options

-temppath= - directory path, <temporary_path>, to create temporary sort and purged record files. (This standalone option is the equivalent of the TMP_PATH server configuration option.) The default location is the current directory.
-<sectors> - sector size, <sectors>. (in multiples of 128). The sector parameter is especially useful if you need to adjust a file’s PAGE_SIZE (index node size) to match the FairCom Server. FairCom DB standalone models default to 16 sectors (2048 byte node size) while the FairCom Server defaults to 64 sectors (8192 bytes).

FairCom Server Options (Must be passed before any optional switches)

<UserID> : client user name to logon to a FairCom Server.
<UserPassword> : client user password to authenticate to a FairCom Server.
<ServerName> : FairCom Server name for a client to connect.

Environment Variable to Enable Advanced Encryption

Note: If c-tree does not support advanced encryption and this environment variable is set, the c-tree initialization will fail.

Considerations

Even without the -x8 option, some extended create block settings such as the huge file, extended header, and 6-byte transaction number options, are always applied to new files.
This option requires all associated index files that are referenced in the data file's IFIL structure to exist, as the utilities use OpenIFile() to open the data file and all associated index files to read the extended create block values.
When an application calls the standalone version of the Xtd8 file compact or rebuild functions (for example, CMPIFILX8() or RBLIFILX8()), index files created by the compact or rebuild no longer have the 6-byte transaction number attribute enabled if the specified extended create block's x8mode field has the ctNO6BTRAN bit set.
A rebuild or compact will fail with the new error code TFLN_ERR (943) when a client library that supports the new rebuild or compact option format attempts to use this feature with a FairCom Server that does not support this new format.

Option to set index's automatic segment attributes

If a segmented index file was deleted and rebuilt using the ctrbldif utility or the rebuild API function, the new index file was not segmented. The index file's segment attributes, which are stored in the index file, were lost when the index file was deleted.

Beginning with V10.3, an option is provided with the rebuild and compact utilities that causes index files that are created by the utilities to be created using automatic segments of the specified size and maximum. The option is:

-idxseg=M@S

where M is the maximum number of segments and S is the segment size. S is interpreted as megabytes by default. Two suffixes, MB and GB, are also supported. MB indicates that the value is in megabytes and GB indicates that the value is in gigabytes.

Examples:

50 segments of 10 MB each:

ctrbldif test.dat -idxseg=50@10

100 segments of 300 MB each:

ctrbldif test.dat -idxseg=100@300MB

20 segments of 3 GB each:

ctrbldif test.dat -idxseg=20@3GB

Note: If the index files exist, the -idxseg option has no effect. It is only when the rebuild or compact utility creates new index files because the old index files do not exist that the -idxseg option affects the automatic segment properties of the index files.

-oldsec - Updates in handling of security attributes

In V10.3 and later, the ctrbldif utility assigns data file owner/group/permissions to index files and ctcmpcif assigns original data file owner/group/permissions to compacted data/index files.

ctrbldif mark.dat -oldsec ADMIN ADMIN FAIRCOMS

ctredirect - IFIL Update Utility for Redirected Filenames

Updates IFIL Filenames for redirection to alternate locations.

The Redirect feature is used to allow a file originating in one directory structure to be repositioned into another directory location following autorecovery, dynamic dump restore, or replication. As a result, if the IFIL resource of the file contained a path, this path would be incorrect after the file was redirected to the new location. To support copying c-tree files from one directory location to another (on the same system or on a different system) and accessing them in their new location, it is necessary to update any filename paths in a c-tree data file's IFIL resource.

ctredirect is used to update the file names contained in the IFIL resource of the specified files. ctredirect accepts as command-line options the name of a text file containing the list of files whose IFIL resources are to be updated and the name of a text file containing the filename redirection rules.

A comment may be placed in the redirection rules file by starting the line with a semi-colon (;).

Note: The rules must match the path in the IFIL resource, which may differ from the path used to open the file. The rules must precisely match the path in the IFIL resource, which may differ from the path used to open the file, therefore you may not use wild cards.

Operational Model:

Standalone

Usage

ctredirect -f <filenamelist> -r <redirectlist> -n <sect>

Where:

-f <filenamelist> - Name of a text file containing the names of c-tree data files whose IFIL resources are to be updated.
-r <redirectlist> - Name of a text file containing the redirection rules. The rules are in the format: <current path> <new path> (see the example below).
-n <sect> - Node sector size. The default is 64, corresponding to the server default PAGE_SIZE of 8192.

Example

Consider the current directory contains the data files customer.dat and inventory.dat, and their IFIL resources specify the path /export/home/users/marketing/data for the data and index files.

The current directory contains the file myfiles.txt listing the following file names:

customer.dat

inventory.dat

The current directory also contains the file myrules.txt containing the following redirection rules:

/export/home/users/marketing/data /export/home2/users/support/data

ctredirect is then called to change the paths in the IFIL resource of these two data files:

./ctredirect -f myfiles.txt -r myrules.txt

Updating the file paths in the IFIL resources of the specified files...

[ OK ] customer.dat

[ OK ] inventory.dat

Successfully changed IFIL resource for all specified files

ctsbld - Superfile rebuild pre-pass

The FairCom DB Superfile Recover Utility reconstructs data file members in place and empties index file members. Index file members must be rebuilt via RBLIFIL() or ctindx.

Operational Model:

Standalone

Usage:

ctsbld <superfile name>

Description

ctsbld processes the superfile in place. This utility “cleans” the data files and empties the index files. This utility may also be incorporated into your application using the function SuperfilePrepassXtd(). Afterward, the superfile must be processed by the RebuildIFile() function before it is usable. This utility is only operational in single-user mode. The SuperfilePrepassXtd() function is available in single-user and client/server models.

ctsbldm - Rebuild Superfile Index Members

Operational Model:

Standalone

Usage:

ctsbldm <DataFileName> [<Optional PageSize>]

For example:

ctsbldm suphost!myfile.dat 16384

Description

Ordinarily, the procedure to rebuild a superfile is to use ctsbld to prepare the entire superfile, making the data good and returning space used by the indexes. Then each set of ISAM files is rebuilt. This approach is used since a superfile entwines all of the member files and no assumptions are made about the integrity of the members.

ctsbldm permits the indexes associated with a non-corrupt superfile member data file to be rebuilt without having to process the entire superfile. It assumes that the data file contains an IFIL describing all the indexes, that the data file is not corrupt, and that the superfile is in good shape except for the indexes in question.

One possible use for this utility is when recovery cannot repair an index. This rarely occurs, but may be caused by a loop in the index leaf nodes. If this index is a member of a superfile, we can now attempt to rebuild just the indexes, and not process the entire superfile. The utility forces all indexes associated with the data file to be rebuilt.

The utility operates by opening the data and index files, deleting the index files, and then rebuilding the index files.

ctscmp - Superfile Compact Utility

Operational Model:

Standalone

Usage:

ctscmp <filename> [Options] [Y] [new_sect_size]

Arguments

<filename>: The name of the superfile to compact or scan.
Options:
- -resetdistinct: Reset distinct key count attribute in IIDX ikeydup fields.
- -scanonly: Scan the superfile only; do not compact it. No new superfile will be created. The -scanonly option generates details from the header, extended header, IFIL, and DODA. It outputs the number of key values, active records, deleted records and resource records. Using this utility with the -scanonly option is useful for comparing superfiles before and after compacting them.
- -aes32: Enable AES encryption with 32 byte key.
- -aes24: Enable AES encryption with 24 byte key.
- -aes16: Enable AES encryption with 16 byte key.
- -cleartext: Disable file encryption.
- -v11: Allows compatability with V11.
Y: An optional parameter; if specified, the utility compacts the file without prompting to confirm (otherwise you will be prompted).
<new_sect_size>: An optional sector size of the resulting file (sect*128 = page size) to use for the new superfile. If omitted, the new superfile is created using the same page size as the original superfile.

Description

The superfile compact utility, ctscmp, compacts and rebuilds a superfile and all its members provided the required IFIL resources are present in the superfile data members. It creates a compacted version of the file in place, using a temporary file named CTREE.TMP as a destination and then renaming it.

Use cases for this functionality include eliminating wasted disk space, changing page size or encryption attributes, or repairing problems with an index.

The utility is designed for the Standalone single-user model. It supports processing data and index member names with extensions other than .dat and .idx.

A “% complete” progress indicator has been added to the ctscmp utility.

The ctscmp utility uses a standard error and warning message format. Error messages begin with the text ‘Error:’ and warning messages with the text ‘Warning:’, so errors and warnings can easily be located in the ctscmp output.

The default ctscmp behavior is to preserve existing encryption settings. Options to enable or disable encryption have been added. Working with AES encrypted files requires the correct master password and a master password verification file "ctsrvr.pvf". See Master Password Verification Options.

There is an ISAM-based version of the superfile compact utility named ctscmpi. This version uses ISAM c-tree functions rather than low-level c-tree functions to add records to the new compacted superfile. The advantage of the ISAM approach is that records are indexed as they are added, so the utility does not require a separate rebuild phase to repopulate the indexes as the ctscmp utility does. Using this technique can significantly reduce the time required to compact a superfile.

The ctscmp utility returns a non-zero value when it fails. This can be useful for a script that calls the utility to be able to detect when it fails.

ctsfex - Superfile Data Export Utility

The ctsfex utility exports data from a superfile to another superfile or to individual files identical to the superfile members. This high-performance utility exports the data without index updates, transaction control, or file extension. After the data export is completed, the indexes are rebuilt and transaction control and file extension properties are restored on the newly-created files. This utility also supports a multi-threaded version allowing each thread to process a superfile member.

Operational Model:

Client
Standalone

Usage:

ctsfex sf1 [-f sf2 | -d dir] [-cCrovV] [-t thds] [-e key]
[-m recs] [-n size] [-T <path>] [-ctscmp <args>]
[-s svn] [-u uid] [-p upw] [-i]

Where:

-f sf2 - Export to superfile sf2
-d dir - Export to standard files in directory dir
-c - Create the target files without exporting data.
-C - Create AND export data for _isp@ and .group tables only.
-r - Force replacement (overwrite) of existing files.
-o - Force open of a corrupted file using the ctOPENCRPT file mode.
-v - Open the superfile and its members to validate a superfile.
-V - Used in conjunction with -v, this option validates records with a read.
-m recs - Send record counter to stdout every recs records.
-e key - Encrypt the target files with key
-t threads - Maximum number of concurrent threads to use.
-T <path> - Specify the location of temporary files created by the rebuild operation.
-s server_name - Server name for connecting to the server.
-u userID - User ID for connecting to the server.
-p userpassword - User password for connecting to the server.
-ctscmp <args> - Invoke the superfile compact utility, ctscmp with <args>.
-i - Read records from superfile members using the first index instead of physical order (if superfile member has no indexes, physical order is used for that member).

Standalone Usage

In standalone mode an additional option is available to specify the index node size:

-n size - Index node size to use (standalone mode only).

Page Size

Default PAGE_SIZE settings are now considered in superfile compact and export utilities, ctsfil and ctsfex. In V11.9.5 and later, this size is 32768 (sect = 256). Prior to V11.9.5, it was 8192 (sect = 64). Server and standalone of these utilities have been updated to use the new default page size. ctsfex supports a parameter for use with superfiles created with different page sizes.

ctsfil - Superfile contents

Usage

ctsfil <superfile name>

Description

ctsfil lists the members of the superfile, indicating whether each member is an index file, fixed-length data file, or variable-length data file.

Page Size

ctsmon - Server Heartbeat Monitor

ctsmon is used to check the current running status of a FairCom DB Server. Frequently, this is the "heartbeat" monitoring component in a clustering failover solution.

Usage

ctsmon [-u name] [-p pw] [-s s1,s2,...] [-t nn] [-c nn] [-d nn] [-o api_option]

where

-u name - User name to connect to the server with. Default is none.
-p pw - User password. Default is none.

It is assumed that all servers in the server list (-s) have the same user ID and password.

-s Server_name - Name of the Faircom server to log into.

The server name has the form of name[@host]. If no @host is given the local host is assumed. @host can also be given as using an I/P address Example: FAIRCOMS@192.168.1.1 Default is FAIRCOMS

-t nn - Time out value in seconds. Default is 30 seconds.
-c nn - Loop count for checking the server. This parameter has three possible values:
- +nn - Exit after nn times regardless if the server is up or down.
- 00 - Loop forever regardless if the server is up or down.
- -nn - Loop forever while the server is up, if unable to log into the server, exit after nn loops.
-d nn - Set debugging trace level to n (0 to 5).
-o api_option - An optional API call to retrieve additional information from server demonstrating the server is functional.

api_option is one of the following:

appname - First APP_NAME_LIST entry
curtime - Current server time in seconds since 1970
commits - Current number of database commits (ctGSMS.sct_trend)

Return status

-1 - Usage error, or internal error.

0 - Server responded.

+n - Error returned from InitISAMXtd call.

Description

This utility is intended for use with cluster management systems such as the Open Cluster Framework (OCF) and Pacemaker on Linux, and Solaris and Microsoft cluster managers. This is usually run in the background as a daemon process. It is a useful utility to use with any monitoring the current availability status of a FairCom DB server process.

The availability check is a basic connection/disconnection attempt. If the connection succeeds, it is assumed the server is active and available. If a further check such as the ability to retrieve data is required, the api_option can be used to trigger that request.

The source code of this utility is provided in the FairCom DB SDK and can be further customized for specific usages.

Example

ctsmon -u ADMIN -p ADMIN -s 10.6.1.1 -t 20 -c 3

ctstat - Statistics Utility

The FairCom DB Statistics Utility, ctstat, is a client utility used to display statistics collected by FairCom DB. This utility provides valuable real-time monitoring of critical FairCom DB operations.

Operational Model:

Client

Usage

# ctstat report_type [-s svn] [-u uid] [-p upw]
[-i int [cnt]] [-h frq] [-d] [-m] [-t]

Reports:

-clf	Get the CHECKLOCK_FILE list entries
-deleted_files	Deleted Files Report
-file [csv]	File Activity Report
-filelist listname	List the entries on the specified file list
-filelocks file [N]	List all locks on a data file. Display Nth key
-fileops	File Operation Activity Report
-func	Function Timing Report
-funcfile	Function timing by File Report
-iotime on\|off	Turn disk I/O call timing on or off
-isam	ISAM Activity Report
-ma logfile	Log aggregate memory allocations to the specified file
-meminfo	Server Memory Info Report
-mf logfile	Log all memory allocations to the specified file
-ml	Get current memory allocation statistics
-mr min,max	Log only memory allocation statistics
-ms	Output memory allocation statistics
-mt options	Set memory allocation tracking options. Examples: -mt +ALL -mt +PI8TYP, -MBATYP
-mu	Unload module debug symbols
-sql	SQL Activity Report
-text	System Activity Report
-tranlog_flush	Transaction Log Flush Fixed Histogram Report
-userinfo	User Report with stats from USERINFO() function
-userinfox	User Report with stats from USERINFOX() function
-userlocks user	List all locks held by a user
-vaf file...	Admin-File Report
-vah handle...	Admin-User Report by User Handle
-var	Admin-Replication Reader Report
-vas	Admin-System Report
-vat	Admin-Transaction Report
-vau user...	Admin-User Report by User Name
-vtf file...	Tivoli-File Report
-vts	Tivoli-System Report
-wrktime on\|off\|reset	Turn function call timing on or off, or reset statistics

Options:

-d	Show cache stats as delta
-h frq	Print a description header every frq outputs
-i int [cnt]	Pause int seconds for optional cnt times
-I int [cnt]	Pause int seconds also before first row for optional cnt times
-m	Show memory file stats when using -vaf report
-p upw	User password
-s svn	c-tree Server name
-t	Output timestamp with header
-u uid	User name

Authentication File

; User Id
USERID ADMIN
; User Password
PASSWD <pass>

Use the -1 option to specify the name of the encoded file.

-filelocks Notes

The -filelocks option lists all locks on a data file and, optionally, displays the Nth key. The lock offset and the associated keys are not read at the same time. Since we are reading records locked by other users to generate the key, there is no guaranteed relationship between the lock and the displayed key. The following are possible scenarios:

The displayed key is from before or after any changes made by the lock holder.
The locked offset no longer holds a valid record (it has been deleted, or updated and moved).
The locked offset could have been locked/modified/unlocked more than once between the time the lock offset was acquired and the time the record is read, so the offset could hold an entirely different record than what was originally locked.

The -filelocks file [key] command supports c-tree's standard wildcard filename matching for the specified file, allowing locks from multiple files to be displayed. The standard wildcards (used by ctsrvr.cfg keywords such as MEMORY_FILE and REPLICATE, etc) are:

* - Multi-character match

? - Single-character match

^ - Negation (must be first character)

-fileops Notes

Database resource consumption requires close monitoring of all file I/O activities. FairCom DB tracks values for additional physical file operations. The following values are tracked:

logical file opens
logical file closes
physical file opens
physical file closes
file creates
file renames
file deletes

Each value is a cumulative value since server startup, stored as an 8-byte integer field in the system snapshot structure.

The counters include every call, not just successful calls. Note that the physical file open count can be smaller than the physical file close count, because when a file is created it does not increment the physical file open count.

The ctstat utility supports a new option, -fileops, that displays these counters. Example:

ctstat -fileops -t -i 1 -h 10 -u ADMIN -p ADMIN -s FAIRCOMS

logopn/s logcls/s phyopn/s phycls/s filcre/s filren/s fildel/s total/s

0 0 0 0 0 0 0 0

154 158 23 26 13 0 3 377

959 926 70 63 25 0 4 2047

1016 989 4 0 0 1 1 2011

574 562 0 0 0 0 0 1136

481 486 2 0 0 0 0 969

425 417 0 0 0 0 0 842

376 380 0 0 0 0 0 756

The ctstat -fileops option requires a server that uses snapshot version 21 or later. If this option is used with a server that uses an earlier version of the utility, the utility fails with the error message:

Error: The -fileops option requires snapshot version 21, but this server is using snapshot version <version>.

where <version> is the older snapshot version that FairCom DB is using.

The snapshot file parsing utility, ctsnpr, has been updated to support the new SNAPSHOT.FCS file format.

-tranlog_flush notes:

The -tranlog_flush displays the transaction log flush histogram values. Use the -d option to display delta values. Otherwise, ctstat displays the cumulative histogram values. For example:

C:\> ctstat -tranlog_flush -h 1 -t -d -u ADMIN -p ADMIN -s FAIRCOMS -d

Fri Oct 4 16:24:30 2024

Since server startup:

tranlog flush count: 21761

tranlog flush time: 25.737986

tranlog flush average: 0.001183

Since last checkpoint:

tranlog flush count: 2

tranlog flush time: 0.007201

tranlog flush average: 0.003600

Since last sample:

tranlog flush count: 324

tranlog flush time: 0.495657

tranlog flush average: 0.001530

adaptive histogram adjustments: 3

time since most recent adjustment: 0

time since fixed histogram last cleared: 18

fixed histogram adaptive histogram

count time count time

132 < 0.001000 0 < 0.000031

0 < 0.002000 0 < 0.000062

192 < 0.003000 0 < 0.000093

0 < 0.004000 0 < 0.000124

1 < 0.005000 0 < 0.000155

0 < 0.006000 0 < 0.000186

0 < 0.007000 0 < 0.000217

0 < 0.008000 0 < 0.000248

0 < 0.009000 0 < 0.000279

0 < 0.010000 0 < 0.000310

0 < 0.011000 0 < 0.000341

0 < 0.012000 0 < 0.000372

0 < 0.013000 0 < 0.000403

0 < 0.014000 0 < 0.000434

0 < 0.015000 0 < 0.000465

0 < 0.016000 0 < 0.000496

0 < 0.017000 0 < 0.000527

0 < 0.018000 0 < 0.000558

0 < 0.019000 0 < 0.000589

0 < 0.020000 0 < 0.000620

0 < 0.021000 1 < 0.000651

0 < 0.022000 0 < 0.000682

0 < 0.040000 [0 avg] 1 < 0.001240 [0 avg]

0 >= 0.040000 0 >= 0.001240

-userinfox Notes

In V11.6 and later, the -userinfox option displays extended connection information. For example:

ctstat -userinfox -u ADMIN -p ADMIN -s FAIRCOMS -h 1 -t -i 1

status lastrequest trntime mem fils time uid/tid/nodename commprotocol readops readbytes writeops writebytes datarqsts datahits indexrqsts indexhits

0s busy TRANEND 0s 96K 1 3s CTQA/21/(2) ctqa FSHAREMM 643 35831372 706 16111449 18569 17930 2535 2535

0s busy USERINFOX -- 76K 2 11s ADMIN/22/ctstat FSHAREMM 4 16544 0 0 12 11 0 0

0s busy TRANEND 0s 70K 1 3s CTQA/23/(3) ctqa FSHAREMM 566 31750812 507 14227936 23180 22614 1467 1467

0s busy FRSVREC 0s 75K 1 3s CTQA/24/(4) ctqa FSHAREMM 754 41842916 520 16605870 32476 31722 2484 2484

0s busy FRSVREC 0s 90K 1 3s CTQA/25/(5) ctqa FSHAREMM 833 46135384 649 19387611 25274 24441 3602 3602

0s busy TRANEND 0s 94K 1 3s CTQA/26/(6) ctqa FSHAREMM 760 43814980 548 16663966 28098 27339 2700 2700

2s busy RRDREC 2s 236K 5 3s CTQA/27/(13) ctqa FSHAREMM 560 11343722 644 3677007 853 793 2033 1898

0s busy FRSREC 0s 253K 2 3s CTQA/28/(8) ctqa FSHAREMM 819 52887780 1690 53585664 40324 39518 6342 6341

0s busy TRANABT 0s 260K 2 3s CTQA/29/(9) ctqa FSHAREMM 776 50069732 1456 45076587 38689 37925 5968 5968

0s busy TRANEND 0s 183K 2 3s CTQA/30/(10) ctqa FSHAREMM 0 0 2198 44113920 0 0 776 776

0s busy NXTREC 0s 2956K 1 2s CTQA/31/(18) ctqa FSHAREMM 455 29557760 361 15667386 10848 10397 2 2

2s busy BATSETX 3s 16950K 6 3s CTQA/32/(12) ctqa FSHAREMM 17139 142302504 45462 42391289 10232 8130 80823 80756

-userlocks Notes

For the -userlocks report:

If UserID is a number, it is interpreted as a task ID.
If UserID is a string, it is interpreted as a name, and information on locks held by each task ID with a matching name is returned.

Because the -userlocks report may generate a large number of server calls (for each task ID and file), the -userlocks report interval may be increased up to 60 seconds, depending on the number of matching users and files involved.

Filelocks and Userlocks Note: Dumping large quantities of locks in a very active system could affect performance.

List files on internal server lists

In V12 and later, ctstat provides a list of files on specific internal server lists, such as ctKEEPOPEN.

Example

C:\> ctstat ‑filelist keepopen_list ‑u ADMIN ‑p ADMIN ‑d ‑h 1 ‑t ‑s FAIRCOMS

Fri Dec 20 09:29:39 2019

value member scale filename

1 0 0 mark.dat

-isam header changed to avoid misunderstanding

The header of the ctstat -isam command displays a /s suffix to indicate that the statistics are per second. For example, the following syntax will produce the output below:

ctstat.exe -s FAIRCOMS -u ADMIN -p ADMIN -h 2 -isam

add/s del/s upd/s read/s total/s

0 0 0 0 0

add/s del/s upd/s read/s total/s

0 0 0 0 0

Note: This constitutes a Compatibility Change.

Admin-System Report -vas

The admin-system (-vas) report displays FairCom DB Server system-wide statistics in the areas of cache usage, disk I/O, open files, established client connections, file locks, and transactions.

Example

Below is a sample admin-system report produced by executing the command:

ctstat -vas -u ADMIN -p ADMIN -s FAIRCOMS -h 10 -i 2


	cache			disk i/o		files	connect		locks		transactions
d%h	%m	i%h	%m	r/s	w/s	cur/max	cur/max	cur	l%h	%m	dead	act	t/s	r/t	w/t
99	1	99	1	1	1	2581/10000	2/128	0	100	0	0	1	36	0	0
99	1	99	1	2	0	2581/10000	2/128	0	100	0	0	1	35	0	0
99	1	99	1	2	0	2581/10000	2/128	0	100	0	0	1	28	0	0
99	1	99	1	0	0	2581/10000	2/128	0	100	0	0	1	22	0	0

The columns shown in this report are described as follows:

d%h Data cache hit rate

%m Data cache miss rate [100 - Data cache hit rate]

i%h Index cache hit rate

%m Index cache miss rate [100 - Index cache hit rate]

r/s Disk reads per second

w/s Disk writes per second

cur Current number of open files

max Server limit on number of open files

cur Current number of client connections

max Server limit on number of client connections

cur Number of locks currently held

l%h Lock hit rate [(lock attempts - locks blocked - locks denied) / lock

attempts]

%m Lock miss rate [100 - Lock hit rate]

dead Number of dead locks detected

act Current number of active transactions

t/s Number of transactions per second

r/t Number of read operations per transaction

w/t Number of write operations per transaction

log save Avg log save time per interval (microseconds) when used with the delta (-d) option

Tivoli-System Report -vts

The Tivoli-system (-vts) report displays FairCom DB Server system-wide statistics in the areas of cache usage, disk I/O, open files, established client connections, file locks, and transactions. The Tivoli-system report displays much of the same statistics that the admin-system (-vas) report displays, but in a format appropriate for input to tools such as the Tivoli monitoring application.

Example

Below is a sample Tivoli-system report produced by executing the command:

ctstat -vts -u ADMIN -p ADMIN -s FAIRCOMS -h 10 -i 2

#%cachehit %cachemiss r/s w/s maxfiles openfiles totalconnections activetransactions numdeadlock trans-r/s trans-w/s %hashhit %hashmiss transactions/s

92 8 0 0 10000 18 1 0 0 0 0 100 0 0

92 8 0 9 10000 18 1 0 0 0 17 100 0 1

92 8 0 0 10000 18 1 0 0 0 0 100 0 1

92 8 0 1 10000 18 1 0 0 0 1 100 0 1

92 8 0 0 10000 18 1 0 0 0 0 100 0 1

Note: The header line shown in this example is written as a single output line although it may be shown on multiple lines here.

The columns shown in this report are described as follows:

%cachehit Data and index cache hit rate

%cachemiss Data and index cache miss rate

r/s Disk reads per second

w/s Disk writes per second

maxfiles Server limit on number of open files

openfiles Current number of open files

totalconnections Current number of client connections

activetransactions Current number of active transactions

numdeadlock Number of dead locks detected

trans-r/s Number of read operations per transaction

trans-w/s Number of write operations per transaction

%hashhit Lock hit rate

%hashmiss Lock miss rate

transactions/s Number of transactions per second

Admin-File Report -vaf

The admin-file (-vaf) report displays FairCom DB Server statistics for the specified file. Note that multiple data or index files can be specified on the command line. Below is a sample admin-file report produced by executing the command:

ctstat -vaf mark.dat mark.idx -u ADMIN -p ADMIN -s FAIRCOMS -h 10

r/s w/s entries locks l%h %m dlock recrd node t filename

2 4 11863 4 100 0 0 128 n/a F mark.dat

0 4 11863 2 96 4 0 n/a 32768 I mark.idx

1 3 12192 4 100 0 0 128 n/a F mark.dat

0 9 12192 3 97 3 0 n/a 32768 I mark.idx

2 4 12730 5 100 0 0 128 n/a F mark.dat

0 3 12730 1 97 3 0 n/a 32768 I mark.idx

2 4 13236 5 100 0 0 128 n/a F mark.dat

0 2 13236 0 97 3 0 n/a 32768 I mark.idx

The columns shown in this report are described as follows:

r/s Disk reads per second for the file

w/s Disk writes per second for the file

entries Number of data records or key values in file

locks Number of locks currently held on file

l%h Lock hit rate for the file

%m Lock miss rate for the file

dlock Number of dead locks detected for the file

recrd Record length if data file, otherwise n/a

node Node size if index, otherwise n/a

t File type (F=fixed-length data, V=variable-length data, I=index)

filename Name of the file

Tivoli-File Report -vtf

The Tivoli-file (-vtf) report displays FairCom DB Server statistics for the specified file in a format appropriate for input to tools such as the Tivoli monitoring application.

Below is a sample Tivoli-file report produced by executing the command:

ctstat -vtf mark.dat mark.idx -u ADMIN -p ADMIN -s FAIRCOMS -h 10

#r/s w/s currentlocks waitinglocks filename

0 0 5 0 mark.dat

0 0 1 1034 mark.idx

1 3 4 0 mark.dat

0 6 0 1120 mark.idx

3 5 5 0 mark.dat

0 0 0 1208 mark.idx

2 4 4 0 mark.dat

0 0 2 1324 mark.idx

2 4 5 0 mark.dat

0 3 2 1402 mark.idx

The columns shown in this report are described as follows:

r/s Disk reads per second for the file

w/s Disk writes per second for the file

currentlocks Number of locks currently held on file

waitinglocks Cumulative lock wait count

filename Name of the file

Admin-User Report -vau

The admin-user report, -vau user..., displays FairCom DB Server statistics for the specified users. All existing connections whose user ID match the specified user ID are displayed.

Example

Below is a sample admin-user report produced by executing the command:

ctstat -vau GUEST -u ADMIN -p ADMIN -s FAIRCOMS -h 10

log function sec fil lok l%h %m dlock tid/uid/nodename

7s TRANEND 0 2 1 98 2 0 GUEST/10/

7s ADDREC 0 2 2 98 2 0 GUEST/12/

7s ADDREC 0 2 1 98 2 0 GUEST/13/

7s ADDREC 0 2 0 98 2 0 GUEST/14/

0s ctSNAPSHOT 0 2 0 0 0 0 ADMIN/15/ctstat

7s ADDREC 0 2 0 98 2 0 GUEST/17/

The columns shown in this report are described as follows:

log Total logon time for client

function Function client is currently executing

sec Current function request time

fil Current number of files open by this client

lok Current number of locks held by this client

l%h Lock hit rate for this client

%m Lock miss rate for this client

dlock Number of deadlocks detected for this client

tid/uid/nodename Server thread ID/User ID/Node name for this client

Function Timing Report -func

The function timing report (-func) displays FairCom DB Server statistics for each c-tree function that a client has called at least once since function timing was enabled. See -wrktime.

Below is a sample function timing report produced by executing the command:

ctstat -func -u ADMIN -p ADMIN -s FAIRCOMS -h 10

function counter secs

FRSVSET 10 0.002

NXTVSET 20 0.001

GETDODAX 2 0.000

COMMBUF 1 0.000

ctSNAPSHOT 10 0.002

The columns shown in this report are described as follows:

function Function name

counter Cumulative number of times this function has been called

secs Cumulative elapsed time for this function

Text Report -text

The following command generates a report in text format:

ctstat -text -u ADMIN -p ADMIN -s FAIRCOMS -h 10

This command writes a FairCom DB Server system snapshot to the file SNAPSHOT.FCS (output to the data folder). See the file SNAPSHOT.FCS for the detailed server statistics.

Example Output (truncated for brevity)

Server Shutdown System Snapshot

Wed May 22 15:36:41 2019

server version of structure: 21

bit map of var len contents: 0x

snapshot time stamp (seconds since 1970): 1558557401

elapsed server operation time: 173106.444

high res timer ticks per sec: 10000000

compatibility flag #1: 00002000x

compatibility flag #2: 00020000x

compatibility flag #3: 00000000x

compatibility flag #4: 00000000x

diagnostic flag #1: 40000000x

diagnostic flag #2: 00000000x

diagnostic flag #3: 00000000x

ct_udefer yield threshold (usec): 0

ct_udefer 64 yield duration (usec): 0

delete node thread queue reads: 2908

delete node thread queue writes: 2908

delete node thread queue rewrites: 0

delete node thread queue abandons: 0

delete node thread queue removals: 0

delete node thread queue no action: 0

count of lock attempts: 21708

subcount of hdr lock attempts: 2135

count of locks freed: 21707

count of locks denied: 0

count of locks blocked: 0

subcount of header blocks: 0

count of blocks released: 0

count of dead locks: 0

count of killed locks: 0

current count of locks held: 1

current count of blocked requests: 0

maximum count of locks held: 161

COMMIT LOCK loop defer (millisecs): 10

COMMIT LOCK loopers: 0

COMMIT LOCK total loop count: 0

cumulative blk record lock count: 0

cumulative blk record lock time: 0.000 (seconds)

maximum blk record lock time: 0.0000

cumulative blk index lock count: 0

cumulative blk index lock time: 0.000 (seconds)

maximum blk index lock time: 0.0000

cumulative transaction count: 369

cumulative transaction time: 534.255 (seconds)

average transaction time: 1.44784

transaction time detail:

122 < 0.05000

8 < 0.10000

5 < 0.15000

3 < 0.20000

16 < 0.25000

63 < 0.30000

69 < 0.35000

20 < 0.40000

11 < 0.45000

12 < 0.50000

5 < 0.55000

2 < 0.60000

1 < 0.65000

0 < 0.70000

0 < 0.75000

0 < 0.80000

1 < 0.85000

0 < 0.90000

0 < 0.95000

0 < 1.00000

1 < 1.05000

0 < 1.10000

1 < 2.00000 [avg of 0 per unit width]

29 >= 2.00000

maximum transaction time: 102.2966

checkpoint delay (microseconds): 0

commit delay block to clear ration: 2

commit delay cohort size base: 50

commit delay nominal time (microseconds): 2000

I/O Time Statistics -iotime

The FairCom DB Server SNAPSHOT feature includes support for collecting disk read and write timings on a per-file basis when high-resolution timer support is activated. Use the ctstat utility’s -iotime option to toggle the collection of disk I/O timings.

Turn on disk I/O timings:

# ctstat -iotime on -u ADMIN -p ADMIN -s FAIRCOMS

Turn off disk I/O timings:

# ctstat -iotime off -u ADMIN -p ADMIN -s FAIRCOMS

The ctstat utility’s -vaf option also outputs differential I/O timings for each file when the FairCom DB Server returns version 2 (or higher) GFMS structure statistics.

Example

C:\> ctstat -vaf mark.dat mark.idx -h 1 -i 10

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

0 0 0 0 26239 1 100 0 0 128 n/a F mark.dat

0 0 0 0 26239 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

128 237 1 12 108309 2 100 0 0 128 n/a F mark.dat

0 2 0 0 108308 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

121 243 1 14 186164 2 100 0 0 128 n/a F mark.dat

2 27 0 4 186163 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

109 219 1 10 256356 2 100 0 0 128 n/a F mark.dat

3247 3296 39 77 256355 0 99 1 0 n/a 8192 I mark.idx

r/s w/s read time write time entries locks l%h %m dlock recrd node t filename

103 206 1 10 322381 4 100 0 0 128 n/a F mark.dat

5623 5640 67 124 322380 1 99 1 0 n/a 8192 I mark.idx

I/O Statistics per File -file

The FairCom DB Server SNAPSHOT feature supports a mode to write snapshot statistics for all files open by the FairCom DB Server to disk. Use the ctstat utility’s -file option. The snapshot statistics for all open files are then written to the SNAPFILE.FCS file in comma-delimited or human-readable format.

CSV Example

Write statistics for all open files to SNAPFILE.FCS in comma-delimited format using the ctstat utility:

# ctstat -file csv -i 1 1

Sample SNAPFILE.FCS Contents

On-Demand File Snapshot

Mon Jun 25 16:40:51 2007

physical file size,logical file size,serial number,active entries,tran high mark,update timestamp,max file size,read ops,bytes read,write ops,bytes written,memory file high bytes,read time (msec.),write time (msec.),index height,file id,server id,time id,node size,record length,permanent file mode,max leaf key bytes,max non-leaf key bytes,file type,key length,key member number,number of members, super file type,max leaf marks,wrthdr sequence number,total lock attempts,header lock attempts,total lock wait count,header lock wait count,deadlocks,total locks denied,total locks freed,total blocks released,current locks held,current blocked requests,max special cache pages,current special cache pages,number of buffer pages,number of data cache pages,number of channels,number of users with file open,current memory record count,highest memory record count,killed locks,max segments,active segments,update flag,file type,duplicate key flag,index delete type,key padding byte,flavor,alignment,pointer size,file name

16384,16384,0,15,0,0,0,0,0,5,16768,0,0,0,-1,0x00000000,0x00000000,0x00000000,8192,0,0x00000000,8148,8174,1,12,0,0,0,2048,3,44,23,0,0,0,0,44,0,0,0,0,0,0,0,1,1,0,0,0,1,0,0xff,0,0,0,32,2,8,4,I0000001.FCS

====================

Human Readable Example

Write statistics for all open files to SNAPFILE.FCS in human-readable format using the ctstat utility:

# ctstat -file -i 1 1

Sample SNAPFILE.FCS Contents

On-Demand File Snapshot

Tue Jun 26 09:58:28 2007

phyrec numrec sernum nument hghtrn tstamp mxfilz fredops fredbyt fwrtopts fwrtbyt mhghbyt fredtim fwrttim idxhgt fileid servid timeid nodsiz reclen logtyp maxkbl maxkbn filtyp keylen kmem nmem suptyp maxmrk hdrseq floktry flokhlk flokblk flokhbk flokdlk flokdny flokfre flokrel flokcur fblkcur datlmt datspl bufcnt datcnt numchn fusrcnt memcnt hghcnt flokkil segmax seglst updflg ktype autodup deltyp keypad flflvr flalgn flpntr filename

16384 16384 0 15 0 0 0 1 8192 10 57728 0 0 0 -1 0x00000000 0x00000000 0x00000000 8192 0 0x00000000 8148 8174 1 12 0 0 0 2048 3 68 35 0 0 0 0 68 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0xff 0 0 0 32 2 8 4 I0000001.FCS

====================

Connection Information -userinfo and -userinfox

Connection reporting options -userinfo and -userinfox are available to display additional statistics about existing user connections.

New information included in this alternative output:

The status and idle time of the connection.
The last FairCom DB Server request made.
Time spent in a transaction.
Amount of memory consumed by the client.
Number of files open by the client.
The time the user has been logged in.
User ID, Thread ID, and Nodename of the user.
User communication protocol used to connect to the FairCom DB Server. Because the column labeled uid/tid/nodename is not fixed in size, the space between nodename and commprotocol is a tab character. Even if the output is not properly aligned, the parsing will be easy by looking for a '\t' (tab).

-userinfo Example

ctstat.exe -userinfo -u ADMIN -p ADMIN -s FAIRCOMS -h 5

status lastrequest trntime mem fils time uid/tid/nodename commprotocol

0s busy ADDVREC 1s 596K 46 11s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m00s idle tmexec -- 7626K 7 58m00s ADMIN/23/SLCDESK SQL_SHAREMM

0s busy ADDVREC 1s 189K 46 11s ADMIN/25/cttctx(p26576,t9220) FSHAREMM

0s busy DELVREC 0s 185K 46 11s ADMIN/26/cttctx(p26576,t31684) FSHAREMM

0s busy DELVREC 0s 186K 46 11s ADMIN/27/cttctx(p26576,t28564) FSHAREMM

0s busy DELVREC 0s 187K 46 11s ADMIN/28/cttctx(p26576,t27780) FSHAREMM

0s busy ADDVREC 0s 592K 46 12s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m01s idle tmexec -- 7626K 7 58m01s ADMIN/23/SLCDESK SQL_SHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/25/cttctx(p26576,t9220) FSHAREMM

0s busy ADDVREC 0s 185K 46 12s ADMIN/26/cttctx(p26576,t31684) FSHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/27/cttctx(p26576,t28564) FSHAREMM

0s busy ADDVREC 0s 186K 46 12s ADMIN/28/cttctx(p26576,t27780) FSHAREMM

0s busy DELVREC 0s 593K 46 13s ADMIN/22/cttctx(p26576,t27692) FSHAREMM

58m02s idle tmexec -- 7626K 7 58m02s ADMIN/23/SLCDESK SQL_SHAREMM

-userinfox Example