Determines if each occurrence of an internal FairCom Server checkpoint will cause a time stamp message to be sent to the FairCom Server console screen and to the CTSTATUS.FCS file.
CTSTATUS_MASK (CTSTATUS_MASK, CTSTATUS_MASK)
Allows certain types of entries in CTSTATUS.FCS to be suppressed.
Controls the size of the FairCom Server status file.
CTSTATUS_LOCATION
Sets the location of CTSTATUS.FCS.
CTSTATUS_ROTATE
Enables rotation of CTSTATUS.FCS according to a specific time interval.
Enables FairCom DB to monitor available disk space and to shut down the database engine when the disk space falls below the specified limit.
Activates a disk space threshold mechanism to detect when a disk volume is getting full (the specified number of bytes must remain available on a disk volume after a file has been extended).
Allows volume-specific disk full checks.
Causes the client number, function number, function name, and file name are displayed in a scrolling fashion on the FairCom Server console screen.
Enables additional tracking and checking of internal memory suballocations - Consult with FairCom support team as this option can significantly impact performance in certain use cases.
Deprecated: Monitors the number of active record locks.
Sends a message to the console whenever allocated memory exceeds the next memory threshold.
Sends debug output to the console every time the net memory allocation count changes by a multiple of the threshold value.
Suppress the message sent to the system console if a file without a matching name does match the unique file ID when a file open is attempted.
Allows entries to be placed on a queue for client side processing and monitoring of server events.
Specifies a request time in seconds for monitoring function that exceed the specified time.
Captures user information pertaining the to file specified to SNAPSHOT.FCS in addition to the system snapshot.
Enables automatic snapshots at specified intervals.
Changes the default histogram intervals (box width) for the lock waiting time histograms.
Controls the width of the fixed transaction log flush time histogram, and the initial width of the adaptive transaction log flush time histogram.
Changes the default histogram intervals (box width) for the transaction time histogram.
Captures user information to SNAPSHOT.FCS in addition to the system snapshot.
Specifies a keyword indicating contents to be stored in the System Event Log.
Enables system status reports triggered by the specified event.
Enables system status reports triggered by the specified event.
Diagnostics
Diagnostics keywords are intended to be used to help collect data to be analyzed for troubleshooting. The use of these keywords should be avoided in normal operation as they may cause additional overhead (e.g., logging).
Causes a time stamp message goes to the FairCom Server console screen each time the FairCom Server detects and resolves a dead lock.
DIAGNOSTICS SNAPSHOT_AUTOMATIC
Write system snapshots to the human-readable SNAPSHOT.FCS text file.
Enables collecting disk read and write timings on a per-file basis when high-resolution timer support is activated.
Write system snapshots to the human-readable SNAPSHOT.FCS text file.
Enables collecting FairCom DB function call counts and timings on a per-c-tree file basis.
CHECKPOINT_MONITOR
CHECKPOINT_MONITOR <YES | NO | DETAIL>
This keyword takes one of three arguments: YES, NO, and DETAIL. If YES, each occurrence of an internal FairCom DB Server checkpoint will cause a time stamp message to be sent to the FairCom DB Server console screen and to the CTSTATUS.FCS file. The checkpoint is a snapshot of the FairCom DB Server at an instance in time and is used during automatic recovery. The checkpoint provides for a measure of the system activity. The DETAIL argument causes six intermediate milestones to be output for each checkpoint in addition to the beginning and ending checkpoint messages. These intermediate outputs aid in analyzing how the checkpoint procedure interacts with applications. If there is no system activity, no checkpoints will occur. This keyword should be used for debugging purposes only since performance may be compromised.
Default: NO
CTSTATUS_LOCATION
CTSTATUS_LOCATION <directory>
CTSTATUS_LOCATION sets the location of the status log file, CTSTATUS.FCS. If not specified, or if set to a relative path, the location of CTSTATUS.FCS will be relative to the LOCAL_DIRECTORY directory.
CTSTATUS_MASK
CTSTATUS_MASK <mask>
Allows certain types of entries in CTSTATUS.FCS to be suppressed.
Accepted <mask> values
| DYNAMIC_DUMP_FILES | Suppress the logging of the names of the files backed up by a dynamic dump operation. |
| MATCH_FILE_ID |
Suppresses "Matching file Ids" messages. Such messages can be output several situations:
|
| PRIME_CACHE_AT_STARTUP | Suppress log messages while cache priming is enabled. |
| WARNING_AUTO_TRNLOG |
Suppresses the logging of the message: "WARNING: could not turn on AUTO_PREIMG/AUTO_TRNLOG for ... (filename)" when the AUTO_PREIMG or AUTO_TRNLOG configuration option matches the name of a c-tree index file that was not created with support for transaction control. |
| WARNING_FCRP_ERR | Eliminates FCRPT_ERR (14) errors in CTSTATUS.FCS. |
| WARNING_PAGESIZE | Disables logging PAGE_SIZE warning messages to CTSTATUS.FCS, |
| WARNING_PAGESIZE_SEEALL | Logs all PAGE_SIZE warnings to CTSTATUS.FCS for debugging. |
| WRITE_ERR |
Suppresses WRITE_ERR messages. |
| VDP_ERROR | Suppresses logging of communications errors. |
Default: Log all messages.
CTSTATUS_ROTATE
CTSTATUS_ROTATE <DAILY | WEEKLY [<WEEKDAY>] | MONTHLY [<MONTHDAY>] | YEARLY> KEEP <<NUMBER_TO_KEEP> | ALL>
CTSTATUS_ROTATE enables rotation of the status log file, CTSTATUS.FCS, according to a specified time interval:
- DAILY rotation occurs at midnight.
- WEEKLY rotation occurs at midnight on Sunday or the specified WEEKDAY (SUN, MON, TUE, WED, THU, FRI, SAT).
- MONTHLY rotation occurs at midnight on the first day of the month or the specified MONTHDAY (between 1 and 28 inclusive).
- YEARLY rotation occurs at midnight on January 1.
<NUMBER_TO_KEEP> is the number of logs that will be kept in addition to the current log. If set to ALL, then archived logs are not removed.
CTSTATUS_SIZE
CTSTATUS_SIZE <file_size | negative_file_size | 0>
CTSTATUS_SIZE controls the size of the FairCom DB Server status file. The argument to CTSTATUS_SIZE is the approximate maximum size in bytes for CTSTATUS.FCS. When this limit is reached, CTSTATUS.FCS is renamed to T0000001.FCS and a new status file is created. The T#.FCS file numbers increase each time the limit is reached, similar to the transaction log files, i.e., the next time the maximum size is reached, CTSTATUS.FCS is renamed to T0000002.FCS.
To limit the number of archived status logs, set a negative value for CTSTATUS_SIZE. Only T0000001.FCS will be kept, being replaced each time CTSTATUS.FCS is archived.
A value of 0 allows the file to expand to a size limited by the operating system and storage availability.
Default: -32000000
DBENGINE_CHECK
In V11 and later, this Replication Agent configuration keyword enables monitoring of changes to the FairCom DB "engine":
DBENGINE_CHECK = 1
Enables tracking changes to the ctsrvr.cfg, CTSTATUS.FCS, *.lib, and *.dll.
Schedules a future action for double checking the FairCom DB API Engine changes. It is scheduled with a delay of 5 seconds by default, and all the pending checks for a given c-tree installation are grouped to avoid several re-checks. The action execution is able to change an existing installation, add a new one, or delete an old one.
DEADLOCK_MONITOR
DEADLOCK_MONITOR <YES | NO>
If YES, each time the FairCom DB Server detects and resolves a dead lock situation, a time stamp message goes to the FairCom DB Server console screen.
Note: This keyword is used primarily for debugging since this feature consumes additional overhead.
Default: NO
DISK_FULL_ACTION (Original)
DISK_FULL_ACTION <action> <volume> <limit>
Enables FairCom DB to monitor available disk space and to shut down the database engine when the disk space falls below the specified limit.
This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.
You can include more than one DISK_FULL_ACTION in a server configuration.
Scriptable actions are supported. See DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM)).
This keyword can be specified multiple times.
This keyword is independent of the DISK_FULL_LIMIT and DISK_FULL_VOLUME keywords.
Windows Examples
DISK_FULL_ACTION shutdown D:\ 500 MB
DISK_FULL_ACTION shutdown "C:\Users\Administrator\My Documents" 1 GB
Unix Examples
DISK_FULL_ACTION shutdown /users/administrator 500 MB
DISK_FULL_ACTION shutdown "/users/administrator/my documents" 1 GB
Messages
The following messages appear in CTSTATUS.FCS when this option is used:
FairCom DB logs a message for each DISK_FULL_ACTION option it finds in ctsrvr.cfg:
- User# 00001 Configuration info : DISK_FULL_ACTION threshold set for volume S:\ to 7516192768 bytes
FairCom DB logs a message to indicate that the available disk space monitoring thread has started:
- User# 00009 Thread Start: ctDISKFULLactionthrd
FairCom DB logs a message when it detects disk space below the specified threshold which causes the database engine to shut down:
- User# 00009 DISK_FULL_ACTION: Space on volume S:\ is 5338173440, which is below threshold of 7516192768. Initiating database engine shutdown.
See Also
- DISK_FULL_LIMIT
- DISK_FULL_VOLUME
- DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))
DISK_FULL_ACTION (SUBSYSTEM)
FairCom Server supports a configuration option to call an external script when a specified DISK_FULL_LIMIT or DISK_FULL_VOLUME limit is reached. This allows server administrator to configure an external process to run when a disk-full limit is reached, for example, the process could write information to a log or notify the administrator.
To configure an external process to run when the space on the specified volume is below the specified limit, set the DISK_FULL_ACTION configuration option in ctsrvr.cfg with the run action.
The supported syntax is:
SUBSYSTEM EVENT DISK_FULL_ACTION {
volume VOLUME
limit LIMIT
run EXE [OPTIONS]
freq FREQUENCY
maxruntime MAXRUNTIME
}
The volume, limit, and run parameters are required. The freq and maxruntime are optional.
- VOLUME is the name of the disk volume to check: a drive name (for example, C:\) on Windows systems, or a directory name (for example, /users/faircom/data) on Unix/Linux systems.
- LIMIT is the threshold of available space that triggers the running of the script. The KB, MB, GB, TB, and PB suffixes can be used for this value (for example: "limit 100 GB" means 100 gigabytes).
- EXE is the name of an executable, batch file, or Unix shell script to run, and OPTIONS are optional command-line options to pass to the external program.
- FREQUENCY indicates the frequency at which this condition is checked. The default is to check the condition every 60 seconds. Minimum frequency is 1; maximum frequency is 32,000,000.
- MAXRUNTIME indicates the maximum amount of time, in seconds, that the external script is permitted to run. If the external script has not terminated after the specified amount of time, the server terminates the script. The default is 60 seconds. A setting of 0 means the script will never be forced to terminate.
Note: When setting a MAXRUNTIME value, consider that no other DISK_FULL_ACTION checks will take place until the script completes, regardless of the FREQ setting. This is of particular importance if more than one DISK_FULL_ACTION is created.
In V12 and later, disk full monitoring keywords have been added to the default ctsrvr.cfg. These keywords are present in this config file, but they have been commented out for so they are there if you need them, but they are NOT enabled.
; SUBSYSTEM EVENT DISK_FULL_ACTION {
volume VOLUME
limit LIMIT
run EXE [OPTIONS]
freq FREQUENCY
maxruntime MAXRUNTIME
}
Example DISK_FULL_ACTION with external process action
The following configuration option causes the batch file diskfull.bat to be run when writing to a c-tree data or index file detects that the space on C:\ has dropped below 1 GB. This disk-full condition is checked once every 60 seconds.
SUBSYSTEM EVENT DISK_FULL_ACTION {
volume C:\
limit 1 GB
run diskfull.bat
freq 60
maxruntime 120
}
Reading parameters sent from the server to the external process
FairCom Server passes a set of ASCII key/value pairs to the external process using standard input, where the input format is key=value, one per line. The keys that FairCom Server passes to the external process are:
- volume - The VOLUME setting from the DISK_FULL_ACTION configuration option.
- limit - The disk-full limit in bytes that was specified for that volume.
- available - The amount of disk space in bytes that is currently available on that volume.
Writing parameters from the external process to the server
The external process can optionally pass ASCII key/value pairs back to the server by writing them to standard output in the format key=value, one entry per line. FairCom Server recognizes the following keys:
- action=continue - (indicating to continue FairCom Server operation; this is the default)
or
- action=shutdown - (indicating to shut down FairCom Server)
- comments=MESSAGE - A user-defined message that will be written to CTSTATUS.FCS.
Note: FairCom Server reads only the first 1024 bytes of the output data returned by the script. All data after the first 1024 bytes of output data is ignored.
If the external script's output is not in the supported format, a message is written to CTSTATUS.FCS. Example:
DISK_FULL_ACTION: Command diskfull.bat output does not match format var=value: line 3 column 8
Unlimited MAXRUNTIME script
An unlimited script runtime can be set with MAXRUNTIME 0. When configured as unlimited and a script is run due to the LIMIT and RUN options, the script will never be forced to terminate.
Note: When setting a MAXRUNTIME value, consider that no other DISK_FULL_ACTION checks will take place until the script completes, regardless of the FREQ setting. This is of particular importance if more than one DISK_FULL_ACTION is created.
DISK_FULL_LIMIT
DISK_FULL_LIMIT <bytes available>
FairCom DB Servers for Unix and Windows support the DISK_FULL_LIMIT keyword, which activates a disk space threshold mechanism to detect when a disk volume is getting full. The DISK_FULL_LIMIT configuration keyword takes as its argument the number of bytes that must remain available on a disk volume after a file has been extended. If the update operation fails, a message is written in CTSTATUS.FCS naming the file involved.
FairCom DB Servers that do not support this feature ignore the DISK_FULL_LIMIT keyword.
Note: When the disk is full, the FairCom DB files cannot be extended. The FairCom DB operations that attempt to add or update data in a file and require the files to be extended will fail with specific errors. This diagnostic keyword is used to put supplemental information in the status file and alert the user.
Default: No disk full check
See Also
- DISK_FULL_VOLUME
- DISK_FULL_ACTION
- DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))
DISK_FULL_VOLUME
DISK_FULL_VOLUME /path
Allows volume-specific disk full checks. DISK_FULL_VOLUME takes as its argument a concatenated volume name and limit. A path separator must occur between the volume name and the threshold limit, which may be zero.
In Unix this is in the form /name/<limit>. The following example places a disk full threshold of one million bytes on the volume /home:
DISK_FULL_VOLUME /home/1000000
For Windows, the form of the argument is <DRIVE>:\<limit>. The following example places a 1MB threshold on drive E:
DISK_FULL_VOLUME e:\1048576
Note: When the disk is full the FairCom DB files cannot be extended. The c-tree operations that attempt to add or update data in a file and require the files to be extended will fail with specific errors. This diagnostic keyword is used to put supplemental information in the status file and alert the user.
Default: Off
See Also
- DISK_FULL_LIMIT
- DISK_FULL_ACTION
- DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))
FUNCTION_MONITOR
FUNCTION_MONITOR <YES | NO | file_name>
If YES, the client number, function number, function name, and file name are displayed in a scrolling fashion on the FairCom DB Server console screen. Alternatively, the same information, along with the return value and error codes for each function, can be routed to a file by specifying a file name. This keyword should be used primarily for debugging since this feature consumes additional overhead.
Note: Activate the function monitor dynamically under the FairCom DB Server for Windows by selecting View > Function Monitor Window.
Default: NO
Notes:
FUNCTION_MONITOR can be enabled without restarting the server using the ctadmn command-line tool as follows:
- Start the ctadmn tool and connect to the server.
- Select the following from the menu:
10 "Change Server Settings"
- Select the following from the next menu:
1 "Configure function monitor"
- Enter fmon.log at the prompt:
Enter the new function monitor value.
Allowed values are: YES or NO or filename
Function monitor information will be written to the FairCom DB process working directory in a text file named fmon.log.
When you are done, be sure to shut off function monitoring: Repeat the steps above, but disable function monitoring by entering NO in the final step.
HEAP_DEBUG_LEVEL
FairCom Database Engine heavily utilizes memory operations to optimize performance. To avoid frequent OS memory requests, it maintains a sophisticated memory suballocator. This internal subsystem allocates larger blocks of memory from the O/S and sub allocates them as needed. All memory gets and puts from this suballocator system are tagged, tracked and checked. On rare occasions, heap memory corruption can occur resulting in an eventual illegal memory access attempt, ultimately terminating FairCom DB operations. Debugging these conditions is extremely complex. FairCom DB V12 and FairCom RTG V3 enhance memory diagnostics when required.
Debugging options can now be enabled when corrupted memory is suspected triggering additional tracking and checking of internal memory suballocations. Use of these options should only be considered in consultation with your FairCom support team as these can significantly impact performance in certain use cases.
HEAP_DEBUG_LEVEL <level> with values 0-3 (Default 0):
- HEAP_DEBUG_LEVEL 0 uses the default memory allocator; no checks are performed.
- HEAP_DEBUG_LEVEL 1 enables the debug allocator, with some detection for memory buffer write overruns or use-after-free bugs. There is no additional memory overhead, and the CPU overhead is small.
- HEAP_DEBUG_LEVEL 2 does the checks from HEAP_DEBUG_LEVEL 1 and adds a small redzone before/after each allocation, which is checked at free. This may detect some buffer write overruns, write underruns, or double frees. This option uses an additional 16 bytes of memory on each allocation.
Levels 1 and 2 generate a CTSTATUS entry and stack trace at free when a memory fault is detected.
- HEAP_DEBUG_LEVEL 3 uses the system's virtual memory subsystem to immediately detect illegal memory overruns (both reads and writes) by aligning allocations at the end of a 4K page. The following types of errors may immediately generate a segmentation fault (core dump) on the invalid access: memory read/write overrun, double free, and use-after-free.
Memory write underruns or some small write overruns may instead be detected at free and generate a CTSTATUS message and stack trace.
Memory allocation failures are logged to CTSTATUS logging when HEAP_DEBUG_LEVEL 3 is enabled.
Because Level 3 debugging uses an extra 4K-8K bytes for all allocations, it is possible to restrict this to particular size(s) of allocations using HEAP_DEBUG_EXCLUSION_LIST <N>.
Be sure to see the Performance Note below.
HEAP_DEBUG_EXCLUSION_LIST <N> [,<N> ...] - When HEAP_DEBUG_LEVEL 3 (only) is specified, it is possible to have particular allocation sizes use the default (non-debug) allocator to reduce the overall memory overhead of this debugging method. By default, HEAP_DEBUG_LEVEL applies to all allocation sizes. The particular values to specify would typically be recommended by FairCom support based on analysis of prior core dumps.
HEAP_DEBUG_EXCLUSION_LIST # allocation size range (in bytes):
# Bytes
1 <=16
2 17-32
3 33-64
4 65-128
5 129-256
6 257-512
7 513-1024
8 1025-2048
9 2049-4096
10 4097-8192
Limitations: When Level 3 debugging is enabled, memory usage statistics are not tracked for those bins.
Performance Note
HEAP_DEBUG_LEVEL 3 has a negative impact on performance on any machines with 4 or more CPU cores. FairCom does not recommend the use of Level 3 on any machines with 4 or more CPU cores where performance is important.
If performance is an issue, FairCom recommends using the Level 2 version of heap debug on any location where you've seen a memory-related crash occur and where performance is important. Our testing has not revealed a noticeable performance impact with Level 2, even on a large test box with 72 cores.
Note 1:
If allocation stack traces are enabled (such as with ctstat -mt +ALL), an additional stack is dumped showing the allocation location of the buffer.
A use-after-free might cause a segmentation fault (core dump).
Note 2:
HEAP_DEBUG_LEVEL 2 has a benefit to stability: If the buffer overrun is 8 bytes or less, the heap will NOT be corrupted because only the extra redzone memory is modified. The server will stay up and only a stack trace will be generated.
Linux
On Linux, the HEAP_DEBUG_LEVEL 3 configuration option requires raising the kernel limit vm.max_map_count to be twice the number of allocations. FairCom can help you to determine the minimum value to set the Linux kernel parameter, vm.max_map_count. (This value will generally rise and fall with the amount of server activity, so include a safety factor based on how much busier it might be.) Setting vm.max_map_count too small will result in allocation failures if the number of allocations exceeds that limit, which could lead to a server crash.
This can be changed until next system reboot with the command:
sysctl -w vm.max_map_count=<N>
Example using HEAP_DEBUG_LEVEL 3 restricted to sublist #2:
Look at current memory usage at a busy server time using this command:
ctstat -ml -u admin -p ADMIN_PASSWD -s FAIRCOMS
The output will show sublist #2 as "PI2TYP." The next column shows the total allocations in bytes.
PI2TYP 98304 7400 0.01%
Divide the first column by 32 (the max size of this allocation range) to get the number of current allocations on this sublist: 98304/32=3072, and then double that: 6144.
You also need to include the MBATYP:
MBATYP 73388776- 73388776-
Divide the first column by 8192 (this would be the worst case assumption for this list), and double the result:
73388776/8192 * 2 = 17916
The sum of these values is 2x the allocation count: 17916+6144 = 24060. Apply a safety multiplier such as x10.
This is the minimum value you need to set as the Linux kernel parameter:
vm.max_map_count
This value will generally rise and fall with the amount of server activity, so include a safety factor based on how much busier it may become. Setting vm.max_map_count too small will result in allocation failures if the number of allocations exceeds that limit, which could lead to a server crash.
After setting the kernel value (Linux only) with vm.max_map_count, add the following server keywords to enable the debug suballocator on sublist #2 (PI2TYP) only:
HEAP_DEBUG_LEVEL 3
HEAP_DEBUG_EXCLUSION_LIST 1,3,4,5,6,7,8,9,10
Stack Dump Message
When a Stack Dump is generated, the following message will be logged to CTSTATUS.FCS.
"A Heap Fault was detected and stack dump created"
If you have this logic enabled, FairCom recommends routine reviews of CTSTATUS.FCS to determine if a Stack Dump has been created. If you see this message, please send the Stack Dump, and the CTSTATUS.FCS message to FairCom Support for inspection.
LOCK_MONITOR
Deprecated. See the notes below for other monitoring tools.
LOCK_MONITOR <threshold value>
Monitors the number of active record locks. This keyword, or the SetOperationState() function call, enables a lock monitor that indicates when the number of lock calls over unlocks equals a multiple of the threshold value, or if it goes below a threshold.
For example, LOCK_MONITOR 100 sends a message to the console each time the number of lock calls over unlocks equals a multiple of 100. Likewise, if the number of unbalanced lock calls falls below these thresholds, a message goes to the console. Ordinarily, when the number of locks are in balance (i.e., excess locks over unlocks equals zero) no message is routed to the console unless a message indicating an excess of locks has already been sent to the console. If you wish the message to be sent whenever the number of excess locks equals zero, enter the threshold value as a negative. For example, LOCK_MONITOR -100
Default: No lock monitor
Tools for Monitoring Locks
Several elements in the Snapshot Structure contain information about locks. For information about monitoring the state of locks, see the File Snapshot Structure in the FairCom DB Programmer's Reference Guide.
For a graphical view of locks and other system information, try using the FairCom DB Monitor and the FairCom DB Gauges available in your tools folder and in the Windows Start menu.
MEMORY_MONITOR
MEMORY_MONITOR <Bytes | NO>
Sends a message to the console whenever allocated memory exceeds the next memory threshold. The parameter specifies a size in bytes. For example, MEMORY_MONITOR 500000 sends a message every time memory consumption exceeds the next 500,000 byte range of memory. The message is also sent when memory usage decreases for each absolute memory block. This keyword should be used primarily for debugging, as there is some additional overhead for this feature.
Default: NO
MEMORY_TRACK
MEMORY_TRACK <allocation threshold value>
Sends debug output to the console every time the net memory allocation count changes by a multiple of the threshold value. The count is the number of memory allocation requests. See also DIAGNOSTICS TRACK_LOGON.
Default: 0 (indicates do not track)
MONITOR_MASK
MONITOR_MASK
When a file open is attempted, the FairCom DB Server checks to see if either a file with the same name has already been opened, or if a file with the same unique ID has already been opened. By default, if a file without a matching name does match the unique file ID FairCom DB sends a message to the system console indicating the names of the two files involved. Suppress this message by adding the following entry:
MONITOR_MASK MATCH_FILE_ID
Default: Enabled
MONITOR_MASK
MULTILINE_STATUS_LOG_MESSAGE YES | NO
Configure the server to write a single line message.
Thu Apr 11 17:07:04 2024 - User# 00001 Startup FairCom DB SQL Server - V13.0.0.125(Build-240406) OEM: 0x39
Thu Apr 11 17:07:04 2024 - User# 00001 For Windows 64bit Operating System
Thu Apr 11 17:07:04 2024 - User# 00001 LICENSE: License file in use: .\ctsrvr39001112.lic
Thu Apr 11 17:07:04 2024 - User# 00001 Using configuration file <../config/ctsrvr.cfg>
Thu Apr 11 17:07:04 2024 - User# 00001 DH_JVM_OPTION_STRINGS=-Xms100m;-Xmx300m;-XX:+UseG1GC
Thu Apr 11 17:07:04 2024 - User# 00001 Windows native reader/writer lock support enabled
Thu Apr 11 17:07:04 2024 - User# 00001 Process ID: 40200
Thu Apr 11 17:07:04 2024 - User# 00001 Alternative server name...
Thu Apr 11 17:07:04 2024 - User# 00001 FAIRCOMS
Thu Apr 11 17:07:04 2024 - User# 00001 Using configuration directory: ../config
Thu Apr 11 17:07:04 2024 - User# 00001 Compatibility bit maps: (1)00002000x (2)00020000x (3)00000000x (4)00000000x (5)00000000x
Thu Apr 11 17:07:04 2024 - User# 00001 Diagnostic bit maps: (1)40000000x (2)00000000x (3)00000000x
Thu Apr 11 17:07:04 2024 - User# 00001 64-bit File Address Support
Thu Apr 11 17:07:04 2024 - User# 00001 6 Byte Transaction Numbers
Thu Apr 11 17:07:04 2024 - User# 00001 Commit Read Lock Support
Thu Apr 11 17:07:04 2024 - User# 00001 NOWAIT usrsema enabled
Thu Apr 11 17:07:04 2024 - User# 00001 Smart File Sync Support
Thu Apr 11 17:07:04 2024 - User# 00001 ctIOSEMA I/O semaphore optimization is enabled
Thu Apr 11 17:07:04 2024 - User# 00001 ctIOSEMAmemfile I/O semaphore optimization is enabled
Thu Apr 11 17:07:04 2024 - User# 00001 ctFeatGNSEMAhsh index cache mutex optimization is enabled
Thu Apr 11 17:07:04 2024 - User# 00001 ctFeatDBSEMAhsh data cache mutex optimization is enabled
Thu Apr 11 17:07:04 2024 - User# 00001 Memory suballocator allocation unit: 16
Thu Apr 11 17:07:04 2024 - User# 00001 FairCom DB SQL Enterprise Edition
Historically, CTSTATUS.FCS log messages have had the timestamp on one line and the message on the next line.
Default: NO
PERF_MONITOR
PERF_MONITOR <option>
The SYSMON feature allows entries to be placed on a queue for client side processing and monitoring of server events. This option allows event entries to be specified in the ctsrvr.cfg configuration file. The following options are supported:
ALL_EVENTS
CHECKPOINT
LOG_EXTENSION
LONG_TRANSACTION
When checkpoints are monitored, an entry is written to the SYSMON_PERF client-side queue for the beginning and end of each checkpoint. Log extensions similarly generate begin and end entries. A long transaction generates one entry.
ALL_EVENTS is equivalent to listing each individual option.
When one or more event types have been enabled, then the first entry written into the SYSMON_PERF queue provides the information necessary to convert the high-resolution times into seconds and/or dates.
See Also
REQUEST_TIME_MONITOR
REQUEST_TIME_MONITOR <request_time_interval>
FairCom DB supports monitoring function request times that exceed a specified time limit with the REQUEST_TIME_MONITOR <request_time_interval> keyword. <request_time_interval> is a time in seconds. A value of -1 disables the feature, meaning that it cannot be dynamically turned on using the ctSETCFG() function, as described below. A non-negative value (the minimum value is 10) specifies a request time in seconds. An internal thread checks for requests whose time has exceeded the specified request time. When the thread finds such a request, it writes a message to CTSTATUS.FCS, creates a process stack dump (on systems that support this feature), and logs diagnostic information to the file RQSTMON.FCS.
The ctSETCFG() function can be used to change the request time monitor interval value. For example:
ctSETCFG( setcfgREQUEST_TIME_MONITOR, "60" );
sets the request time interval to 60 seconds. If the caller of ctSETCFG() is not a member of the ADMIN group, ctSETCFG() returns error LADM_ERR (589, member of ADMIN group required). Attempting to specify a negative value returns PBAD_ERR (749, bad parameter value). Attempting to enable this feature when REQUEST_TIME_MONITOR -1 is specified in the configuration file returns SETO_ERR (804, cannot override configuration option).
The previously unused field scttrnavl in the system snapshot structure (ctGSMS) has been renamed sctrqtmonint and is set to the request time monitor interval value. This change resulted in the system snapshot structure version change from 7 to 8.
Default: NO
SNAPSHOT_FILENAME
SNAPSHOT_FILENAME <file name>
By default, only the system snapshot is captured with SNAPSHOT_INTERVAL. SNAPSHOT_FILENAME is used to capture user information to SNAPSHOT.FCS as well.
SNAPSHOT_FILENAME <file name>
Files added to the snapshots are said to be activated. Files may be activated whether or not the automatic snapshots are turned on in the configuration file. However, the activation has no effect until snapshots are written to the SYSLOG files.
The <file name> argument may include wildcard matching characters where “*” matches an arbitrary number of any characters and “?” matches exactly one of any character. A pattern of simply “*” matches any user or file name. See FairCom DB Standard Wildcards. For example, the following keywords activate any file ending in “.dat”, and the file journal.idx:
SNAPSHOT_FILENAME *.dat
SNAPSHOT_FILENAME journal.idx
File name case sensitivity depends on the platform. For example, Windows is case insensitive and Unix is case sensitive. The file names activated must match the file name used to first open the file. In particular, paths used in the activation list and during the call to open the file must match.
See Also
SNAPSHOT_INTERVAL
The SNAPSHOT_INTERVAL keyword enables automatic snapshots at specified intervals:
SNAPSHOT_INTERVAL <minutes>
By default, only the system snapshot is captured. To add user- or file-specific snapshots to the data captured, use one or more of the following configuration entries:
SNAPSHOT_USERID <user ID>
SNAPSHOT_FILENAME <file name>
By default, this keyword logs the output to the c-tree Server System Event Log (SYSLOGDT.FCS) discussed at these links:
Use the DIAGNOSTICS SNAPSHOT_AUTOMATIC keyword to change the output to the SNAPSHOT.FCS human readable file.
See Also
SNAPSHOT_LOCKWAIT_USEC
SNAPSHOT_LOCKWAIT_USEC <lock wait histogram interval width in microseconds>
Histograms of waiting times for blocked lock requests are available: one for waiting times for blocked data record lock requests, and one for waiting times for blocked index lock requests (please note that the index locks are not controlled by the user). There is a small amount of overhead associated with mutex calls to collect clean statistics.
SNAPSHOT_LOCKWAIT_USEC can change the default histogram intervals (box width) for the lock waiting time histograms (default of 10,000 µsec or 0.01 seconds).
SNAPSHOT_LOGFLUSHTIME_USEC
SNAPSHOT_LOGFLUSHTIME_USEC <histogram interval width in microseconds>
The SNAPSHOT feature includes two histograms of transaction log flush times to help identify if slow transaction log writes are slowing down server performance. One is of fixed width, and the other has an adaptive width*. These histograms can be viewed in the ctstat utility.
SNAPSHOT_LOGFLUSHTIME_USEC controls the width of the fixed histogram, and the initial width of the adaptive histogram (which defaults to 10 microseconds). This may be changed at server startup by specifying SNAPSHOT_LOGFLUSHTIME_USEC in ctsrvr.cfg. It can also be changed at runtime via ctSETCFG() or the ctadmn utility (select 10. Change Server Settings and then 10. Change the specified configuration option). When the fixed width is changed or the initial adaptive width is automatically adjusted, the histogram box values are reset to zero.
*The first time the server starts, the server sets the adaptive histogram box width to the default value or the SNAPSHOT_LOGFLUSHTIME_USEC value specified in ctsrvr.cfg. As the server runs, on each checkpoint operation, it determines if a significant fraction of the log save values since the last checkpoint are at the low or high end of the histogram range. If so, the server changes the histogram box width to a value that centers the average log save time in the histogram boxes.
SNAPSHOT_TRANTIME_USEC
SNAPSHOT_TRANTIME_USEC <tran time histogram interval width in microseconds>
The SNAPSHOT feature includes a histogram of transaction times. There is a small amount of overhead associated with mutex calls to collect clean statistics.
SNAPSHOT_TRANTIME_USEC can change the default histogram intervals (box width) for the transaction time histogram (default of 50,000 µsec or 0.05 seconds).
SNAPSHOT_USERID
SNAPSHOT_USERID <user ID>
By default, only the system snapshot is captured with SNAPSHOT_INTERVAL. SNAPSHOT_USERID is used to capture user information to SNAPSHOT.FCS as well.
Users added to the snapshots are said to be activated. Users may be activated whether or not the automatic snapshots are turned on in the configuration file. However, the activation has no effect until snapshots are written to the SYSLOG files.
The <user ID> argument may include wildcard matching characters: “*” matches an arbitrary number of any characters, and “?” matches exactly one of any character. A pattern of simply “*” matches any user or file name. See FairCom DB Standard Wildcards. For example, the following keywords activate all users:
SNAPSHOT_USERID *
User IDs are not case sensitive.
See Also
SYSLOG
SYSLOG <option>
FairCom DB optionally maintains a system event log, SYSLOG. This is maintained in two system files: SYSLOGDT.FCS and SYSLOGIX.FCS. These files comprise a FairCom DB data file and index pair with a record for each recordable system event. Unlike the text based CTSTATUS.FCS, SYSLOG can be encrypted such that entries cannot be added, deleted, or modified with a simple text editor, and vendors can log application specific entries.
The System Event Log contents are controlled by SYSLOG configuration keywords in ctsrvr.cfg, the ctsrvr.set settings file, or from the command line. They are entered as pairs in the form of: SYSLOG <keyword>. As many of these pairs as desired may be used at the discretion of your application vendor.
Current SYSLOG options include:
| ADMIN_API | Only allow users in the ADMIN group to use the SystemLog() function to create vendor-defined entries in the log. |
| CTSTATUS | Log each entry to CTSTATUS.FCS in the System Event Log, except for those entries which occur before or after the system logging monitor is in operation. |
| DELETE_FILE | Log file deletes and restores. |
| DISABLE_API | Do not allow any calls to the SystemLog() function for user defined entries. |
| DYNAMIC_DUMP | Log the beginning and end of dynamic dumps and a result for each file dumped. |
| ENCRYPT | Encrypt the SYSLOG files. Requires ADVANCED_ENCRYPTION YES |
| LOGFAIL_PURGE | Causes an automatic purge of the oldest entries in the log if the system cannot add a record to SYSLOGDT.FCS. All the entries occurring on the oldest day are deleted unless there are only entries for the current day in which case no entries are purged. After a successful purge, an attempt is made to add the new entry that triggered the automatic purge. If this add succeeds, the system log operation continues in its usual fashion. |
| LOGFAIL_CTSTATUS | If there is no LOGFAIL_PURGE entry in the configuration file, or if the purge fails, the log entries will be rerouted to CTSTATUS.FCS if LOGFAIL_CTSTATUS is in the configuration file. This disables SYSLOG CTSTATUS; i.e, no more entries are made to the system log. |
| LOGFAIL_TERMINATE |
If there is no automatic purge or it fails, and if there is no re-routing to CTSTATUS.FCS, either the system log will stop operation, or if LOGFAIL_TERMINATE is in the configuration file, the FairCom Server will shut down. Note: USE LOGFAIL_TERMINATE WITH CAUTION! |
| NONE | Used in a settings file to eliminate additional SYSLOG entries in a server configuration file. |
| RESTORE_POINT |
Use RESTORE_POINT to log created restore points, recovery with restore points, and transaction rollback to restore points through utility programs. |
| SQL_STATEMENTS | Enables SQL statement logging to the SYSLOG audit logs.This information includes connection information, improved timing, and logging the statement before it is actually executed for a detailed audit trail of all SQL operations. |
| SYSLOG_EXCLUDE_SQL_USER <name> |
Allows specifying a user <name> to be excluded from this logging. All SQL statements are written to the log by default when SQL query logging is enabled by SYSLOG SQL_STATEMENTS,. Multiple users can be excluded by specifying the keyword multiple times. No validation is made that the <name> specified matches an existing user name. |
| TRUNCATE |
Over time the SYSLOGDT.FCS and SYSLOGIX.FCS files can become quite large requiring file maintenance to reduce the size. The SYSLOG() function supports purging records, optionally filtered by time and by event code. However, purging all entries one record at a time is slow, and storage device space is not released until new records are added over time. A better solution is the TRUNCATE capability available with V13. With SYSLOG TRUNCATE in your configuration file, when a complete purge is requested (no filtering by time or event) the file is truncated rather than deleting individual records. This approach is much faster and avoids limitations with space reuse. Any user reading SYSLOG files when they are truncated will encounter the FBLK_ERR error . After receiving this error, the user must close and reopen the files to proceed.. |
| USER_INFO | Log all logons, logoffs (including SQL users as of V12), and changes to user logon profiles. |
Note: Build 210901 and earlier had a 4GB limit to the SYSLOG files. It is highly recommended to limit information recorded in this file in high volume systems, and include the LOGFAIL_PURGE option to clear data as the file grows. To disable the creation of huge system log files, add SYSLOG NONHUGE to the server configuration file. Builds after 210901 no longer have a 4GB file size limit.
History
- V11.8 and later support SQL_STATEMENTS for detailed statement logging.
- V11.8. and later support auditing SQL logon/logoff events with USER_INFO.
SYSLOG SQL_STATEMENTS Configuration Keyword
This configuration keyword logs executed SQL statements in SYSLOG:
SYSLOG SQL_STATEMENTS
A SYSLOG SQL_STATEMENTS (SYSLOG, SYSLOG) log entry is written after statement execution so it can also include the error code (if any).
The variable part of the SYSLOG entry contains statement information in JSON format similar to SQL_DEBUG LOG_STMT.
Below is a sample showing how it is displayed by the ctalog utility:
Class = 16 (SQL)
Event = 1 (SQL statement)
Date = 09/24/2020
Time = 17:40:11
Sequence number = 37
Error code = -20005
User ID = 'admin'
Node name = 'isql'
Variable-length information:
---------------------------------------------------
{"timestamp":"Tue Sep 24 17:40:27 2020","ipaddr":"127.0.0.1","db":"CTREESQL","user":"admin","thread":29,"statement":"select * from missingtable"}
---------------------------------------------------
SYSVIEW_WHAT
SYSVIEW_WHAT <info>
This option supports the system status reports triggered by specified events such as increasing the number of log files or beginning a dynamic dump. <info> is one of the following to determine what to include in the report.
USER_COUNT
USER_LIST
FILES_COUNT
FILES_LIST
TRANS_COUNT
TRANS_LIST
MEMORY
LOGS
ALL
One or more configuration entries can be specified in ctsrvr.cfg. If the ALL parameter is used, any other entry is redundant. The LIST options (for example., USER_LIST) also report the simple COUNT statistics (for example., USER_COUNT) such that it is redundant to specify both a LIST and COUNT. The only reason to use the COUNT option is to reduce output.
The reports are added to the end of the SYSVIEW.FCS text file. Each report is time stamped and the triggering event is noted.
Example
SYSVIEW_WHEN LOG_ACTIVE_CHG
SYSVIEW_WHEN DYNDMP_BEG
SYSVIEW_WHAT ALL
See Also
Note: This sever feature is not currently enabled.
SYSVIEW_WHEN
SYSVIEW_WHEN <event>
This option supports the system status reports triggered by specified events such as increasing the number of log files or beginning a dynamic dump. <event> is one of the following to determine when the report is generated:
CHECKPOINT_BEG
CHECKPOINT_END
SHUTDOWN
LOG_ACTIVE_CHG,
LOG_SIZE_CHG
DYNDMP_BEG
DYNDMP_END
ALL
The reports are added to the end of the SYSVIEW.FCS text file. Each report is time stamped and the triggering event is noted.
Example
SYSVIEW_WHEN LOG_ACTIVE_CHG
SYSVIEW_WHEN DYNDMP_BEG
SYSVIEW_WHAT ALL
See Also
Note: This sever feature is not currently enabled.
DIAGNOSTICS SNAPSHOT_AUTOMATIC
Write system snapshots to the human-readable SNAPSHOT.FCS text file.
DIAGNOSTICS SNAPSHOT_AUTOMATIC
Writes any automatic snapshots to SNAPSHOT.FCS instead of to the SYSLOG files. However, only the system snapshot is written. Snapshots for activated users and/or files are ignored.
See Also:
DIAGNOSTICS SNAPSHOT_IOTIME
DIAGNOSTICS SNAPSHOT_IOTIME
Enables collecting disk read and write timings on a per-file basis when high-resolution timer support is activated.
DIAGNOSTICS SNAPSHOT_SHUTDOWN
Write system snapshots to the human-readable SNAPSHOT.FCS text file.
DIAGNOSTICS SNAPSHOT_SHUTDOWN
Writes a system snapshot to SNAPSHOT.FCS at the start of the server shutdown process. However, only the system snapshot is written. Snapshots for activated users and/or files are ignored.
FairCom recommends this option in all configuration files to establish baseline statistics over time.
DIAGNOSTICS SNAPSHOT_WORKTIME
DIAGNOSTICS SNAPSHOT_WORKTIME
Enables collecting FairCom DB function call counts and timings on a per-c-tree file basis.