Configure the replication agent

Use ctreplagent.cfg to configure FairCom's replication agent

Use these options to configure your replication agent

Essential information

  • If you have changed your ADMIN passwords, the target_auth.set and source_auth.set files will need to be regenerated.
  • By default, the replicated files will be placed on the target in a directory structure that matches the directories on the source. The redirect option can be used if the path on the target machine does not match the source.
  • The unique_id keyword is required in the multi-server environments. It is a good practice to set this keyword in any environment to set the stage for more advanced topologies.
  • Be sure that log_change_details is commented out. This key word is only used for diagnostic testing because it can increase the size of the log and degrade performance.
  • To see if replication is working, use the repadm utility with the getstats command-line option.
 
 

Example configuration file

The replication agent operations are directed by a single text-based configuration file ctreplagent.cfg. An example is shown below:

; FairCom c-treeACE Server Replication Agent Configuration File - v12.0.0.89 Build(201020)

;The unique ID of the Replication Agent for multi server replications
unique_id REPLAGENT1

; Target server connection info
target_authfile ../config/replication/target_auth.set
target_server FAIRCOMS@localhost

; Source server connection info
source_authfile ../config/replication/source_auth.set
source_server FAIRCOMS@SourceIP

;syncagent yes
; Read 8 KB batches from source server
batch_size 8192

; Use a 5-second timeout when reading from source server
read_timeout_ms 5000

; Log file name
log_file_name  ../config/replication/replagent1.log

; Write change details to log file
;log_change_details changes.log

; The exception_mode option determines how the replication agent handles errors
; that occur when applying operations to the target server.
;
; exception_mode transaction
; When an operation fails, abort the transaction. This option preserves
; atomicity of transactions: either all or none of the operations for the
; transaction are applied to the target c-tree Server.
;
; exception_mode operation
; When an operation fails, continue the transaction, skipping only the
; operation that failed. This option does not preserve atomicity of
; transactions: only the operations that the replication agent successfully
; applies to the target c-tree Server are committed.

exception_mode transaction

; The source server will remember the replication agent's log position

; even when the replication agent disconnects.
;remember_log_pos yes

; file filter specified in the external XML rules
file_filter <../config/replication/filter1.xml

; enables replicating create operations and other schema definition operations
replicate_data_definitions yes

; End of File
 
 

Diagnostic and monitoring options

  • debug_minlog
    Enables the replication agent to log messages to ctreplagent.log when it changes its minimum log number.
  • diagnostics
    Enables diagnostic logging to the replication agent log file.
  • exception_logging
    Configures exception logging.
  • function_timing
    Enables the replication agent to collect function timings.
  • log_change_details
    Logs the change details to a file.
  • log_file_name
    Defines the filename and location of the replication log.
  • log_level
    • Defines what types of messages the replication agent will log. Supported values:
Value Type of messages logged
log_level off none
log_level debug debug, info, warning, and error messages
log_level info info, warning, and error messages
log_level warning warning and error messages
log_level error error messages
  • notify_events
    Triggers the execution of an external program, named replmon, when certain events occur.
  • system_event_log
    Specifies for the replication agent to write its messages to the Windows event log in addition to the replication agent log file.
 
 

Transaction log read options

  • batch_size
    Sets the batch size to read from the source server's transaction logs.
  • convert_subst
    Causes the source server to return file names of replicated files with substituted drives converted to the actual path.
  • full_filenames
    Determines whether the replication agent will return a full or relative file path.
  • read_timeout_ms
    Sets the polling timeout value to use when reading from the source server.
  • remember_log_pos
    Requests the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.
  • start_position
    Specifies the starting position on the source server in the replication agent configuration file.
 
 

Parallel replication options

 
 

Server connection options

  • socket_timeout
    Specifies a timeout option on the replication agent's send and receive calls.
  • source_authfile
    Sets the source server user connection information.
  • source_nodeid
    Forces the required node ID for the source server.
  • source_server
    Sets the source server connection information.
  • source_tls_cert_file
    Specifies the name of the certificate file to use when connecting to the source server using TLS.
  • source_use_tls
    Causes the connection to the source server to use TLS.
  • target_authfile
    Sets the target server user connection information.
  • target_nodeid
    Forces required node ID for the target server.
  • target_server
    Sets the target server connection information.
  • target_tls_cert_file
    Specifies the name of the certificate file to use when connecting to the target server using TLS.
  • target_use_tls
    Causes the connection to the target server to use TLS.
 
 

Replication file set definition options

  • file_filter
    Specifies an XML file filter from an external file.
  • resync_file_list
    Specifies the name of the resync file list that contains the names of all the data files to deploy.
  • unique_id
    Sets the unique ID of the replication agent for multi-server replication.
 
 

Transaction apply options

  • check_update
    Checks record updates for adds, deletes, and rewrites.
  • delete_to_recycle_bin
    Enables the replication agent to use the recycle bin when it deletes a file on the target server.
  • exception_mode
    Specifies how the replication agent handles errors that occur when applying operations to the target server.
  • extension_library
    Registers an external library to be used by the replication agent.
  • lock_retry_count
    Specifies how many times to attempt to lock a record.
  • lock_retry_sleep
    Specify how much time to sleep between record lock attempts.
  • redirect
    Allows files to be redirected to an alternative location on the target server.
  • replicate_data_definitions
    Enables the replication of data definition operations.
  • use_target_alignment
    Specifies for the replication agent to use the alignment setting of the file on the target server.
 
 

Replication agent configuration Options

batch_size

Set the batch size to read from the source server's transaction logs. When using serial (not parallel) replication a larger batch size can improve throughput. Default is 8192. Recommended for large numbers of replicated tables 1M (1048576).

batch_size <batchBufferSizeInBytes>


Default: 8192

 
 

check_dependencies

Enable transaction dependency checking when using parallel replication with more than one apply thread.

check_dependencies <yes|no>

Default: yes

If the replication agent is configured to use parallel replication and check_dependencies yes is specified, the replication agent checks the dependencies between transactions it reads from the source server when it schedules the transactions to be applied by the replication agent apply threads.

 
 

check_update

(V11.5 and later) Configure basic conflict detection for operations that modify data records.

check_update <yes | no | log_only>

Default: yes

When check_update yes is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and fails the operation.

When check_update log_only is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and attempts to apply the operation.

The entry that is logged to the exception log in case of error 1105 includes both the information read from the source server (old and new keys and record images) and the current record image read from the target server.

 
 

convert_subst

For Windows systems only: When reading replicated file names from the source server's transaction logs, convert substituted drive letters to the actual path.

convert_subst <yes | no>

Default: no

 
 

debug_minlog

Enable diagnostic logging of the replication agent's minimum transaction log requirement.

debug_minlog <yes | no>

Default: no

When debug_minlog yes is specified, the replication agent writes a message to ctreplagent.log each time its minimum required transaction log number changes.

 
 

delete_to_recycle_bin

When replication file deletes, make the deletes recoverable.

delete_to_recycle_bin <yes | no>

Default: yes

Enable the Replication Agent to use the recycle bin when it deletes a file on the target server. Otherwise, the file is deleted from disk immediately.

 
 

diagnostics

Enables diagnostic logging to the replication agent log file.

diagnostics <diagnosticsOption>

Default: not set

Supported options:

  • diagnostics analyzer
    enable logging of dependency analysis
  • diagnostics dependency_graph
    enable logging of dependency graph
  • diagnostics file_open
    enable logging of file open/close operations by apply threads
  • diagnostics file_open_err
    enable logging of file open errors
  • diagnostics latency
    enable logging of replication latency
  • diagnostics logread
    enable logging of log read thread
  • diagnostics logship
    enable logging of log ship thread
  • diagnostics tran_state
    enable logging of transaction state processing by log read thread
 
 

exception_logging

(Supported in V11.5 and later) Configure exception logging.

exception_logging <on | off | errors_only | target_server>

Default: errors_only

exception_logging on enables logging of every operation in a transaction where at least one operation failed.

exception_logging off turns off logging of failed operations.

exception_logging errors_only enables logging of only those operations that failed to be applied.

exception_logging target_server causes the replication agent to write exception log messages to a table on the target server instead of the server in which the replication agent is running.

The options exception_logging on, exception _logging off, and exception_logging errors_only are mutually exclusive. If more than one of these options is specified in the replication agent configuration file, the last option that is specified takes effect.

 
 

exception_mode

Configure how the Replication Agent handles errors that occur when applying operations to the target server.

exception_mode <transaction | operation>

Default: transaction

Modes:

  • transaction When an operation fails, abort the transaction. This option preserves atomicity of transactions: either all or none of the operations for the transaction are applied to the target FairCom DB Server.
  • operation When an operation fails, continue the transaction, skipping only the operation that failed. This option does not preserve atomicity of transactions: only the operations that the Replication Agent successfully applies to the target FairCom DB Server are committed.
 
 

extension_library

Register an external library to be used by the Replication Agent.

extension_library <libraryName>

Default: not set

See our Replication Extension documentation on this feature.

Example:

extension_library ctrepluser.dll
 
 

file_filter

Specify an XML file filter for the Replication Agent to use to determine which files it replicates.

file_filter <xmlFilterFileName>

Default: not set

Two forms of usage are supported.

  1. The XML file exists on the system where the source server is running.
    file_filter filter1.xml
  2. The XML file exists on the system where the replication agent is running.
    file_filter <./filter2.xml

See Also

 
 

file_group_parallel_apply

Configure the use of parallel replication within a file group.

file_group_parallel_apply <yes | no>

Default: no

When using replication file groups, file_group_parallel_apply yes enables parallel replication of transactions within a file group. Otherwise, replication of transactions within a file group is performed serially.

 
 

full_filenames

Convert file names that the replication agent reads from the source server's transaction logs to the full path.

full_filenames <yes | no>

Default: no

When the Replication Agent reads a 'file open' change entry from the source server, the entry contains a file number, and the Replication Agent makes a call to the server to get the file name. The source server returns the filename that was read from the transaction log, which is the filename that was specified by the client that opened the file. This name might specify either a full or relative path.

The configuration option full_filenames yes causes the Replication Agent to return the filename with the full path.

 
 

function_timing

(V11.5 and later) Enable the Replication Agent to collect function timings. This can be helpful to analyze replication performance.

function_timing <on | off>

Default: off

 
 

lock_retry_count

Specify how many times to attempt to lock a record. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.

lock_retry_count <count>

Default: 0 (meaning no retries)

See Also

Record Lock Error Retry and Diagnostics

 
 

lock_retry_sleep

Specify how much time to sleep between record lock attempts. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.

lock_retry_sleep <sleep_ms>

Default: 0 (meaning no sleep)

See Also

  • Record lock error retry and diagnostics
 
 

log_change_details

Log change details to the specified text file on the system where the Replication Agent is running.

log_change_details <changeDetailLogFileName>

Default: not set

Note

This configuration should only be used during diagnostic testing due to the extensive performance impact from the numerous log entries.

 
 
 

log_file_name

Defines the filename and location of the replication log.

log_file_name <filename>

By default, replication outputs a log file named ctreplagent.log in its working directory. This path and filename can be configured for locating this file in a central area and to specifically identify which replication environment it is associated with.

 
 

notify_events

Trigger execution of an external program, named replmon, by the Replication Agent when certain events occur.

notify_events <yes | no>

Default: no

In V11 and later, the Replication Agent supports a configuration option, notify_events yes, that triggers execution of an external program. This external execution can be a Windows batch file, Unix shell script, or executable and is expected to be named replmon. When an event in the list of event codes below occurs, replmon is called.

Two string values are passed to replmon upon event notification: an error code and one of the numeric event codes listed below. replmon can then process this exception notification as appropriate for the environment.

  1. Replication Agent is starting
  2. Replication Agent is shutting down
  3. Replication Agent is entering paused state
  4. Replication Agent is resuming operation
  5. Replication Agent connected to source server
  6. Replication Agent connected to target server
  7. Replication Agent lost connection to source server
  8. Replication Agent lost connection to target server
  9. Replication Agent disconnected from source server
  10. Replication Agent disconnected from target server
  11. Replication Agent terminating due to exception

Example

In the example below, replmon.bat logs the date, time, event description, and error code to the file replevent.log:

@echo off

if "%1" == "1" (
 set evdesc=Replication agent is starting. 
) else if "%1" == "2" (
 set evdesc=Replication agent is shutting down. Error code %2. 
) else if "%1" == "3" (
 set evdesc=Replication agent has been paused. 
) else if "%1" == "4" (
 set evdesc=Replication agent is resuming normal operation. 
) else if "%1" == "5" (
 set evdesc=Replication agent connected to source server. 
) else if "%1" == "6" (
 set evdesc=Replication agent connected to target server. 
) else if "%1" == "7" (
 set evdesc=Replication agent lost connection to source server. Error code %2. 
) else if "%1" == "8" (
 set evdesc=Replication agent lost connection to target server. Error code %2. 
) else if "%1" == "9" (
 set evdesc=Replication agent disconnected from source server. 
) else if "%1" == "10" (
 set evdesc=Replication agent disconnected from target server. 
) else if "%1" == "11" (
 set evdesc=Replication agent terminating due to exception. Exception code %2. 
) else (
 set evdesc=Replication agent event code %1. Error code %2. 
) 

for /f "usebackq tokens=*" %%j in (`date /t`) do set dt=%%j 
for /f "usebackq tokens=*" %%j in (`time /t`) do set tm=%%j 
echo %dt%%tm% %evdesc% >> replevent.log 

On Windows, The Replication Agent installs an exception handler so it can notify the external process if an unhandled exception occurs.

 
 

num_analyzer_threads

Set the number of analyzer threads used with parallel replication. Analyzer threads coordinate the internal sequence of operations for correct ordering of transactions to the target server when replication is parallelized.

num_analyzer_threads <n>

Default: 1

  • <n> - number of threads for the analyzer engine to use.

See Also

 
 

num_apply_threads

Set the number of threads to use when applying transactions using parallel replication.

 num_apply_threads <n>

Default: 1

  • <n> - number of threads

See Also

 
 

parallel_apply

Enable parallel replication.

 parallel_apply <yes | no>

Default: no

See Also

 
 

read_timeout_ms

Set the polling timeout value to use when reading the source server's transaction logs.

read_timeout_ms <timeoutInMilliseconds> 

Default: 5000

 
 

redirect

By default, the replication agent uses the file path from the source server's transaction logs when accessing the replicated files on the target server. The redirect option allows file names to be redirected to an alternative location on the target server. A simple string match and substitute method is used to modify the new path location.

redirect </path/on/source> </path/on/target>

Default: not set

 
 

remember_log_pos

Request the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.

For example, logs are required to be retained when the target server or network connection to the target goes offline for a period of time. When the target server is again available, replication will then commence from it's last current position through all the retained logs thereby bringing the system back to a synchronized state.

 remember_log_pos <yes | no>

Default: yes

Note

Replication agent threads store their current minimum required log in the REPLSTATEDT.FCS table on the target server. This file should be removed when the replication state is reset between a pair of source and target servers. For example, after resyncing files, or starting a fresh environment.

 

See Also

 
 

replicate_data_definitions

Replicate data definition operations.

replicate_data_definitions <yes | no>

Default: no

The data definition operations that are replicated when this option is set to 'yes' include:

  • Create a file at ISAM level
  • Change field definitions (DODA)
  • Create an index file
  • Add a resource
  • Update a resource
  • Delete a resource
  • Delete a file at ISAM level
  • Drop an index
  • Rename a file
  • Add an identity field
  • Delete an identity field
  • Reset an identity field
  • Set extended attributes for an identity field
  • Create a data file at low level
  • Delete a data file at low level
  • Change the serial number for a file
  • Update IFIL resource
  • Update header
  • Add change id field
  • Delete change id field
  • Set null key value
 
 

resync_file_list

resync_file_list <filename>

Default: not set

Specifies the name of the resync file list that contains the names of all the data files to deploy. Used when using Replication Agent Manager to deploy file groups to a Replication Agent.

 
 

socket_timeout

A loss of network connectivity can cause the Replication Agent to hang in a socket send or receive call to a source or target server. To allow the Replication Agent to detect and recover from this situation, V11 and later now supports a configuration option to set a timeout on its send and receive calls. The following option can be specified in the Replication Agent configuration file, ctreplagent.cfg:

socket_timeout <timeout_in_seconds>

Default: 5

A setting of zero disables the timeout.

 
 

source_authfile

Set the source server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:

USERID ADMIN

PASSWD ADMIN

source_authfile <sourceAuthFileName>

Default: not set

 
 

source_nodeid

Forces required node ID for the source server. This is provided as an arbitrary dotted notation such as 1.1.1.1.

Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.

source_nodeid <dotted_notation>

Default: not set

See Also

  • target_nodeid
  • Specify Required Node IDs for Source and Targeting
 
 

source_server

Set the source server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.

source_server <serverName>

Default: not set

 
 

source_tls_cert_file

source_tls_cert_file <certificateFileName>

Default: not set

Specifies the name of the certificate file to use when connecting to the source server using TLS. Requires source_use_tls yes to be enabled.

 
 

source_use_tls

source_use_tls <yes | no>

Default: no

source_use_tls yes causes the connection to the source server to use TLS.

 
 

start_position

start_position <startPositionOptions>

Default: not set (see note below for default behavior)

The Replication Agent supports setting its starting position on the source server in the Replication Agent configuration file, ctreplagent.cfg, with the start_position configuration option. The option accepts three possible formats:

start_position #current

start_position <log number> <log position> <minimum log>

start_position <log number> <log position> <last commit log> <last commit position>

Examples


Use the source server's current transaction log position and minimum log:

start_position #current

Start with transaction log 10 at offset 62697158, and set minimum required log to 7:

start_position 10 62697158 7

Start with transaction log 12 at offset 7782963, and set last commit position to log 12 offset 7287745:

start_position 12 7782963 12 7287745

The Replication API function, ctReplAgentSetConfiOption(), can be used to set the position. For example:

ctReplAgentSetConfigOption(ctRAOPT_start_position, "10 62697158 7", &repstt, lineno, configfile);

Default start position behavior


If start_position is not specified in the replication agent configuration file, the Replication Agent's start position is set as follows:

  1. If a file named ctreplagent_<unique_id>.ini exists, the Replication Agent uses the start position that is specified in that .ini file.
  2. Otherwise, if the Replication Agent has previously saved its start position on the target server, the Replication Agent uses that start position.
  3. Otherwise, the Replication Agent uses the source server's current transaction log position as its start position.
 
 

sync_log_writes

Enable immediate flushing of transaction log writes.

sync_log_writes <yes | no>

Default: yes

sync_log_writes yes causes each write to the local transaction log to be flushed to disk.

 
 

system_event_log

system_event_log <machineName>

Default: not set

This option is only supported on Windows systems

When this option is specified, the replication agent writes its messages to the Windows event log in addition to the replication agent log file.

<machineName> is the name of the machine where you want to write event log messages. Specify localhost to write to the local system's event log.

 
 

target_authfile

Set the target server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:

USERID ADMIN

PASSWD ADMIN

target_authfile <targetAuthFileName>

Default: not set

 
 

target_nodeid

Forces a required node ID for the target server. This is provided as an arbitrary dotted notation such as 1.1.1.1.

Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.

 target_nodeid <dotted_notation>

Default: not set

See Also

 
 

target_server

Set the target server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.

target_server <targetServerName>

Default: not set

 
 

target_tls_cert_file

target_tls_cert_file <certificateFileName>

Default: not set

Specifies the name of the certificate file to use when connecting to the target server using TLS. Requires target_use_tls yes to be enabled.

 
 

target_use_tls

target_use_tls <yes | no>

Default: no

target_use_tls yes causes the connection to the target server to use TLS.

 
 

unique_id

unique_id <replagentUniqueId>

Default: REPLAGENT

Set the unique ID of the Replication Agent for multi-server replication. The unique ID is case-insensitive, and is limited to 32 bytes. This ID is also referred to as the agent ID.

 
 

use_target_alignment

use_target_alignment <yes | no>

Default: yes

When use_target_alignment yes is specified, the Replication Agent uses the alignment setting of the file on the target server when applying a record image that it has read from the source server to the target server. Otherwise, the client library's alignment is used.