Working with Records

The FairCom DB API API record management hides from the user all the complexities of maintaining a generic record buffer for tables. While fixed-length records may be easier to implement and maintain, variable-length records present a greater challenge for the developer.

When a table contains variable-length records, each record read may require buffers of different sizes. The FairCom DB API record manager performs the function of expanding and shrinking record buffers to deal with the necessities of variable-length records. The FairCom DB ISAM API also requires different API calls for fixed-length and variable-length records, but the FairCom DB API presents to the user one common API that deals with both fixed- and variable-length records.

Only the application architecture and system resources limit the number of FairCom DB API record buffers associated with a table.

Allocating a record handle

A record handle is required before any record operations can take place. A record handle is instantiated with ctdbAllocRecord() and freed with ctdbFreeRecord().

When allocating a record handle you need to provide a properly-allocated table handle. The table associated with the record handle must be opened before any record operations can take place.

/* allocate a record handle */
CTHANDLE hRecord = ctdbAllocRecord(hTable);
if (!hRecord)
   printf("Record allocation failed\n");

Sharing the same context

Every time a record handle is allocated with ctdbAllocRecord(), the record buffer acquires its own context, which means that each record buffer operates independently. Record operations that move the current record position of one record buffer will not interfere with any others.

There are situations where it may be necessary to have several different record buffers sharing the same record context. In this case the developer can allocate the first record buffer using ctdbAllocRecord() to acquire a new context. Calling ctdbDuplicateRecord() can then duplicate the initial record. The duplicated records will share the same context of the original record buffer, but the duplicated record handle will not share the same memory buffer for holding the record data.

/* Create two records sharing the same context */
CTHANDLE hRecord1 = ctdbAllocRecord(hTable);
CTHANDLE hRecord2 = NULL;
if (hRecord1) {
    hRecord2 = ctdbDuplicateRecord(hRecord1);
    if (!hRecord2)
        printf("Duplicate record failed\n");
}

Record buffer layout

Based on the fields added by the user, FairCom DB API will organize the record buffer as follows:

The user fields are stored in the table record in the same order in which they were added.. To initiate the operation on the records, a record handle must be declared and instantiated. Once the table associated with the record handle is opened, the record handle can be used to store the information that come from or go to the fields in the table.

Resetting a record

A record handle may be reset to its initial state, i.e. to the same state it had just after being allocated, by calling ctdbResetRecord(), which performs the following actions on a record handle:

• The context associated with the record is released
• Record sets are turned off
• Memory allocated for record buffer is released
• Record properties are reinitialized.

/* reset a record */
if (ctdbResetRecord(hRecord) != CTDBRET_OK)
   printf("Reset record failed\n");

All records associated with a table can be reset in one ctdbResetAll() call.

/* reset all records */
if (ctdbResetAll(hTable) != CTDBRET_OK)
   printf("Reset all records failed\n");

When a table is closed, all records associated with that table are automatically reset by ctdbCloseTable().

Releasing a record handle

When a record handle is no longer needed, release it with ctdbFreeRecord(), which releases any context associated with the record, turns off record sets, and releases all resources associated with the record handle. The released record handle is automatically removed from the associated table's list of active record handles.

/* release the record handle */
ctdbFreeRecord(hRecord);

User-managed record buffers

While FairCom DB API performs all record buffer memory management dynamically, the API is flexible enough to permit the user to perform record buffer memory management. ctdbSetRecordBuffer() allows developers to control how FairCom DB API performs data record memory management.

/* define my own static record structure */
typedef struct
{
   NINT     id;
   TEXT     name[32];
   CTDATE   birthday;
} MyRecordDef;

/* declare my own record buffer variable */
MyRecordDef MyRecord;

/* declare and allocate a record handle */
CTHANDLE hRecord = ctdbAllocRecord(hTable);

/* tell record handle to use my own record variable */
if (ctdbSetRecordBuffer(hRecord, &MyRecord, sizeof(MyRecord), CTRECBUF_STATIC) != CTDBRET_OK)
   printf("Set record buffer failed\n");

ctdbSetRecordBuffer() takes as parameters the record handle, a user-supplied record buffer address, the size of the buffer, and the record buffer mode.

The valid record buffer modes are:

To reset the record buffer management back to CTRECBUF_AUTO, after it has been changed to CTRECBUF_STATIC, call ctdbSetRecordBuffer() passing the record handle, NULL buffer, with length of zero and CTRECBUF_AUTO mode.

/* reset the record buffer to automatic */
if (ctdbSetRecordBuffer(hRecord, NULL, 0, CTRECBUF_AUTO) != CTDBRET_OK)
   printf("Reset record buffer failed\n");

The default index

Some record operations require an index to be specified, including record navigation, finding records, and record sets. The default index property is used by the record functions that require an index. Any user defined index, RECBYT, or ROWID may be used as the index controlling the search. Use ctdbSetDefaultIndex() or ctdbSetDefaultIndexByName() to set the default index for a record handle.

/* set the default index to "MyIndex" */
if (ctdbSetDefaultIndexByName(hRecord, "MyIndex") != CTDBRET_OK)
   printf("Set default index failed\n");

Use ctdbGetDefaultIndex() to retrieve the current index or ctdbGetDefaultIndexName() to retrieve the name of the current default index.

/* make sure the default index is set to "MyIndex" */
if (strcmp(ctdbGetDefaultIndexName(hRecord), "MyIndex") != 0)
   if (ctdbSetDefaultIndexByName(hRecord, "MyIndex") != CTDBRET_OK)
      printf("Set default index failed\n");

The default index is set initially to the first user defined index. Once an index is chosen as the default index, all subsequent searches will be done using this index until a new default index is selected. The default index dictates the order in which records are retrieved by using ctdbFirstRecord(), ctdbLastRecord(), ctdbNextRecord(), and ctdbPrevRecord().

Perform a chronological search in the records in a table using ctdbFirstRecord() and ctdbNextRecord(), given the ROWID index as the default index:

ctdbSetDefaultIndex(hRecord, CTDB_ROWID_IDXNO);

Selecting the RECBYT index

The RECBYT index can be selected as the default index in order to perform record navigation with ctdbFirstRecord(), ctdbNextRecord(), ctdbPrevRecord(), and ctdbLastRecord(), ordered by the record offset. Pass the index value of CTDB_RECBYT_IDXNO to ctdbSetDefaultIndex() to default to the RECBYT index.

/* set the RECBYT index as the default index */
if (ctdbSetDefaultIndex(hRecord, CTDB_RECBYT_IDXNO) != CTDBRET_OK)
   printf("Set RECBYT as default index failed\n");

Selecting the ROWID index

The ROWID index can be selected as the default index in order to perform record navigation with ctdbFirstRecord(), ctdbNextRecord(), ctdbPrevRecord(), and ctdbLastRecord(), ordered by the ROWID value of each record. Pass the index value of CTDB_ROWID_IDXNO to ctdbSetDefaultIndex() to default to the ROWID index.

/* set the ROWID index as the default index */
if (ctdbSetDefaultIndex(hRecord, CTDB_ROWID_IDXNO) != CTDBRET_OK)
   printf("Set ROWID as default index failed\n");

Navigating records

FairCom DB API provides features that allow navigation among records of a particular table. You can position the current record at the first record, last record, next record, previous record, or seek to a specific record if you know the record offset.

Setting a different default index for the record may change the order of the record navigation.

The record navigation function not only updates the current position of a record, but the record data is read from disk and placed in the record buffer inside the record handle. As the record is read from disk, the new record flag is set to false to indicate that the data in the record buffer originated from an existing record in the table.

First record

Call ctdbFirstRecord() to position the current record at the first record of a table and retrieve the record data from disk into the specified record handle’s record buffer. If the table is empty (contains no records), ctdbFirstRecord() returns INOT_ERR (101).

/* get the first record of the table */
if (ctdbFirstRecord(hRecord) != CTDBRET_OK)
   printf("First record failed\n");

Last record

Call ctdbLastRecord() to position the current record at the last record of a table and retrieve the record data from disk into the specified record handle’s record buffer. If the table is empty (contains no records), ctdbLastRecord() returns INOT_ERR (101).

/* get the last record of the table */
if (ctdbLastRecord(hRecord) != CTDBRET_OK)
   printf("Last record failed\n");

Next record

ctdbNextRecord() positions the current record at the next record of a table and retrieves the record data from disk into the specified record handle’s record buffer. If the current record was the last record of a table, ctdbNextRecord() returns INOT_ERR (101).

/* get the next record */
if (ctdbNextRecord(hRecord) != CTDBRET_OK)
   printf("Next record failed\n");

A current record must exist before ctdbNextRecord() is called or an ICUR_ERR (100) error is returned indicating that there can be no next record if the current record does not exist. Establish a current record by calling ctdbFirstRecord(), ctdbLastRecord(), ctdbFindRecord(), or ctdbSeekRecord().

Previous record

ctdbPrevRecord() positions the current record at the previous record of a table and retrieves the record data from disk into the specified record handle’s record buffer. If the current record was the first record of a table, ctdbPrevRecord() returns INOT_ERR (101).

/* get the previous record */
if (ctdbPrevRecord(hRecord) != CTDBRET_OK)
   printf("Previous record failed\n");

A current record must exist before ctdbPrevRecord() is called or an ICUR_ERR (100) error is returned indicating that there can be no previous record if the current record does not exist. You can establish a current record by calling ctdbFirstRecord(), ctdbLastRecord(), ctdbFindRecord(), or ctdbSeekRecord().

Seek to record

If you know in advance the offset of a given record, ctdbSeekRecord() can be used to make it the current record.

You can use ctdbGetRecordPos() and ctdbSeekRecord() to implement a bookmark system for your records. Retrieve the record offset with ctdbGetRecordPos(), that is the bookmark value, and later on you can quickly come back to the record by calling ctdbSeekRecord().

/* get record bookmark */
CTOFFSET GetBookMark(CTHANDLE hRecord)
{
CTOFFSET Retval;

ctdbGetRecordPos(hRecord, &Retval);
return Retval;
}

/* goto record bookmark
CTDBRET GotoBookmark(CTHANDLE hRecord, CTOFFSET bookmark)
{
   return ctdbSeekRecord(hRecord, bookmark);
}

Finding records

You can search for records in a table using one of the ctdbFindRecord(), ctdbFindTarget() and ctdbFindRowid() functions. FairCom DB API performs the find operations against the table indexes, when an index entry is found, the record data is loaded from disk into the record handle's record buffer.

Before you can call ctdbFindRecord(), you need to prepare the data you want to find:

1. Clear a record buffer.
2. Set the default index with the appropriate value.
3. Populate the fields that make up the index segment.
4. Call ctdbFindRecord() with the appropriate find mode.

/* find record which product code is DISKETTE */
CTHANDLE hRecord = ctdbAllocRecord(hTable);

/* clear the record buffer */
ctdbClearRecord(hRecord);

/* set the default index to index 0 */
ctdbSetDefaultIndex(hRecord, 0);

/* populate the 'product' field (field 1)*/
ctdbSetFieldAsString(hRecord, 1, "DISKETTE");

/* find the record */
if (ctdbFindRecord(hRecord, CTFIND_EQ) != CTDBRET_OK)
   printf("Record not found\n");

ctdbFindRecord(), ctdbFindTarget() and ctdbFindRowid() return CTDBRET_OK if a record is found or INOT_ERR (101) if no record is found. Any other return value indicates an error condition.

Record Sets

An alternative and more efficient way to retrieve records is the use of Record Sets. A record set contains part of a table, representing a group of records that have keys that match a certain target. For example, an index contains the following keys:

CASE DISKCASE DISKDRIVE DISKETTE KEY KEYBOARD KEYCAP

When scanning through the index looking for records, record navigation functions work on the entire index file. Navigating to the first record obtains the first key in the index, CASE, and moving to the next record obtains each of the following keys in turn, on through KEYCAP.

When creating a record set using ctdbRecordSetOn(), with a target of "DISK" and a target length of 4, FairCom DB API returns only the records in which the first 4 bytes of the key start with "DISK".

The record navigation functions will operate only on the records that match the set until the record set operation is terminated with a call to ctdbRecordSetOff(). To work with multiple record sets at once, use more than one record handle.

Record sets allow a faster search for records, in particular in the client/server mode since fewer records may be retrieved and less traffic on the network will be generated. When worked in conjunction with filters (see below), record sets allow for simple queries.

To use record sets, follow the steps below:

1. Clear the record buffer
2. Set the index you want to use
3. Populate the index segments you would like to participate in the set
4. Establish the Set

Creating a record set

To create a record set, perform the following steps

1. Clear a record handle
2. Select the default index
3. Populate the field, or fields, that form the partial index
4. Call ctdbRecordSetOn() to create a record set

/* start the record set */
ctdbClearRecord(hRecord);
ctdbSetFieldAsString(hRecord, 0, "DISK");
if (ctdbRecordSetOn(hRecord, 4) != CTDBRET_OK)
   printf("Starting record set failed\n");

Once the record set is created, ctdbFirstRecord(), ctdbNextRecord(), ctdbLastRecord() and ctdbPrevRecord() will return only the records that match the record set criteria.

Terminating a record set

Once the record set is no longer necessary, you can turn it off by calling ctdbRecordSetOff().

/* terminate the record set */
if (ctdbRecordSetOff(hRecord) != CTDBRET_OK)
   printf("Terminating record set failed\n");

Once the record set is terminated, ctdbFirstRecord(), ctdbNextRecord(), ctdbLastRecord() and ctdbPrevRecord() will operate again will all records of a table.

Record Filters

FairCom DB API allows users to define record filters using ctdbFilterRecord(). When a filter is set, all records retrieved from a table are filtered against the given expression, and only records matching filter criteria are returned.

/* set filter */
if (ctdbFilterRecord(hRecord, "ZIPCODE == 12345") != CTDBRET_OK)
   printf("Start filter failed\n");

Only the user who sets a filter will have their records filtered. The filter is turned off when the table is closed, or when ctdbFilterRecord() is called with a NULL or an empty string " " as the filter expression. Only one filter may be active per table per user at once, so if a new filter is set to a table with an existing filter for the specific table, the old filter will be overridden.

/* terminate filter */
if (ctdbFilterRecord(hRecord, NULL) != CTDBRET_OK)
   printf("Terminate filter failed\n");

Use ctdbGetFilter() to retrieve the current filter expression. If no filters are active, ctdbGetFilter() returns NULL. Use ctdbIsFilteredRecord() to test if filters are active for a table.

When used in the client/server model, this feature has the potential to increase the performance since just the records matching the criteria will be returned, reducing the network traffic.

Record filters, when used in conjunction with sets (ctdbRecordSetOn()), provide the FairCom DB API API with simple query capabilities.

FairCom DB Expression Parser and Grammar

A powerful expression parser/analyzer provides for complex conditional expressions that can be defined and evaluated at runtime.

Filter expression syntax closely follows the C language syntax for expressions, including order of precedence. An expression interpreted by the expression parser should compile without errors with a standard C compiler. As in C, you cannot compare strings directly like LastName > 'S'. However, the expression parser has a number of built-in functions that allow the comparison of strings. Example:

strcmp( LastName, "S" ) > 0

The expression handling assures proper alignment considerations are handled, and ensures buffer size of any record being evaluated is big enough.

Routines that evaluate conditional expressions maintain fixed data record lengths and total data record lengths. This permits correct alignment adjustments and detects if insufficient data is available. The latter condition results in a CVAL_ERR (598) error. The easiest way to produce a CVAL_ERR (598) is to read only the fixed-length portion of a data record, and have an expression that relies on fields in the variable-length portion of the record.

For additional control, a Conditional Expression Callback Function is available. This allows advanced control through an external user created function.

See also:

• Data Filters
• Conditional Index Support
• Partitioned Files
• cndxparse
• cndxeval
• cndxrun
• cndxfree
• ctparsedoda
• getcndxmem
• putcndxmem

Constants

The expression parser uses the constants below internally as a 32-bit signed or unsigned integer:

• Character constants: any valid char enclosed in single quotes, e.g., 'a'
• Signed Integer constants: values from - 2,147,438,647 to 2,147,438,647
• Unsigned Integer constants: values from 0 to 4,294,967,295
• Hexadecimal constants: values from 0x00000000 to 0xffffffff. Any combination of lower case or upper case letters are accepted, e.g., 0Xbc4f or 0xF5C56d
• Date constants:

{d'yyyy-[m]n-[d]d'}
yyyy is 4 digit year
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
Examples:
{d'2024-12-06'}
{d'2024-8-6'}

'[m]m-[d]d-yyyy' or '[m]m/[d]d/yyyy'
yyyy is 4 digit year
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
Examples:
'12-06-2024'
'8-6-2024'
'8/6/2024'

'yyyy-[m]m-[d]d' or 'yyyy/[m]m/[d]d'
yyyy is 4 digit year
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
Examples:
'2024-12-06'
'2024-8-6'
'2024/8/6'

'[d]d-mmm-yyyy' or '[d]d/mmm/yyyy'
[d]d is 1 or 2 digit day
mmm is 3 letter month name
yyyy is 4 digit year
month names (case insensitive) are:
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
Examples:
'1/jan/2024'
'12-feb-2024'

• Time constants:

{t'[h]h:[m]m:[s]s'}
[h]h is 1 or 2 digit hour
[m]m is 1 or 2 digit minute
[s]s is 1 or 2 digit second
Examples:
{t'23:11:33'}
{t'1:2:3'}

'[h]h:[m]m:[s]s[:[i][i]i]'
[h]h is 1 or 2 digit hour
[m]m is 1 or 2 digit minute
[s]s is 1 or 2 digit second
[i][i]i is optional 1 to 3 digit millisecond
Examples:
'1:2:3'
'12:33:44.123'

• Timestamp constants:

{ts'yyyy-[m]m-[d]d [h]h:M[M]:s[s]'}
yyyy is 4 digit year
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
[h]h is 1 or 2 digit hour
[M]M is 1 or 2 digit minute
[s]s is 1 or 2 digit second
Examples:
{ts'2024-08-12 22:33:44'}
{ts'2024-08-12 2:3:4'}

'[m]m-[d]d-yyyy [h]h:[M]M:[s]s[:[i][i]i]' or '[m]m/[d]d/yyyy [h]h:[M]M:[s]s[:[i][i]i]'
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
yyyy is 4 digit year
[h]h is 1 or 2 digit hour
[M]M is 1 or 2 digit minute
[s]s is 1 or 2 digit second
[i][i]i is optional 1 to 3 digit millisecond
Examples:
'08-12-2024 22:33:44'
'08/12/2024 2:3:4.123'

'yyyy-[m]m-[d]d [h]h:[M]M:[s]s[:[i][i]i]' or 'yyyy/[m]m/[d]d [h]h:[M]M:[s]s[:[i][i]i]'
yyyy is 4 digit year
[m]m is 1 or 2 digit month
[d]d is 1 or 2 digit day
[h]h is 1 or 2 digit hour
[M]M is 1 or 2 digit minute
[s]s is 1 or 2 digit second
[i][i]i is optional 1 to 3 digit millisecond
Examples:
'2024-08-12 22:33:44'
'2024/08/12 2:3:4.123'

'[d]d-mmm-yyyy [h]h:[M]M:[s]s[:[i][i]i]' or '[d]d/mmm/yyyy [h]h:[M]M:[s]s[:[i][i]i]'
[d]d is 1 or 2 digit day
mmm is 3 letter month name
yyyy is 4 digit year
[h]h is 1 or 2 digit hour
[M]M is 1 or 2 digit minute
[s]s is 1 or 2 digit second
[i][i]i is optional 1 to 3 digit millisecond
month names (case insensitive) are:
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
Examples:
'08-dec-2024 22:33:44'
'08/dec/2024 2:3:4.123'

Any integer larger than the maximum size allowed for integers, or any number with decimal points, or any numbers in scientific notation are interpreted as a floating point constant by the expression parser .

String constants are similar to C string constants and represent any text enclosed by double quotes, for example, "This is a string". The maximum size of a string constant defaults to 255 characters. The filter expression parser allows the following escape characters in string constants or character constants:

Variables

A filter expression variable is actually the name of the fields defined for the table. There is a limit of 128 characters for the name of variables and the names are case sensitive.

When a user specifies a variable name, the filter parser searches the table definition for a field of that name. The parser uses the type of the field and converts it to the types used internally by the expression evaluator. The conversion of field types is as follows:

Please note that "int" is a LONG, "unsigned" is a ULONG and "char*" is a pTEXT.

Field names that match a valid expression reserved word:

Consider a field named "year", which collides with the function YEAR. The expression "[year] == 2000" is needed to handle "year" as a field name rather than the function.

Parentheses

Use parentheses exactly like they are used in C expressions. There are no limits on the number of parentheses you may use in an expression, as long as each open parenthesis has a closing parenthesis. Parentheses are also used to enclose the arguments of built-in functions.

Predefined Functions

The FairCom DB conditional expression parser has numerous built-in functions for advanced conditional filtering possibilities. They are described in the Predefined Functions section of the FairCom DB Developer's Guide.

Type Casting

The filter expression parser allows you to use explicit type casts in expressions. This is very useful if you are comparing fields of different types and want to control the result of an expression.

For example, suppose "Salary" is a CT_MONEY field and "Average" is a CT_DFLOAT field; type casts can be used as illustrated in the following expression: (Salary - (int)Average) > 500

The following type casts may be used in conditional expressions:

• (int) or (long): Convert the result of expression to integer (32 bit).
• (unsigned [int | long]): Convert the result of expression to unsigned integer (32 bit).
• (double): Convert the result of expression to double.

You cannot type cast a string expression.

Automatic Type Promotion

When mixing different types in an expression without explicit type casting, the conditional expression parser automatically promotes the types using the following rule:

1. signed and unsigned integers - promoted to unsigned integer (64-bit)
2. signed integer and double - promoted to double
3. unsigned integer and double - promoted to double

In the great majority of cases, mixing strings with numeric values returns a parsing error.

Custom Application Expressions

Use the conditional expression parser/analyzer to evaluate application-specific expressions. The FairCom DB expression parser is a full standalone feature in and of itself, and can be used directly in your own applications for purposes other than data filters, conditional indexes, or partitioned files. Two core API functions provide a powerful interface into this advanced expression handling.

FairCom DB’s expression parser/analyzer requires two key steps:

1. Call cndxparse() to parse your expression, producing an expression tree that the expression analyzer later evaluates.
2. Call cndxeval() to evaluate your expression tree using data from a buffer in memory.

For a complete sample program, see ctexpr.c in the ctree/source directory.

Parsing Expressions

Parsing your expression involves three steps:

1. Define a DODA structure.
2. Parse the DODA into a record schema and field name list.
3. Parse your expression to produce an expression tree.

Sample code to perform these steps is shown below. This code assumes FairCom DB has been initialized prior to calling ctparsedoda().

Expression Parsing Example

#include "ctcndx.h"  /* For PTREE type */
/* Define a DODA structure. */
DATOBJ doda[] = {
  {"CustomerNumber", 0,  CT_INT4U},
  {"ZipCode",        4,  CT_FSTRING,  9},
  {"State",         13,  CT_FSTRING,  2},
  {"LastName",      15,  CT_STRING,  37},
  {"FirstName",     52,  CT_STRING,  37},
  {"Address",       89,  CT_STRING,  49},
  {"City",         138,  CT_STRING,  37}
};

COUNT retval;  /* Return code.        */
pTEXT schema;  /* Record schema.      */
pTEXT names;   /* Field name list.    */
PTREE ptree;   /* Expression tree.    */
pTEXT expr;    /* Expression string.  */
/* Parse the DODA into a record schema and field name list. */
if ((retval = ctparsedoda(doda, 7, &schema, &names)) != 0)
    printf("Error %d parsing DODA.\n", retval);

/* Parse your expression to produce an expression tree. */
expr  = "stricmp(LastName, \"Smith\") == 0
        && CustomerNumber > 10000";
ptree = cndxparse(schema, names, expr, strlen(expr));
if (!ptree)
    printf("Error: Unable to parse expression.\n");
else
    printf("Successfully parsed expression.\n");

To invoke the expression parser to parse a string expression and produce an expression tree, call cndxparse(), which is declared as follows:

PTREE cndxparse(pConvMap Schema, pTEXT Names, pTEXT InputText,
                NINT InputTextSize)

Schema is a pointer to a record schema derived from a DODA definition. Names is a pointer to a list of the field names from the DODA. InputText points to a NULL-terminated string expression, and InputTextSize is the length of InputText.

One of the most useful features of the expression parser is its ability to associate symbolic names in expressions with data in a buffer in memory. To use this ability, you must define a record schema, known as a DODA (Data Object Definition Array). A DODA is an array of field specifications, each of which contains a field name, field offset, field type, and a field length. By providing the expression parser with a DODA, you may include references to DODA field names in your expressions. See the sample code shown later in this section for an example of a DODA definition.

While a DODA is conveniently defined in your application using an array of DATOBJ structures, cndxparse() does not take a DODA in DATOBJ form, but instead accepts a record schema and a list of the field names from the DODA. In order to simplify converting your DODA into the required record schema and field name list, FairCom has written a utility function, ctparsedoda(). This function can be found in the sample file ctexpr.c.

ctparsedoda() is declared as follows:

COUNT ctparsedoda(pDATOBJ doda, UCOUNT numfld, ppTEXT ppschema,
                  ppTEXT ppnames)

Evaluating Expressions

Having produced an expression tree, you are ready to evaluate the expression using the expression analyzer. To evaluate your expression, call cndxeval(), which is declared as follows:

COUNT cndxeval(PTREE Tree, pVOID Recptr, pConvMap Schema)

• Tree is an expression tree returned by cndxparse().
• Recptr points to a buffer containing data you wish to evaluate with your expression.
• Schema is the record schema ctparsedoda() produced from your DODA definition. The record schema is used to associate data in Recptr with field names specified in your expression.

Note: Before you call cndxeval() the first time, you must ensure that a run-time stack has been allocated for the expression analyzer.

To summarize, evaluating your expression involves three steps:

1. Allocate a run-time stack for the expression analyzer (first time only).
2. Set up a buffer containing the field data used in the expression.
3. Evaluate the expression.

If you wish, you can repeat steps 2) and 3) multiple times. Sample code to perform these steps is shown below. It is assumed the Get_Buffer() routine allocates a record buffer and initializes it with data conforming to the field definitions specified in the DODA.

Expression Evaluation Example

COUNT retcidx;  /* Result of expression evaluation.    */
pTEXT recbuf;   /* Record buffer.                      */

/* Allocate a run-time stack for the expression analyzer (first time only). */
if (!ctcidxStk)  {
    ctcidxStk = (pVOID) getcndxmem(CNDX_MAX_STACK * ctSIZE(PLEAF));
    if (!ctcidxStk)  {
        printf("Unable to allocate memory for run-time stack.\n");
        ctrt_exit(1);
    }
}

/* Set up a buffer containing the field data used in the expression. */
Get_Buffer(&recbuf);

/* Evaluate the expression. */
retcidx = cndxeval(ptree, recbuf, (pConvMap)schema);
if (retcidx<0)
    printf("The expression cannot be evaluated for this record
           - error %d.\n", uerr_cod);
else if (retcidx)
    printf("The expression evaluates to TRUE for this record.\n");
else
    printf("The expression evaluates to FALSE for this record.\n");

Remember to always free any memory used by your record schema, field name list, and expression tree when you are finished with them. Use the following FairCom DB functions to do so:

• mbfree(schema)
• mbfree(names)
• cndxfree(ptree)

Conditional Expression Callback Function

FairCom DB provides a developer-defined callback function to perform additional advanced custom data filtering and conditional index evaluation. Instead of calling c-tree’s internal expression evaluator to analyze criteria, a user-defined function is called, allowing advanced filtering and control with application specific code.

Callback Functions

The module ctclbk.c contains the routines declared below. ctfiltercb() is called when a user-defined filter must be evaluated. Whenever a file open retrieves stored conditional index callback expressions, when a conditional index callback expression is created, or when a SetDataFilter() callback expression is created, the ctfiltercb_init() routine is called with a pointer to the callback expression. Whenever a file is closed, or a SetDataFilter() is cleared, the ctfiltercb_uninit() routine is called for each callback expression.

NINT ctfiltercb_init(pTEXT Clbk pinHan)
NINT ctfiltercb(pTEXT Clbk, pVOID Recptr, pConvMap Schema,
                   VRLEN fixlen, VRLEN datlen pinHan)
NINT ctfiltercb_uninit(pTEXT Clbk pinHan)

ctfiltercb_init() is expected to return zero (0) on error and non-zero on initialization, though at this point the error return is ignored. ctfiltercb_init() should set a state variable so the ctfiltercb_uninit() knows whether the init was called for this particular callback filter.

ctfiltercb_uninit() is expected to return zero (0) on error and non-zero on uninitialization, though at this point the error return is ignored. ctfiltercb_uninit() should check a state variable to determine whether or not ctfiltercb_init() was called for this particular callback filter.

Stub functions with simple debug print statements are part of the base distribution. A developer taking advantage of expression callback routines must adapt these callback functions to their particular requirements.

Callback Expression String

As described above, Clbk points to an ASCII string beginning with the "at" sign ‘@’. The string following ‘@’ is completely arbitrary, and is interpreted to determine what type of callback routine is intended, assuming more than one type of callback is required by the application. A simple scheme would use callback strings with a unique character in the second position (following ‘@’). This permits a simple switch statement routing callbacks to desired code, as shown in the following pseudo-code:

/*
** my callback strings are:   @CustomerNumber
**                            @ZeroBalance
**                            @TotalFunds
*/

switch (*(Clbk + 1)) {
    case 'C':
        do the Customer Number check
        break;
    case 'Z':
        do the Zero Balance check
        break;
    case 'T':
        do the Total Funds check
        break;
    default:
        set the return code to -CVAL_ERR to indicate the
         filter could not be evaluated
        break;
}

In this example scheme, the only significant portion of the callback designator is the first character after the @.

Record Batches

There are situations where a record fetch, a record insert, or a record delete operation has to be applied to a group of related records. For example, if an invoice record is being displayed, it may be necessary to retrieve all invoice items for that invoice as well. The following code fragment shows a typical FairCom DB API C function to extract all items of an invoice, given the invoice number.

FairCom DB API C API Example

void GetInvoiceItems(CTHANDLE hRecord, NINT Invoice)
{
    NINT count = 0;
    TEXT target[32];
    VRLEN len = sizeof(target);
 
    /* find the first record that match the Invoice */
    ctdbClearRecord(hRecord);
    ctdbSetFieldAsSigned(hRecord, 0, Invoice);
    if (ctdbFindRecord(hRecord, CTFIND_GE) == CTDBRET_OK)
    {
        LONG val;
       
        /* make sure all record invoice numbers match */
        ctdbGetFieldAsSigned(hRecord, 0, &val);
        while (val == (LONG)Invoice)
        {
            val = -1;
            count++;
            if (ctdbNextRecord(hRecord) == CTDBRET)OK)
                ctdbGetFieldAsSigned(hRecord, 0, &val);
        }
    }
    printf("%d records found\n", count);
}

The code fragment above shows how easy it is to use FairCom DB API functionality to retrieve a number of related records from a table. A close inspection of the code reveals that while it is simple to implement, it does have a hidden issue that may affect the overall performance of the system, specially when running in client/server mode: for every record that is fetched, the client must place a request to the server, which must travel the network, the c-tree Server must interpret the client request, perform the record retrieval operation and return the record to the client, again traveling across the network.

It is obvious the operation described above would be more efficient if we could, in just one call, request the c-tree Server to perform a given operation on a group of related records, and return all the resulting records, or as many records as possible, in one client request.

The ability of perform record retrievals, insert or delete operations on a group of related records is implemented under FairCom DB API as record batch operations. The same code fragment above implemented with FairCom DB API C API batch record facility would look as follows:

FairCom DB API C API Batch Example

void GetInvoiceItems(CTHANDLE hRecord, NINT Invoice)
{
    NINT count = 0;
 
    /* set the partial target key */
    ctdbClearRecord(hRecord);
    ctdbSetFieldAsSigned(hRecord, 0, Invoice);
    /* set the batch operation */
    if (ctdbSetBatch(hRecord, CTBATCH_GET, sizeof(Invoice), 0) == CTDBRET_OK)
    {
        /* retrieve records */
        while (ctdbNextBatch(hRecord) == CTDBRET_OK)
            count++;
        /* terminate batch operations */
        ctdbEndBatch(hRecord);
    }
    printf("%d records found\n", count);
}

The code fragment above is even simpler than the initial example, and it also performs far fewer c-tree Server calls to perform the operations. FairCom DB API batch record functionality is implemented on top of c-tree ISAM batch operations.

Batches with FairCom DB API

The following types of batch operations may be performed:

Retrieve records by partial key

All records with keys matching a partial target key are loaded into an internally maintained buffer region. If there are more records than fit in the buffer, those that fit are loaded, and subsequent calls will retrieve the remaining records.

Retrieve records by index range

Retrieve all records that match an index range expression. All matched records are loaded into an internally maintained buffer region. If there are more records than fit in the buffer, those that fit are loaded, and a subsequent call will retrieve the remaining records.

Retrieve records by physical order

All records of a table are loaded by physical order into an internally maintained buffer region. If the selected records do not fit in the buffer, those that fit are loaded, and subsequent calls will retrieve the remaining records.

Insert a group of records

A group of new records are loaded into an internally maintained buffer region and this group of records are inserted into a table.

Delete a group of records

All records with a key matching a partial target key are deleted.

Batch Modes

You must specify one of the following Mandatory modes when calling the ctdbSetBatch() function or the CTRecord:SetBatch() method:

The following modes are optional and can be OR-ed to the mandatory mode to specify other details on how the batch operation is to be performed.

Starting a new batch operation

The way a new batch operation is started will depend on the type of batch operation you are performing:

• Retrieving records by partial key
• Retrieving records by index range
• Retrieving records by physical order
• Deleting a group of records
• Inserting a group of records

Retrieving records by partial key

All records with key matching a partial target key are loaded into a buffer region maintained internally by FairCom DB API. If the selected records do not fit in the buffer, those that fit are loaded, and subsequent calls will retrieve the remaining records.

The following steps must be taken to perform a batch retrieval operation based on a partial key:

1. Clear a record buffer by calling ctdbClearRecord().
2. Use ctdbSetFieldAs()... to set the fields that form the partial target key that will be used to select a group of records.
3. Call the ctdbSetBatch() function with CTBATCH_GET mode, to start a new record retrieval batch operation.
4. If the ctdbSetBatch() function returns with no errors, continue to call the ctdbNextBatch() function repeatedly until all related records are retrieved. ctdbNextBatch() returns BTMT_ERR (428) to indicate no more records are available.
5. When you are done with the batch records, call the ctdbEndBatch() function to terminate the batch operation. Please note that another batch operation can only start after the current batch operation is terminated.

To start a new batch operation to retrieve records, you must first establish the partial target key that will identify the group of related records. This is accomplished by clearing a record buffer and setting the fields that form the partial target key. For example, if an invoice item table has one index with segments based on invoice number and item number fields, the following partial code will create a partial target key that will select all invoice item records associated with a particular invoice number:

Example

/* set the partial target key */
ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);

After preparing the partial target key, a new batch operation is started by calling the function ctdbSetBatch(). Continuing from the example above, a new batch operation is started by performing the following call:

Example

/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_GET, sizeof(Invoice), 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* retrieve and display all records */
while (ctdbNextBatch(hRecord) == CTDBRET_OK)
    PrintRecord(hRecord);
 
/* terminate batch operations */
ctdbEndBatch(hRecord);

ctdbSetBatch() takes as first argument a valid record handle. The second argument is the batch mode that will direct how the batch operation will be performed. The chapters below describe the available batch modes in detail.

You must provide the length of the ctdbSetBatch()’s target key in bytes. The length of the target key will indicate how many bytes of the target key should be used to create the partial target key. The last parameter is the size of the buffer used internally by FairCom DB API to handle batch operations. A zero value for the last parameter is an indication that the default value size should be used. The default buffer size is calculated as the size of the fixed portion of the record multiplied by 128.

Retrieving records by index range

All records that match an index range expression are loaded into a buffer region maintained internally by FairCom DB API. If the selected records do not fit in the buffer, those that fit are loaded, and subsequent calls will retrieve the remaining records.

The following steps must be taken to perform an index range batch retrieval of records:

1. Establish an index range by calling the ctdbRecordRangeOn() function.
2. Call the ctdbSetBatch() function with the CTBATCH_RANGE mode to start a new record retrieval batch operation.
3. If the ctdbSetBatch() function returns with no errors, continue to call the ctdbNextBatch() function repeatedly until all related records are retrieved. ctdbNextBatch() returns BTMT_ERR (428) to indicate no more records are available.
4. When you are done with the batch records, call the ctdbEndBatch() function to terminate the batch operation.
5. Call the ctdbRecordRangeOff() function to terminate index range operations.

To start a new batch operation to retrieve records, you must first establish the index range operation that will identify the group of related records.

Example:

/* build target key to be used in index range */
TEXT lRange[32];
VRLEN len = (VRLEN)sizeof(lRange);
NINT op = CTIX_EQ;
 
if (ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);
if (ctdbBuildTargetKey(hRecord, CTFIND_EQ, (pVOID)lRange, &len) != CTDBRET_OK)
    printf("ctdbBuildTargetKey failed with error %d\n", ctdbGetError(hRecord));
 
/* set the index range based on the target key */
if (ctdbRecordRangeOn(hRecord, 1, (pVOID)lRange, NULL, CTIX_EQ&op) != CTDBRET_OK)
    printf("ctdbRecordRangeOn failed with error %d\n", ctdbGetError(hRecord));

After setting the index range operation, you may start a new batch operation by calling the ctdbSetBatch() function.

Example:

/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_RANGE, 0, 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* retrieve and display all records */
while (ctdbNextBatch(hRecord) == CTDBRET_OK)
    PrintRecord(hRecord);
 
/* terminate batch operations */
ctdbEndBatch(hRecord);
 
/* terminate the index range */
ctdbRecordRangeOff(hRecord);

Notice when retrieving records by index range ctdbSetBatch()’s targetLen parameter is set to zero. The last parameter is the size of the buffer used internally by FairCom DB API to handle batch operations. A zero value for the last parameter is an indication that the default value size should be used. The default buffer size is calculated as the size of the fixed portion of the record multiplied by 128.

Retrieving records by physical order

All records of a table are loaded by physical order into a buffer region maintained internally by FairCom DB API. If the selected records do not fit in the buffer, those that fit are loaded, and subsequent calls will retrieve the remaining records.

The following steps must be taken to perform a physical order batch retrieval of records:

1. Call the ctdbSetBatch() function with the CTBATCH_PHYS mode to start a new record retrieval batch operation.
2. If the ctdbSetBatch() function returns with no errors, call ctdbNextBatch() repeatedly until all related records are retrieved. ctdbNextBatch() returns BTMT_ERR (428) to indicate no more records are available.
3. When you are done with the batch records, call the ctdbEndBatch() function to terminate the batch operation.

Example:

/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_PHYS, 0, 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* retrieve and display all records */
while (ctdbNextBatch(hRecord) == CTDBRET_OK)
    PrintRecord(hRecord);
 
/* terminate batch operations */
ctdbEndBatch(hRecord);

Notice when retrieving records by index range ctdbSetBatch()’s targetLen parameter is set to zero since no partial key is need for this operation. The last parameter is the size of the buffer used internally by FairCom DB API code to handle batch operations. A zero value for this parameter is an indication that the default value size should be used. The default buffer size is calculated as the size of the fixed portion of the record multiplied by 128.

A physical order batch read is slightly faster than going through an index because it has fewer reads to do.

See Also

• ctdbSetBatch

Deleting a group of records

If the intended batch operation is to delete a group of selected records, you need to initially set the partial target key to select the group of related records and then start the batch operation to delete the selected records.

Even if no records are retrieved with the delete operation, ctdbEndBatch() must be called to terminate the current batch operation.

The following steps must be taken to perform a batch delete record operation:

1. Clear a record buffer by calling the ctdbClearRecord() function.
2. Use the ctdbSetFieldAsxxx() functions to set the fields that form the partial target key that will be used to select a group of records.
3. Call the ctdbSetBatch() function with the CTBATCH_DEL mode to delete a group of related records.
4. Call the ctdbEndBatch() function to terminate the delete record batch operation.

Example:

/* set the partial target key */
ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);
 
/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_DEL, sizeof(Invoice), 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* end the batch operation */
if (ctdbEndBatch(hRecord) != CTDBRET_OK)
    printf("ctdbEndBatch failed with error %d\n", ctdbGetError(hRecord));

You must provide the length of ctdbSetBatch()’s Target key in bytes. The length of the target key indicates how many bytes of the target key should be used to create the partial target key. The last parameter is the size of the buffer used internally by FairCom DB API to handle batch operations. A zero value for the last parameter indicates the default value size should be used. The default buffer size is calculated as the size of the fixed portion of the record multiplied by 128.

Inserting a group of records

A group of new records are loaded into a buffer region maintained internally by FairCom DB API and this group of records are inserted into a table.

When the batch buffer fills up, the group of records stored in the batch buffer are inserted into the table. If ctdbEndBatch() is called and the batch buffer still contains records, a new insert record operation is performed for the remaining records before the batch operation is terminated.

For transaction controlled files, the batch insertion operation is treated as one all or nothing operation. If no explicit transaction is started, each insertion of records with will start and end its own transaction. Even if an explicit transaction is started, each insertion operation is treated independently through safe points.

Currently, all records insertion operations do not perform any conversion of record images, key values and record position for heterogeneous client/server implementations.

The following steps must be taken to perform a batch insert record operation:

1. Call the ctdbSetBatch() function with the CTBATCH_INS mode, to insert a group of records.
2. For each record to be inserted perform the following operations:
   1. Call the ctdbClearRecord() function to clear a record buffer.
   2. For each field in the record call one of the ctdbSetFieldAs...() functions to set the field data.
   3. Call the ctdbInsertBatch() to insert the record into the batch buffer.
3. Call the ctdbEndBatch() function to indicate no more records will be inserted.

Example:

/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_INS, 0, 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* prepare the first record */
ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);  /* invoice number */
ctdbSetFieldAsSigned(hRecord, 1, 0);        /* invoice item number */
ctdbSetFieldAsSigned(hRecord, 2, 100);      /* item quantity */
ctdbSetFieldAsSigned(hRecord, 3, 1001);     /* item code */
if (ctdbInsertBatch(hRecord) != CTDBRET_OK) /* insert record in batch */
    printf("ctdbInsertBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* prepare the second record */
ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);  /* invoice number */
ctdbSetFieldAsSigned(hRecord, 1, 1);        /* invoice item number */
ctdbSetFieldAsSigned(hRecord, 2, 200);      /* item quantity */
ctdbSetFieldAsSigned(hRecord, 3, 1002);     /* item code */
if (ctdbInsertBatch(hRecord) != CTDBRET_OK) /* insert record in batch */
    printf("ctdbInsertBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* terminate the batch operation */
if (ctdbEndBatch(hRecord) != CTDBRET_OK)
    printf("ctdbEndBatch failed with error %d\n", ctdbGetError(hRecord));

Notice when inserting a group of records ctdbSetBatch()’s targetLen parameter is set to zero as no partial key is needed for this operation. The last parameter is the size of the buffer used internally by FairCom DB API to handle batch operations. A zero value for this parameter indicates the default value size should be used. The default buffer size is calculated as the size of the fixed portion of the record multiplied by 128.

 Retrieving records

If the mode of the batch operation is one of CTBATCH_GET, CTBATCH_RANGE or CTBATCH_PHYS then it may be necessary to retrieve all records that match the batch criteria. The records are retrieved by calling the ctdbNextBatch() function.

ctdbNextBatch() retrieves record data from the batch buffer maintained by FairCom DB API’s record handle. After a successful call to ctdbNextBatch() the field data can be retrieved by calling the appropriate ctdbGetFieldAsxxx() functions.

Example:

/* retrieve records */
while (ctdbNextBatch(hRecord) == CTDBRET_OK)
{
    TEXT invoice[32], item[32];
 
    ctdbGetFieldAsString(hRecord, 0, invoice, sizeof(invoice));
    ctdbGetFieldAsString(hRecord, 1, item, sizeof(item));
    printf("%-11s %s\n", invoice, item);
}

Terminating a batch operation

A batch operation must be terminated by calling the ctdbEndBatch() function. Once a batch operation is started, by calling ctdbSetBatch() no other batch operation is allowed to start until the current batch operation is terminated.

Example:

/* set the partial target key */
ctdbClearRecord(hRecord);
ctdbSetFieldAsSigned(hRecord, 0, Invoice);
 
/* set the batch operation */
if (ctdbSetBatch(hRecord, CTBATCH_DEL, sizeof(Invoice), 0) != CTDBRET_OK)
    printf("ctdbSetBatch failed with error %d\n", ctdbGetError(hRecord));
 
/* end the batch operation */
if (ctdbEndBatch(hRecord) != CTDBRET_OK)
    printf("ctdbEndBatch failed with error %d\n", ctdbGetError(hRecord));

When performing batch retrieval operations, you may cancel the batch operation before retrieving all the records by calling ctdbEndBatch().

If the batch operation is a CTBATCH_RANGE then you must also call the ctdbRecordRangeOff() function to terminate the index range used for the batch operation.

Retrieving batch properties

Once a batch operation is started, the following batch properties can be retrieved by calling the appropriate functions or methods.

Field Mask Support Added to FairCom DB API

In V11.6.1 and later, FairCom DB API introduces a new concept used to optimize client/server communication by retrieving only significant record portions - fields - instead of the entire record.

This feature requires a way to specify which of the record's fields are significant and will be accessed. When retrieving a record, FairCom DB API attempts, when possible, to retrieve only the portion that will be accessed. This reduces the number of bytes transmitted. In the case of batches, it increases the number of records retrieved in a single round-trip with a single batch call.

To achieve this objective, we implemented the concept of a "field mask." A field mask defines which are the significant fields. It is activated when the first field is added to the mask. Once active ctdbWriteRecord and ctdbDeleteRecord fail. The field mask cannot be changed when a batch is active. Only the fields specified in the field mask can be accessed using the ctdbGetFieldAs* functions.

New functions:

• A field mask is activated and populated by calling ctdbAddToFieldMask() (ctdbAddToFieldMask, ctdbAddToFieldMask).
• A field mask is deactivated and emptied by calling ctdbRemoveFieldMask() (ctdbRemoveFieldMask, ctdbRemoveFieldMask).
• Check if a field mask is active with ctdbIsFieldMaskOn() (ctdbIsFieldMaskOn, ctdbIsFieldMaskOn).

Batch column filtering for advanced record retrieval

Using the recently introduced FairCom DB API field mask support, it is now possible to filter columns in batch operations.

In ISAM, set the fldmask member of the PKEYREQ2 structure to be used as a "request" for the BATSETX call, using the BAT_EXTENSIONS mode. The fldmask must point to an array of ULONG sorted in ascending order containing the field number (DODA position) composing the partial record. It is also necessary to set the nfields member to indicate how many entries belong to the fldmask array.

Reading and writing field data to a record buffer

The FairCom DB API record manager has the following functions to read field data from the record buffer:

/* display all field data on screen */
void DisplayData(CTHANDLE hRecord)
{
   NINT count = ctdbGetTableFieldCount(hRecord);
   NINT i;
   TEXT str[256];

   for (i = 0; i < count; i++)
   {
      ctdbGetFieldAsString(hRecord, i, str, sizeof(str));
      printf("Field %d: %s\n", i, str);
   }
}

The following functions should be used to write fields into the data record buffer:

/* add new record */
void AddRecord(CTHANDLE hRecord, pTEXT name, pTEXT address, pTEXT phone)
{
   ctdbClearRecord(hRecord);
   ctdbSetFieldAsString(hRecord, 0, name);
   ctdbSetFieldAsString(hRecord, 1, address);
   ctdbSetFieldAsString(hRecord, 2, phone);
   if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   {
      printf("Add record failed\n");
   }
}

The ctdbSetFieldAs() functions will also clear the null bit flag for the updated field.

When you invoke one of the ctdbGetFieldAs() or ctdbSetFieldAs() functions, you pass the record handle, the field number and the data you want to read from or write to the data record buffer. The ctdbGetFieldAs() or ctdbSetFieldAs() function names specify the type of the data you are trying to read or write not the underlying field type.

If the type of the field you are trying to read from, or write to, is different than the type of the data specified by the ctdbGetFieldAs() or ctdbSetFieldAs() functions, the record manager will automatically convert the field data type to match the data type of the parameter of passed by one of the ctdbGetFieldAs() or ctdbSetFieldAs() function.

For example, if you are writing a report generator application that displays the fields of a record on screen, you can read all fields in the record buffer with ctdbGetFieldAsString() and the record manager will convert the different field types to string. Boolean field type is converted as "True" or "False", numeric values are converted to string and dates and times use the session wide default date and time formats.

Automatic data type conversion

Some automatic data type conversions may not be possible, and an error (CTDBRET_CANTCONVERT or CTDBRET_INVTYPE) will occur, for example, if ctdbSetFieldAsDate() is used to store data in a field whose type is CT_BOOL.

The table below presents the valid automatic conversions performed by ctdbGetFieldAs() or ctdbSetFieldAs() functions for the different field types:

*1 - These are the FairCom DB API field types. The old c-tree Plus field types are also valid and may be used. For new projects, the best approach is to use the new field type convention presented in this table, since this is the naming convention for the FairCom DB SQL field types.

*2 - The value being set must be equal to either “TRUE” or “FALSE” (case insensitive), otherwise the function will fail with CTDBRET_CANTCONVERT (4042).

For every row in the table above, there is a highlighted column. This column represents the natural function that should be used to update that particular field. As discussed, though, any other function (column) marked with an “X” can be used, and the appropriate conversions will be performed.

Null field support

The FairCom DB API record manager API implements the underlying support for null fields, but does not enforce it. If a table was created with null field support, the record manager will set or clear the null bit flag for fields, but there is no check to see if null fields are allowed or not.

When a record buffer is cleared by calling ctdbClearRecord(), all fields are marked as null fields. As you update the field contents by calling ctdbSetFieldAs() functions, the null field for the respective field is cleared indicating that the field contains data.

A specific field can be cleared, and its null field flag is set, by calling ctdbClearField(). Call ctdbIsNullField() to check if a field null flag is set or not.

/* if a field 0 is not null, clear the field */
if (ctdbIsNullField(hRecord, 0)
   ctdbClearField(hRecord, 0);

The FairCom DB API null flag controls the NOT NULL property of a column. Setting this to column has no effect on individual record null values: it is NOT enforced at the FairCom DB API layer. This attribute only applies to the FairCom DB SQL layer for constraint on values. It is useful to set this flag in c-tree data files before SQL import such that the property is maintained.

Defined field size

Call ctdbGetFieldSize() or ctdbGetFieldLength() to retrieve the defined field size. The defined field size is the size of the field declared at the time the table was created.

Actual field length

Call ctdbGetFieldDataLength() to retrieve the actual field size (the number of bytes actually being consumed by the data in that field, in one record (row) of the table). The actual size of variable-length fields, such as CT_VARCHAR and CT_VARBINARY, may vary from the defined size.

/* check if field 0 actual size is different from defined size */
if (ctdbGetFieldSize(hRecord, 0) != ctdbGetFieldDataLength(hRecord, 0))
   printf("Field 0 defined and actual sizes are different\n");

Field address in record buffer

The field address in a record buffer will also vary if the field is located in the variable portion of a record. Call ctdbGetFieldAddress() to retrieve the field address in the record buffer. Fields in the fixed portion of a record buffer will always return the same address.

ctdbGetFieldAddress() takes as parameters the record handle, the field number and return a pointer to the address of the field in the record buffer. ctdbGetFieldAddress() returns NULL if an error occurs and the field address cannot be retrieved.

Field offset in record buffer

A field offset address in the record buffer can also be retrieved by calling ctdbGetFieldOffset(). The offset of fields in the fixed portion of the record never changes, while the offset of fields in the variable portion of a record may change to accommodate the different sizes of variable-length fields.

Check if field is fixed- or variable-length

Call ctdbIsVariableField() to check if a field is variable-length. ctdbIsVariableField() returns YES if the field is variable-length.

/* check if field 0 is variable length */
if (ctdbIsVariableField(hRecord, 0))
   printf("Field 0 is variable length\n");
else
   printf("Field 0 is fixed length\n");

Using field names

Most record functions that operate on fields, take a field number as a parameter to indicate which field to operate on. ctdbGetFieldNumberByName() takes the field name and return its number.

If the field name passed to ctdbGetFieldNumberByName() is invalid, the function returns -1 (negative one).

/* update USERID field */
NINT fieldnbr = ctdbGetFieldNumberByName(hRecord, "USERID");
if (fieldnbr >= 0)
   ctdbSetFieldAsString(hRecord, fieldnbr, "123456");
else
   printf("Invalid field name\n");

Operators

The following operators are allowed in filters and conditional expressions.

Mathematical Operators

Relational Operators

Logical Operators

Binary Operators

NULL Operators

Clearing the record buffer

Call ctdbClearRecord() to clear the record handle. The record buffer is cleared, the full bit flag for all fields are cleared, the new record flag is set to false, the edited record flag is cleared, and the record offset is set to zero.

/* clear the record buffer */
if (ctdbClearRecord(hRecord) != CTDBRET_OK)
   printf("Clear record failed\n");

Clearing a field

A field can be cleared, and its null field flag set, by calling ctdbClearField(). Call ctdbIsNullField() to check if a field null flag is set.

/* if a field 0 is not null, clear the field */
if (ctdbIsNullField(hRecord, 0)
   ctdbClearField(hRecord, 0);

Adding new records

To add a new record to a table, you need to perform the following actions:

1. Clear the record buffer by calling ctdbClearRecord().
2. Populate the fields by calling the ctdbSetFieldAs() functions.
3. Add the record by calling ctdbWriteRecord():

/* add new record */
ctdbClearRecord(hRecord);

/* populate the record buffer */
ctdbSetFieldAsSigned(hRecord, 0, 1000);
ctdbSetFieldAsString(hRecord, 1, "Joe Smith");

/* write the record */
if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   printf("Add record failed\n");

A record is added to a table if the new record flag and edited flag are both set. ctdbClearRecord() sets the new record flag, while one of the ctdbSetFieldAs() functions will set the edited record flag.

If you clear a record without populating the record buffer with data, ctdbWriteRecord() returns no error code. However, no data record is written to disk, since the new record flag is set but the edited record flag is cleared.

If you wish to write a cleared or blank record to disk, you must clear the record buffer, set the edited record flag by calling ctdbSetEditedRecord() and then write the record by calling ctdbWriteRecord().

/* add a blank record to disk */
ctdbClearRecord(hRecord);

/* set the edited record flag */
ctdbSetEditedRecord(hRecord, YES);

/* write the record */
if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   printf("Add record failed\n");

Updating existing records

To update an existing record, you need to perform the following actions:

1. Read the record from disk by calling one of the ctdbFirstRecord(), ctdbLastRecord(), ctdbNextRecord(), ctdbPrevRecord(), ctdbSeekRecord(), ctdbFindRecord(), or ctdbFindTarget() functions.
2. Update one or more fields with ctdbSetFieldAs() functions
3. Write the record by calling ctdbWriteRecord().

/* update the first record */
ctdbFirstRecord(hRecord);

/* change the first field */
ctdbSetFieldAsString(hRecord, 0, "Joe Smith");

/* write the record */
if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   printf("Update record failed\n");

FairCom DB API utilizes two flags to track the edited state of a record: new record and edited. When a record is newly created, the new record flag is set and the edited flag is cleared. The edited flag is set whenever data is moved into the record with any ctdbSetField() operations. When FairCom DB API prepares to write the record, it examines these flags. If this record has been edited, it checks the new record flag to either update an existing record, or add a new record. When you read an existing record with ctdbFirstRecord() or ctdbNextRecord(), etc, the new record flag is cleared and the edited flag is cleared. Again, the edited flag is only set if data is moved into the record with ctdbSetField() operations. If you request FairCom DB API to write the record and the edited flag is not set, no data is written. To force a record update to disk you should call ctdbSetEditedRecord() to set the edited flag directly.

/* update the first record */
ctdbFirstRecord(hRecord);

/* set the edited flag */
ctdbSetEditedRecord(hRecord, YES);

/* write the record */
if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   printf("Update record failed\n");

Duplicate a record by: reading it from disk, setting the edited record flag with ctdbSetEditedRecord(), setting the new record flag with ctdbSetNewRecord(), and calling ctdbWriteRecord() to write it to disk. Please note that a record can only be duplicated if the table has no indexes with unique keys.

/* duplicate the first record */
ctdbFirstRecord(hRecord);

/* set the edited flag */
ctdbSetEditedRecord(hRecord, YES);

/* set the new record flag */
ctdbSetNewRecord(hRecord, YES);

/* write the record */
if (ctdbWriteRecord(hRecord) != CTDBRET_OK)
   printf("Duplicate record failed\n");

Deleting records

To delete a record, you need to perform the following actions:

1. Read the record from disk and set up a record handle that points at that record by calling one of the ctdbFirstRecord(), ctdbLastRecord(), ctdbNextRecord(), ctdbPrevRecord(), ctdbSeekRecord(), ctdbFindRecord(), or ctdbFindTarget() functions.
2. Lock the record before reading it using the session-wide locking mechanism (ctdbLock()), or manually lock the record after reading it by calling ctdbLockRecord().
3. Delete the record by calling ctdbDeleteRecord().

/* read and then lock the first record */
ctdbFirstRecord(hRecord);
ctdbLockRecord (hRecord, CTLOCK_WRITE);

/* delete the record */
if (ctdbDeleteRecord(hRecord) != CTDBRET_OK)
   printf("Delete record failed\n");

Note that using this manual locking scheme is not the safest approach, because a manual record lock cannot be applied until after the record is read from disk. This means there is a window of opportunity for the record to be modified or deleted by another thread before the record is manually locked by this thread. Fortunately, the FairCom DB API API provides a safer locking mechanism than manually locking individual records: session-wide record locking. This locking mechanism automatically locks records as they are read from disk, thereby removing the window of opportunity, resulting in improved data security. Session-wide record locking is discussed further in Data Integrity.

Record properties

New record flag

The new record flag indicates if a record has been cleared and not written to disk yet. The record manager uses this flag to decide if a record should be added or updated.

The new record flag is set when the record handle is allocated or when the record is cleared by ctdbClearRecord(). The new record flag is cleared when a record is read from disk by calling ctdbFirstRecord(), ctdbLastRecord(), ctdbNextRecord(), ctdbPrevRecord(), ctdbSeekRecord(), ctdbFindRecord(), or ctdbFindTarget() functions.

Edited record flag

A record is only written to disk if the edited record flag is set. This flag is set when the record buffer is modified with a call to one of the ctdbSetFieldAs() functions or by calling ctdbSetEditedRecord().

Record offset

This property holds the current record offset of a record. If a record is cleared, the record offset property is zero. All records in a table, even the first record, will have a record offset value greater than zero.

You can retrieve the record offset value by calling ctdbGetRecordPos(). You can set the record offset property, and load the record data at the offset, by calling ctdbSeekRecord().

Record count

Use ctdbGetRecordCount() to retrieve the total number of records in a table. This is a read only property.

/* check if table is empty */
CTUINT64 count;
ctdbGetRecordCount(hRecord, &count);
if (count > 0) then
   printf("Table is not empty\n");

Record ROWID

Use ctdbGetRowid() to retrieve the ROWID value for the current record. This is a read only property.

/* retrieve the first record rowid */
CTROWID rowid;
ctdbFirstRecord(hRecord);
if (ctdbGetRowid(hRecord, &rowid) != CTDBRET_OK)
   printf("Table has no ROWID index\n");

Record Locking

A manual record lock can be acquired by performing the following actions:

1. Read a record from disk and set up a record handle that points at that record by calling ctdbFirstRecord(), ctdbLastRecord(), ctdbNextRecord(), ctdbPrevRecord(), ctdbSeekRecord(), ctdbFindRecord(), or ctdbFindTarget() functions.
2. Lock the record by calling ctdbLockRecord()

Note that using this manual record locking method is not the safest approach, because a record lock can not be applied until after the record is read from disk. This means there is a window of opportunity for the record to be modified or deleted by another thread before the record is locked. Fortunately, the FairCom DB API API provides a safer locking mechanism than manual locking: session-wide locking. This locking mechanism automatically locks records as they are read from disk, thereby removing the afore-mentioned window of opportunity, resulting in improved data security. Session-wide record locking is discussed further in Data Integrity.

Release a record lock by calling ctdbUnlockRecord() or by calling ctdbLockRecord() with CTLOCK_FREE mode. Note that, in certain situations, calling these functions might not immediately release the lock. See ctdbUnlockRecord() for more details.

Check if a record is locked

Call ctdbGetRecordLock() to retrieve the current lock mode acquired for the current record. If the record is not locked, ctdbGetRecordLock() returns CTLOCK_FREE. If the current record is locked, ctdbGetRecordLock() returns the record lock mode.

/* check if record is locked */
if ctdbGetRecordLock(hRecord) != CTLOCK_FREE)
   printf("The record is locked\n");

The following record modes are returned by ctdbGetRecordLock():

Demoting record locks

If a record has acquired a write lock (also called an "exclusive" lock), it is possible to change or demote the write lock to a read lock (also called a "shared lock"), if the record has not been updated. Use ctdbLockRecord() to demote a record from a write lock to a read lock.

/* demote a write lock to a read lock */
if (ctdbGetRecordLock(hRecord) == CTLOCK_WRITE)
   if (ctdbLockRecord(hRecord, CTLOCK_READ) != CTDBRET_OK)
      printf("Record lock demote failed\n");