Configuration reference

audit_log

The audit_log section defines configuration parameters related to audit logging.

• audit_log.extract_key
• audit_log.file
• audit_log.filter
• audit_log.format
• audit_log.nonblock
• audit_log.pipe
• audit_log.spaces
• audit_log.to
• audit_log.syslog.*
  • audit_log.syslog.facility
  • audit_log.syslog.identity
  • audit_log.syslog.server

audit_log.extract_key¶

Since: 3.0.0.

If set to true, the audit subsystem extracts and prints only the primary key instead of full tuples in DML events (space_insert, space_replace, space_delete). Otherwise, full tuples are logged. The option may be useful in case tuples are big.

Type: boolean

Default: false

Environment variable: TT_AUDIT_LOG_EXTRACT_KEY

audit_log.file¶

Specify a file for the audit log destination. You can set the file type using the audit_log.to option. If you write logs to a file, Tarantool reopens the audit log at SIGHUP.

Type: string

Default: „var/log/{{ instance_name }}/audit.log“

Environment variable: TT_AUDIT_LOG_FILE

audit_log.filter¶

Enable logging for a specified subset of audit events. This option accepts the following values:

• Event names (for example, password_change). For details, see Audit log events.
• Event groups (for example, audit). For details, see Event groups.

The option contains either one value from Possible values section (see below) or a combination of them.

To enable custom audit log events, specify the custom value in this option.

Example

filter: [ user_create,data_operations,ddl,custom ]

Type: array

Possible values: „all“, „audit“, „auth“, „priv“, „ddl“, „dml“, „data_operations“, „compatibility“, „audit_enable“, „auth_ok“, „auth_fail“, „disconnect“, „user_create“, „user_drop“, „role_create“, „role_drop“, „user_disable“, „user_enable“, „user_grant_rights“, „role_grant_rights“, „role_revoke_rights“, „password_change“, „access_denied“, „eval“, „call“, „space_select“, „space_create“, „space_alter“, „space_drop“, „space_insert“, „space_replace“, „space_delete“, „custom“

Default: „nil“

Environment variable: TT_AUDIT_LOG_FILTER

audit_log.format¶

Specify a format that is used for the audit log.

Example

If you set the option to plain,

audit_log:
  to: file
  format: plain

the output in the file might look as follows:

2024-01-17T00:12:27.155+0300
4b5a2624-28e5-4b08-83c7-035a0c5a1db9
INFO remote:unix/:(socket)
session_type:console
module:tarantool
user:admin
type:space_create
tag:
description:Create space Bands

Type: string

Possible values: „json“, „csv“, „plain“

Default: „json“

Environment variable: TT_AUDIT_LOG_FORMAT

audit_log.nonblock¶

Specify the logging behavior if the system is not ready to write. If set to true, Tarantool does not block during logging if the system is non-writable and writes a message instead. Using this value may improve logging performance at the cost of losing some log messages.

Примечание

The option only has an effect if the audit_log.to is set to syslog or pipe.

Type: boolean

Default: false

Environment variable: TT_AUDIT_LOG_NONBLOCK

audit_log.pipe¶

Specify a pipe for the audit log destination. You can set the pipe type using the audit_log.to option. If log is a program, its pid is stored in the audit.pid field. You need to send it a signal to rotate logs.

Example

This starts the cronolog program when the server starts and sends all audit_log messages to cronolog standard input (stdin).

audit_log:
  to: pipe
  pipe: 'cronolog audit_tarantool.log'

Type: string

Default: box.NULL

Environment variable: TT_AUDIT_LOG_PIPE

audit_log.spaces¶

Since: 3.0.0.

The array of space names for which data operation events (space_select, space_insert, space_replace, space_delete) should be logged. The array accepts string values. If set to box.NULL, the data operation events are logged for all spaces.

Example

In the example, only the events of bands and singers spaces are logged:

audit_log:
  spaces: [bands, singers]

Type: array

Default: box.NULL

Environment variable: TT_AUDIT_LOG_SPACES

audit_log.to¶

Enable audit logging and define the log location. This option accepts the following values:

• devnull: disable audit logging.
• file: write audit logs to a file (see audit_log.file).
• pipe: start a program and write audit logs to it (see audit_log.pipe).
• syslog: write audit logs to a system logger (see audit_log.syslog.*).

By default, audit logging is disabled.

Example

The basic audit log configuration might look as follows:

audit_log:
  to: file
  file: 'audit_tarantool.log'
  filter: [ user_create,data_operations,ddl,custom ]
  format: json
  spaces: [ bands ]
  extract_key: true

Type: string

Possible values: „devnull“, „file“, „pipe“, „syslog“

Default: „devnull“

Environment variable: TT_AUDIT_LOG_TO

config

The config section defines various parameters related to centralized configuration.

• config.reload
• config.context.*
• config.etcd.*
• config.storage.*

config.reload¶

Since: 3.0.0.

Specify how the configuration is reloaded. This option accepts the following values:

• auto: configuration is reloaded automatically when it is changed.
• manual: configuration should be reloaded manually. In this case, you can reload the configuration in the application code using config:reload().

See also: Reloading configuration.

Type: string

Possible values: „auto“, „manual“

Default: „auto“

Environment variable: TT_CONFIG_RELOAD

credentials

The credentials section allows you to create users and grant them the specified privileges. Learn more in Credentials.

• credentials.roles.*
• credentials.users.*
• <user_or_role_name>.privileges.*

database

The database section defines database-specific configuration parameters, such as an instance’s read-write mode or transaction isolation level.

• database.hot_standby
• database.instance_uuid
• database.mode
• database.replicaset_uuid
• database.txn_isolation
• database.txn_timeout
• database.use_mvcc_engine

database.hot_standby¶

Type: boolean

Default: false

Environment variable: TT_DATABASE_HOT_STANDBY

database.instance_uuid¶

An instance UUID.

By default, instance UUIDs are generated automatically. database.instance_uuid can be used to specify an instance identifier manually.

UUIDs should follow these rules:

• The values must be true unique identifiers, not shared by other instances or replica sets within the common infrastructure.
• The values must be used consistently, not changed after the initial setup. The initial values are stored in snapshot files and are checked whenever the system is restarted.
• The values must comply with RFC 4122. The nil UUID is not allowed.

See also: database.replicaset_uuid

Type: string

Default: box.NULL

Environment variable: TT_DATABASE_INSTANCE_UUID

database.mode¶

An instance’s operating mode. This option is in effect if replication.failover is set to off.

The following modes are available:

• rw: an instance is in read-write mode.
• ro: an instance is in read-only mode.

If not specified explicitly, the default value depends on the number of instances in a replica set. For a single instance, the rw mode is used, while for multiple instances, the ro mode is used.

Example

You can set the database.mode option to rw on all instances in a replica set to make a master-master configuration. In this case, replication.failover should be set to off.

credentials:
  users:
    replicator:
      password: 'topsecret'
      roles: [replication]

iproto:
  advertise:
    peer:
      login: replicator

replication:
  failover: off

groups:
  group001:
    replicasets:
      replicaset001:
        instances:
          instance001:
            database:
              mode: rw
            iproto:
              listen:
              - uri: '127.0.0.1:3301'
          instance002:
            database:
              mode: rw
            iproto:
              listen:
              - uri: '127.0.0.1:3302'

# Load sample data
app:
  file: 'myapp.lua'

Type: string

Default: box.NULL (the actual default value depends on the number of instances in a replica set)

Environment variable: TT_DATABASE_MODE

database.replicaset_uuid¶

A replica set UUID.

By default, replica set UUIDs are generated automatically. database.replicaset_uuid can be used to specify a replica set identifier manually.

See also: database.instance_uuid

Type: string

Default: box.NULL

Environment variable: TT_DATABASE_REPLICASET_UUID

database.txn_isolation¶

A transaction isolation level.

Type: string

Default: best-effort

Possible values: best-effort, read-committed, read-confirmed

Environment variable: TT_DATABASE_TXN_ISOLATION

database.txn_timeout¶

A timeout (in seconds) after which the transaction is rolled back.

See also: box.begin()

Type: number

Default: 3153600000 (~100 years)

Environment variable: TT_DATABASE_TXN_TIMEOUT

database.use_mvcc_engine¶

Whether the transactional manager is enabled.

Type: boolean

Default: false

Environment variable: TT_DATABASE_USE_MVCC_ENGINE

flightrec

The flightrec section describes options related to the flight recorder configuration.

• flightrec.enabled
• flightrec.logs_size
• flightrec.logs_max_msg_size
• flightrec.logs_log_level
• flightrec.metrics_period
• flightrec.metrics_interval
• flightrec.requests_size
• flightrec.requests_max_req_size
• flightrec.requests_max_res_size

flightrec.enabled¶

Enable the flight recorder.

Type: boolean

Default: false

Environment variable: TT_FLIGHTREC_ENABLED

flightrec.logs_size¶

Specify the size (in bytes) of the log storage. You can set this option to 0 to disable the log storage.

Type: integer

Default: 10485760

Environment variable: TT_FLIGHTREC_LOGS_SIZE

flightrec.logs_max_msg_size¶

Specify the maximum size (in bytes) of the log message. The log message is truncated if its size exceeds this limit.

Type: integer

Default: 4096

Maximum: 16384

Environment variable: TT_FLIGHTREC_LOGS_MAX_MSG_SIZE

flightrec.logs_log_level¶

Specify the level of detail the log has. The default value is 6 (VERBOSE). You can learn more about log levels from the log_level option description. Note that the flightrec.logs_log_level value might differ from log_level.

Type: integer

Default: 6

Environment variable: TT_FLIGHTREC_LOGS_LOG_LEVEL

flightrec.metrics_period¶

Specify the time period (in seconds) that defines how long metrics are stored from the moment of dump. So, this value defines how much historical metrics data is collected up to the moment of crash. The frequency of metric dumps is defined by flightrec.metrics_interval.

Type: integer

Default: 180

Environment variable: TT_FLIGHTREC_METRICS_PERIOD

flightrec.metrics_interval¶

Specify the time interval (in seconds) that defines the frequency of dumping metrics. This value shouldn’t exceed flightrec.metrics_period.

Type: number

Default: 1.0

Minimum: 0.001

Environment variable: TT_FLIGHTREC_METRICS_INTERVAL

Примечание

Given that the average size of a metrics entry is 2 kB, you can estimate the size of the metrics storage as follows:

(flightrec_metrics_period / flightrec_metrics_interval) * 2 kB

flightrec.requests_size¶

Specify the size (in bytes) of storage for the request and response data. You can set this parameter to 0 to disable a storage of requests and responses.

Type: integer

Default: 10485760

Environment variable: TT_FLIGHTREC_REQUESTS_SIZE

flightrec.requests_max_req_size¶

Specify the maximum size (in bytes) of a request entry. A request entry is truncated if this size is exceeded.

Type: integer

Default: 16384

Environment variable: TT_FLIGHTREC_REQUESTS_MAX_REQ_SIZE

flightrec.requests_max_res_size¶

Specify the maximum size (in bytes) of a response entry. A response entry is truncated if this size is exceeded.

Type: integer

Default: 16384

Environment variable: TT_FLIGHTREC_REQUESTS_MAX_RES_SIZE

iproto

The iproto section is used to configure parameters related to communicating to and between cluster instances.

• iproto.advertise.*
  • iproto.advertise.client
  • iproto.advertise.peer
  • iproto.advertise.sharding
  • iproto.advertise.<peer_or_sharding>.*
• iproto.*
  • iproto.listen
  • iproto.net_msg_max
  • iproto.readahead
  • iproto.threads
• <uri>.params.*

groups

The groups section provides the ability to define the full topology of a Tarantool cluster.

• groups.<group_name>
• groups.<group_name>.replicasets
• groups.<group_name>.<config_parameter>

groups.<group_name>¶

A group name.

groups.<group_name>.replicasets¶

Replica sets that belong to this group. See replicasets.

groups.<group_name>.<config_parameter>¶

Any configuration parameter that can be defined in the group scope. For example, iproto and database configuration parameters defined at the group level are applied to all instances in this group.

log

The log section defines configuration parameters related to logging. To handle logging in your application, use the log module.

• log.to
• log.file
• log.format
• log.level
• log.modules
• log.nonblock
• log.pipe
• log.syslog.*
  • log.syslog.facility
  • log.syslog.identity
  • log.syslog.server

log.to¶

Define a location Tarantool sends logs to. This option accepts the following values:

• stderr: write logs to the standard error stream.
• file: write logs to a file (see log.file).
• pipe: start a program and write logs to its standard input (see log.pipe).
• syslog: write logs to a system logger (see log.syslog.*).

Type: string

Default: „stderr“

Environment variable: TT_LOG_TO

log.file¶

Specify a file for logs destination. To write logs to a file, you need to set log.to to file. Otherwise, log.file is ignored.

Example

The example below shows how to write logs to a file placed in the specified directory:

log:
  to: file
  file: var/log/{{ instance_name }}/instance.log

Example on GitHub: log_file.

Type: string

Default: „var/log/{{ instance_name }}/tarantool.log“

Environment variable: TT_LOG_FILE

log.format¶

Specify a format that is used for a log entry. The following formats are supported:

• plain: a log entry is formatted as plain text. Example:

  2024-04-09 11:00:10.369 [12089] main/104/interactive I> log level 5 (INFO)
  

• json: a log entry is formatted as JSON and includes additional fields. Example:

  {
    "time": "2024-04-09T11:00:57.174+0300",
    "level": "INFO",
    "message": "log level 5 (INFO)",
    "pid": 12160,
    "cord_name": "main",
    "fiber_id": 104,
    "fiber_name": "interactive",
    "file": "src/main.cc",
    "line": 498
  }
  

Type: string

Default: „plain“

Environment variable: TT_LOG_FORMAT

log.level¶

Specify the level of detail logs have. There are the following levels:

• 0 – fatal
• 1 – syserror
• 2 – error
• 3 – crit
• 4 – warn
• 5 – info
• 6 – verbose
• 7 – debug

By setting log.level, you can enable logging of all events with severities above or equal to the given level.

Example

The example below shows how to log all events with severities above or equal to the VERBOSE level.

log:
  level: 'verbose'

Example on GitHub: log_level.

Type: number, string

Default: 5

Environment variable: TT_LOG_LEVEL

log.modules¶

Configure the specified log levels (log.level) for different modules.

You can specify a logging level for the following module types:

• Modules (files) that use the default logger. Example: Set log levels for files that use the default logger.
• Modules that use custom loggers created using the log.new() function. Example: Set log levels for modules that use custom loggers.
• The tarantool module that enables you to configure the logging level for Tarantool core messages. Specifically, it configures the logging level for messages logged from non-Lua code, including C modules. Example: Set a log level for C modules.

Example 1: Set log levels for files that use the default logger

Suppose you have two identical modules placed by the following paths: test/module1.lua and test/module2.lua. These modules use the default logger and look as follows:

return {
    say_hello = function()
        local log = require('log')
        log.info('Info message from module1')
    end
}

To configure logging levels, you need to provide module names corresponding to paths to these modules:

log:
  modules:
    test.module1: 'verbose'
    test.module2: 'error'
app:
  file: 'app.lua'

To load these modules in your application (app.lua), you need to add the corresponding require directives:

module1 = require('test.module1')
module2 = require('test.module2')

Given that module1 has the verbose logging level and module2 has the error level, calling module1.say_hello() shows a message but module2.say_hello() is swallowed:

-- Prints 'info' messages --
module1.say_hello()
--[[
[92617] main/103/interactive/test.logging.module1 I> Info message from module1
---
...
--]]

-- Swallows 'info' messages --
module2.say_hello()
--[[
---
...
--]]

Example on GitHub: log_existing_modules.

Example 2: Set log levels for modules that use custom loggers

This example shows how to set the verbose level for module1 and the error level for module2:

log:
  modules:
    module1: 'verbose'
    module2: 'error'
app:
  file: 'app.lua'

To create custom loggers in your application (app.lua), call the log.new() function:

-- Creates new loggers --
module1_log = require('log').new('module1')
module2_log = require('log').new('module2')

Given that module1 has the verbose logging level and module2 has the error level, calling module1_log.info() shows a message but module2_log.info() is swallowed:

-- Prints 'info' messages --
module1_log.info('Info message from module1')
--[[
[16300] main/103/interactive/module1 I> Info message from module1
---
...
--]]

-- Swallows 'debug' messages --
module1_log.debug('Debug message from module1')
--[[
---
...
--]]

-- Swallows 'info' messages --
module2_log.info('Info message from module2')
--[[
---
...
--]]

Example on GitHub: log_new_modules.

Example 3: Set a log level for C modules

This example shows how to set the info level for the tarantool module:

log:
  modules:
    tarantool: 'info'
app:
  file: 'app.lua'

The specified level affects messages logged from C modules:

ffi = require('ffi')

-- Prints 'info' messages --
ffi.C._say(ffi.C.S_INFO, nil, 0, nil, 'Info message from C module')
--[[
[6024] main/103/interactive I> Info message from C module
---
...
--]]

-- Swallows 'debug' messages --
ffi.C._say(ffi.C.S_DEBUG, nil, 0, nil, 'Debug message from C module')
--[[
---
...
--]]

The example above uses the LuaJIT ffi library to call C functions provided by the say module.

Example on GitHub: log_existing_c_modules.

Type: map

Default: box.NULL

Environment variable: TT_LOG_MODULES

log.nonblock¶

Specify the logging behavior if the system is not ready to write. If set to true, Tarantool does not block during logging if the system is non-writable and writes a message instead. Using this value may improve logging performance at the cost of losing some log messages.

Примечание

The option only has an effect if the log.to is set to syslog or pipe.

Type: boolean

Default: false

Environment variable: TT_LOG_NONBLOCK

log.pipe¶

Start a program and write logs to its standard input (stdin). To send logs to a program’s standard input, you need to set log.to to pipe.

Example

In the example below, Tarantool writes logs to the standard input of cronolog:

log:
  to: pipe
  pipe: 'cronolog tarantool.log'

Example on GitHub: log_pipe.

Type: string

Default: box.NULL

Environment variable: TT_LOG_PIPE

memtx

The memtx section is used to configure parameters related to the memtx engine.

• memtx.allocator
• memtx.max_tuple_size
• memtx.memory
• memtx.min_tuple_size
• memtx.slab_alloc_factor
• memtx.slab_alloc_granularity
• memtx.sort_threads

memtx.allocator¶

Specify the allocator that manages memory for memtx tuples. Possible values:

• system – the memory is allocated as needed, checking that the quota is not exceeded. THe allocator is based on the malloc function.
• small – a slab allocator. The allocator repeatedly uses a memory block to allocate objects of the same type. Note that this allocator is prone to unresolvable fragmentation on specific workloads, so you can switch to system in such cases.

Type: string

Default: „small“

Environment variable: TT_MEMTX_ALLOCATOR

memtx.max_tuple_size¶

Size of the largest allocation unit for the memtx storage engine in bytes. It can be increased if it is necessary to store large tuples.

Type: integer

Default: 1048576

Environment variable: TT_MEMTX_MAX_TUPLE_SIZE

memtx.memory¶

The amount of memory in bytes that Tarantool allocates to store tuples. When the limit is reached, INSERT and UPDATE requests fail with the ER_MEMORY_ISSUE error. The server does not go beyond the memtx.memory limit to allocate tuples, but there is additional memory used to store indexes and connection information.

Example

In the example below, the memory size is set to 1 GB (1073741824 bytes).

memtx:
  memory: 1073741824

Type: integer

Default: 268435456

Environment variable: TT_MEMTX_MEMORY

memtx.min_tuple_size¶

Size of the smallest allocation unit in bytes. It can be decreased if most of the tuples are very small.

Type: integer

Default: 16

Possible values: between 8 and 1048280 inclusive

Environment variable: TT_MEMTX_MIN_TUPLE_SIZE

memtx.slab_alloc_factor¶

The multiplier for computing the sizes of memory chunks that tuples are stored in. A lower value may result in less wasted memory depending on the total amount of memory available and the distribution of item sizes.

See also: memtx.slab_alloc_granularity

Type: number

Default: 1.05

Possible values: between 1 and 2 inclusive

Environment variable: TT_MEMTX_SLAB_ALLOC_FACTOR

memtx.slab_alloc_granularity¶

Specify the granularity in bytes of memory allocation in the small allocator. The memtx.slab_alloc_granularity value should meet the following conditions:

• The value is a power of two.
• The value is greater than or equal to 4.

Below are few recommendations on how to adjust the memtx.slab_alloc_granularity option:

• If the tuples in space are small and have about the same size, set the option to 4 bytes to save memory.
• If the tuples are different-sized, increase the option value to allocate tuples from the same mempool (memory pool).

See also: memtx.slab_alloc_factor

Type: integer

Default: 8

Environment variable: TT_MEMTX_SLAB_ALLOC_GRANULARITY

memtx.sort_threads¶

Since: 3.0.0.

The number of threads from the thread pool used to sort keys of secondary indexes on loading a memtx database. The minimum value is 1, the maximum value is 256. The default is to use all available cores.

Примечание

Since 3.0.0, this option replaces the approach when OpenMP threads are used to parallelize sorting. For backward compatibility, the OMP_NUM_THREADS environment variable is taken into account to set the number of sorting threads.

Type: integer

Default: box.NULL

Environment variable: TT_MEMTX_SORT_THREADS

process

The process defines configuration parameters of the Tarantool process in the system.

• process.background
• process.coredump
• process.title
• process.pid_file
• process.strip_core
• process.username
• process.work_dir

process.background¶

Run the server as a background task.

Важно

Do not enable the background mode for applications intended to run by the tt utility. For more information, see the tt start reference.

Type: boolean

Default: false

Environment variable: TT_PROCESS_BACKGROUND

process.coredump¶

Type: boolean

Default: false

Environment variable: TT_PROCESS_COREDUMP

process.title¶

Sets the name of the server’s process title (what’s shown in the COMMAND column for ps -ef and top -c commands).

Type: string

Default: „tarantool - {{ instance_name }}“

Environment variable: TT_PROCESS_TITLE

process.pid_file¶

Store the process id in this file.

Type: string

Default: „var/run/{{ instance_name }}/tarantool.pid“

Environment variable: TT_PROCESS_PID_FILE

process.strip_core¶

Whether coredump files should include memory allocated for tuples. (This can be large if Tarantool runs under heavy load.) Setting to true means «do not include».

Type: boolean

Default: true

Environment variable: TT_PROCESS_STRIP_CORE

process.username¶

The name of the system user to switch to after start.

Type: string

Default: box.NULL

Environment variable: TT_PROCESS_USERNAME

process.work_dir¶

A directory where Tarantool working files are stored.

Type: string

Default: box.NULL

Environment variable: TT_PROCESS_WORK_DIR

replication

The replication section defines configuration parameters related to replication.

• replication.anon
• replication.bootstrap_strategy
• replication.connect_timeout
• replication.election_mode
• replication.election_timeout
• replication.election_fencing_mode
• replication.failover
• replication.peers
• replication.skip_conflict
• replication.sync_lag
• replication.sync_timeout
• replication.synchro_quorum
• replication.synchro_timeout
• replication.threads
• replication.timeout

replication.anon¶

Type: boolean

Default: false

Environment variable: TT_REPLICATION_ANON

replication.bootstrap_strategy¶

Specifies a strategy used to bootstrap a replica set. The following strategies are available:

• auto: a node doesn’t boot if half or more of the other nodes in a replica set are not connected. For example, if a replica set contains 2 or 3 nodes, a node requires 2 connected instances. In the case of 4 or 5 nodes, at least 3 connected instances are required. Moreover, a bootstrap leader fails to boot unless every connected node has chosen it as a bootstrap leader.
• config: use the specified node to bootstrap a replica set. To specify the bootstrap leader, use the <replicaset_name>.bootstrap_leader option.
• supervised: a bootstrap leader isn’t chosen automatically but should be appointed using box.ctl.make_bootstrap_leader() on the desired node.
• legacy (deprecated since 2.11.0): a node requires the replication_connect_quorum number of other nodes to be connected. This option is added to keep the compatibility with the current versions of Cartridge and might be removed in the future.

Type: string

Default: auto

Environment variable: TT_REPLICATION_BOOTSTRAP_STRATEGY

replication.connect_timeout¶

A timeout (in seconds) a replica waits when trying to connect to a master in a cluster. See orphan status for details.

This parameter is different from replication.timeout, which a master uses to disconnect a replica when the master receives no acknowledgments of heartbeat messages.

Type: number

Default: 30

Environment variable: TT_REPLICATION_CONNECT_TIMEOUT

replication.election_mode¶

A role of a replica set node in the leader election process.

The possible values are:

• off: a node doesn’t participate in the election activities.
• voter: a node can participate in the election process but can’t be a leader.
• candidate: a node should be able to become a leader.
• manual: allow to control which instance is the leader explicitly instead of relying on automated leader election. By default, the instance acts like a voter – it is read-only and may vote for other candidate instances. Once box.ctl.promote() is called, the instance becomes a candidate and starts a new election round. If the instance wins the elections, it becomes a leader but won’t participate in any new elections.

Type: string

Default: box.NULL (the actual default value depends on replication.failover)

Environment variable: TT_REPLICATION_ELECTION_MODE

replication.election_timeout¶

Specifies the timeout (in seconds) between election rounds in the leader election process if the previous round ended up with a split vote.

It is quite big, and for most of the cases, it can be lowered to 300-400 ms.

To avoid the split vote repeat, the timeout is randomized on each node during every new election, from 100% to 110% of the original timeout value. For example, if the timeout is 300 ms and there are 3 nodes started the election simultaneously in the same term, they can set their election timeouts to 300, 310, and 320 respectively, or to 305, 302, and 324, and so on. In that way, the votes will never be split because the election on different nodes won’t be restarted simultaneously.

Type: number

Default: 5

Environment variable: TT_REPLICATION_ELECTION_TIMEOUT

replication.election_fencing_mode¶

Specifies the leader fencing mode that affects the leader election process. When the parameter is set to soft or strict, the leader resigns its leadership if it has less than replication.synchro_quorum of alive connections to the cluster nodes. The resigning leader receives the status of a follower in the current election term and becomes read-only.

• In soft mode, a connection is considered dead if there are no responses for 4 * replication.timeout seconds both on the current leader and the followers.
• In strict mode, a connection is considered dead if there are no responses for 2 * replication.timeout seconds on the current leader and 4 * replication.timeout seconds on the followers. This improves the chances that there is only one leader at any time.

Fencing applies to the instances that have the replication.election_mode set to candidate or manual. To turn off leader fencing, set election_fencing_mode to off.

Type: string

Default: soft

Possible values: off, soft, strict

Environment variable: TT_REPLICATION_ELECTION_FENCING_MODE

replication.failover¶

A failover mode used to take over a master role when the current master instance fails. The following modes are available:

• off

  Leadership in a replica set is controlled using the database.mode option. In this case, you can set the database.mode option to rw on all instances in a replica set to make a master-master configuration.

  The default database.mode is determined as follows: rw if there is one instance in a replica set; ro if there are several instances.

• manual

  Leadership in a replica set is controlled using the <replicaset_name>.leader option. In this case, a master-master configuration is forbidden.

  In the manual mode, the database.mode option cannot be set explicitly. The leader is configured in the read-write mode, all the other instances are read-only.

• election

  Automated leader election is used to control leadership in a replica set.

  In the election mode, database.mode and <replicaset_name>.leader shouldn’t be set explicitly.

• supervised (Enterprise Edition only)

  Leadership in a replica set is controlled using an external failover agent.

  In the supervised mode, database.mode and <replicaset_name>.leader shouldn’t be set explicitly.

See also: Replication tutorials.

Примечание

replication.failover can be defined in the global, group, and replica set scope.

Example

In the example below, the following configuration options are specified:

• In the credentials section, the replicator user with the replication role is created.
• iproto.advertise.peer specifies that other instances should connect to an address defined in iproto.listen using the replicator user.
• replication.failover specifies that a master instance should be set manually.
• <replicaset_name>.leader sets instance001 as a replica set leader.

credentials:
  users:
    replicator:
      password: 'topsecret'
      roles: [replication]

iproto:
  advertise:
    peer:
      login: replicator

replication:
  failover: manual

groups:
  group001:
    replicasets:
      replicaset001:
        leader: instance001
        instances:
          instance001:
            iproto:
              listen:
              - uri: '127.0.0.1:3301'
          instance002:
            iproto:
              listen:
              - uri: '127.0.0.1:3302'
          instance003:
            iproto:
              listen:
              - uri: '127.0.0.1:3303'

Type: string

Default: off

Environment variable: TT_REPLICATION_FAILOVER

replication.peers¶

URIs of instances that constitute a replica set. These URIs are used by an instance to connect to another instance as a replica.

Alternatively, you can use iproto.advertise.peer to specify a URI used to advertise the current instance to other cluster members.

Example

In the example below, the following configuration options are specified:

• In the credentials section, the replicator user with the replication role is created.
• replication.peers specifies URIs of replica set instances.

credentials:
  users:
    replicator:
      password: 'topsecret'
      roles: [replication]

replication:
  peers:
  - replicator:topsecret@127.0.0.1:3301
  - replicator:topsecret@127.0.0.1:3302
  - replicator:topsecret@127.0.0.1:3303

Type: array

Default: box.NULL

Environment variable: TT_REPLICATION_PEERS

replication.skip_conflict¶

By default, if a replica adds a unique key that another replica has added, replication stops with the ER_TUPLE_FOUND error. If replication.skip_conflict is set to true, such errors are ignored.

Примечание

Instead of saving the broken transaction to the write-ahead log, it is written as NOP (No operation).

Type: boolean

Default: false

Environment variable: TT_REPLICATION_SKIP_CONFLICT

replication.sync_lag¶

The maximum delay (in seconds) between the time when data is written to the master and the time when it is written to a replica. If replication.sync_lag is set to nil or 365 * 100 * 86400 (TIMEOUT_INFINITY), a replica is always considered to be «synced».

Примечание

This parameter is ignored during bootstrap. See orphan status for details.

Type: number

Default: 10

Environment variable: TT_REPLICATION_SYNC_LAG

replication.sync_timeout¶

The timeout (in seconds) that a node waits when trying to sync with other nodes in a replica set after connecting or during a configuration update. This could fail indefinitely if replication.sync_lag is smaller than network latency, or if the replica cannot keep pace with master updates. If replication.sync_timeout expires, the replica enters orphan status.

Type: number

Default: 0

Environment variable: TT_REPLICATION_SYNC_TIMEOUT

replication.synchro_quorum¶

A number of replicas that should confirm the receipt of a synchronous transaction before it can finish its commit.

This option supports dynamic evaluation of the quorum number. For example, the default value is N / 2 + 1 where N is the current number of replicas registered in a cluster. Once any replicas are added or removed, the expression is re-evaluated automatically.

Note that the default value (at least 50% of the cluster size + 1) guarantees data reliability. Using a value less than the canonical one might lead to unexpected results, including a split-brain.

replication.synchro_quorum is not used on replicas. If the master fails, the pending synchronous transactions will be kept waiting on the replicas until a new master is elected.

Примечание

replication.synchro_quorum does not account for anonymous replicas.

Type: string, number

Default: N / 2 + 1

Environment variable: TT_REPLICATION_SYNCHRO_QUORUM

replication.synchro_timeout¶

For synchronous replication only. Specify how many seconds to wait for a synchronous transaction quorum replication until it is declared failed and is rolled back.

It is not used on replicas, so if the master fails, the pending synchronous transactions will be kept waiting on the replicas until a new master is elected.

Type: number

Default: 5

Environment variable: TT_REPLICATION_SYNCHRO_TIMEOUT

replication.threads¶

The number of threads spawned to decode the incoming replication data.

In most cases, one thread is enough for all incoming data. Possible values range from 1 to 1000. If there are multiple replication threads, connections to serve are distributed evenly between the threads.

Type: integer

Default: 1

Environment variable: TT_REPLICATION_THREADS

replication.timeout¶

A time interval (in seconds) used by a master to send heartbeat requests to a replica when there are no updates to send to this replica. For each request, a replica should return a heartbeat acknowledgment.

If a master or replica gets no heartbeat message for 4 * replication.timeout seconds, a connection is dropped and a replica tries to reconnect to the master.

See also: Monitoring a replica set.

Type: number

Default: 1

Environment variable: TT_REPLICATION_TIMEOUT

roles

This section describes configuration parameters related to roles.

• roles
• roles_cfg

roles¶

Since: 3.0.0.

Type: array

Default: nil

Environment variable: TT_ROLES

roles_cfg¶

Since: 3.0.0.

Type: map

Default: nil

Environment variable: TT_ROLES_CFG

security

The security section defines configuration parameters related to various security settings.

• security.auth_delay
• security.auth_retries
• security.auth_type
• security.disable_guest
• security.password_enforce_digits
• security.password_enforce_lowercase
• security.password_enforce_specialchars
• security.password_enforce_uppercase
• security.password_history_length
• security.password_lifetime_days
• security.password_min_length
• security.secure_erasing

security.auth_delay¶

Specify a period of time (in seconds) that a specific user should wait for the next attempt after failed authentication.

The security.auth_retries option lets a client try to authenticate the specified number of times before security.auth_delay is enforced.

In the configuration below, Tarantool lets a client try to authenticate with the same username three times. At the fourth attempt, the authentication delay configured with security.auth_delay is enforced. This means that a client should wait 10 seconds after the first failed attempt.

security:
  auth_delay: 10
  auth_retries: 2

Type: number

Default: 0

Environment variable: TT_SECURITY_AUTH_DELAY

security.auth_retries¶

Specify the maximum number of authentication retries allowed before security.auth_delay is enforced. The default value is 0, which means security.auth_delay is enforced after the first failed authentication attempt.

The retry counter is reset after security.auth_delay seconds since the first failed attempt. For example, if a client tries to authenticate fewer than security.auth_retries times within security.auth_delay seconds, no authentication delay is enforced. The retry counter is also reset after any successful authentication attempt.

Type: integer

Default: 0

Environment variable: TT_SECURITY_AUTH_RETRIES

security.auth_type¶

Specify a protocol used to authenticate users. The possible values are:

• chap-sha1: use the CHAP protocol with SHA-1 hashing applied to passwords.
• pap-sha256: use PAP authentication with the SHA256 hashing algorithm.

Note that CHAP stores password hashes in the _user space unsalted. If an attacker gains access to the database, they may crack a password, for example, using a rainbow table. For PAP, a password is salted with a user-unique salt before saving it in the database, which keeps the database protected from cracking using a rainbow table.

To enable PAP, specify the security.auth_type option as follows:

security:
  auth_type: 'pap-sha256'

Type: string

Default: „chap-sha1“

Environment variable: TT_SECURITY_AUTH_TYPE

security.disable_guest¶

If true, turn off access over remote connections from unauthenticated or guest users. This option affects connections between cluster members and net.box connections.

Type: boolean

Default: false

Environment variable: TT_SECURITY_DISABLE_GUEST

security.password_enforce_digits¶

If true, a password should contain digits (0-9).

Type: boolean

Default: false

Environment variable: TT_SECURITY_PASSWORD_ENFORCE_DIGITS

security.password_enforce_lowercase¶

If true, a password should contain lowercase letters (a-z).

Type: boolean

Default: false

Environment variable: TT_SECURITY_PASSWORD_ENFORCE_LOWERCASE

security.password_enforce_specialchars¶

If true, a password should contain at least one special character (such as &|?!@$).

Type: boolean

Default: false

Environment variable: TT_SECURITY_PASSWORD_ENFORCE_SPECIALCHARS

security.password_enforce_uppercase¶

If true, a password should contain uppercase letters (A-Z).

Type: boolean

Default: false

Environment variable: TT_SECURITY_PASSWORD_ENFORCE_UPPERCASE

security.password_history_length¶

Specify the number of unique new user passwords before an old password can be reused.

Примечание

Tarantool uses the auth_history field in the box.space._user system space to store user passwords.

Type: integer

Default: 0

Environment variable: TT_SECURITY_PASSWORD_HISTORY_LENGTH

security.password_lifetime_days¶

Specify the maximum period of time (in days) a user can use the same password. When this period ends, a user gets the «Password expired» error on a login attempt. To restore access for such users, use box.schema.user.passwd.

Примечание

The default 0 value means that a password never expires.

Type: integer

Default: 0

Environment variable: TT_SECURITY_PASSWORD_LIFETIME_DAYS

security.password_min_length¶

Specify the minimum number of characters for a password.

Type: integer

Default: 0

Environment variable: TT_SECURITY_PASSWORD_MIN_LENGTH

security.secure_erasing¶

If true, forces Tarantool to overwrite a data file a few times before deletion to render recovery of a deleted file impossible. The option applies to both .xlog and .snap files as well as Vinyl data files.

Type: boolean

Default: false

Environment variable: TT_SECURITY_SECURE_ERASING

sharding

The sharding section defines configuration parameters related to sharding.

• sharding.bucket_count
• sharding.discovery_mode
• sharding.failover_ping_timeout
• sharding.lock
• sharding.rebalancer_disbalance_threshold
• sharding.rebalancer_max_receiving
• sharding.rebalancer_max_sending
• sharding.roles
• sharding.sched_move_quota
• sharding.sched_ref_quota
• sharding.shard_index
• sharding.sync_timeout
• sharding.zone

sharding.bucket_count¶

The total number of buckets in a cluster.

sharding.bucket_count should be several orders of magnitude larger than the potential number of cluster nodes, considering potential scaling out in the future.

If the estimated number of nodes in a cluster is M, then the data set should be divided into 100M or even 1000M buckets, depending on the planned scaling out. This number is greater than the potential number of cluster nodes in the system being designed.

Keep in mind that too many buckets can cause a need to allocate more memory to store routing information. On the other hand, an insufficient number of buckets can lead to decreased granularity when rebalancing.

Примечание

This option should be defined at the global level.

Example:

sharding:
  bucket_count: 1000

Type: integer

Default: 3000

Environment variable: TT_SHARDING_BUCKET_COUNT

sharding.discovery_mode¶

A mode of the background discovery fiber used by the router to find buckets. Learn more in vshard.router.discovery_set().

Примечание

This option should be defined at the global level.

Type: string

Default: „on“

Possible values: „on“, „off“, „once“

Environment variable: TT_SHARDING_DISCOVERY_MODE

sharding.failover_ping_timeout¶

The timeout (in seconds) after which a node is considered unavailable if there are no responses during this period. The failover fiber is used to detect if a node is down.

Примечание

This option should be defined at the global level.

Type: number

Default: 5

Environment variable: TT_SHARDING_FAILOVER_PING_TIMEOUT

sharding.lock¶

Whether a replica set is locked. A locked replica set cannot receive new buckets nor migrate its own buckets.

Примечание

sharding.lock can be specified at the replica set level or higher.

Type: boolean

Default: nil

Environment variable: TT_SHARDING_LOCK

sharding.rebalancer_disbalance_threshold¶

The maximum bucket disbalance threshold (in percent). The disbalance is calculated for each replica set using the following formula:

|etalon_bucket_count - real_bucket_count| / etalon_bucket_count * 100

Примечание

This option should be defined at the global level.

Type: number

Default: 1

Environment variable: TT_SHARDING_REBALANCER_DISBALANCE_THRESHOLD

sharding.rebalancer_max_receiving¶

The maximum number of buckets that can be received in parallel by a single replica set. This number must be limited because the rebalancer sends a large number of buckets from the existing replica sets to the newly added one. This produces a heavy load on the new replica set.

Примечание

This option should be defined at the global level.

Example:

Suppose, rebalancer_max_receiving is equal to 100 and bucket_count is equal to 1000. There are 3 replica sets with 333, 333, and 334 buckets on each respectively. When a new replica set is added, each replica set’s etalon_bucket_count becomes equal to 250. Rather than receiving all 250 buckets at once, the new replica set receives 100, 100, and 50 buckets sequentially.

Type: integer

Default: 100

Environment variable: TT_SHARDING_REBALANCER_MAX_RECEIVING

sharding.rebalancer_max_sending¶

The degree of parallelism for parallel rebalancing.

Примечание

This option should be defined at the global level.

Type: integer

Default: 1

Maximum: 15

Environment variable: TT_SHARDING_REBALANCER_MAX_SENDING

sharding.roles¶

Roles of a replica set in regard to sharding. A replica set can have the following roles:

• router: a replica set acts as a router.
• storage: a replica set acts as a storage.
• rebalancer: a replica set acts as a rebalancer.

The rebalancer role is optional. If it is not specified, a rebalancer is selected automatically from master instances of replica sets.

There can be at most one replica set with the rebalancer role. Additionally, this replica set should have a storage role.

Example:

replicasets:
  storage-a:
    sharding:
      roles: [storage, rebalancer]

Примечание

sharding.roles can be specified at the replica set level or higher.

Type: array

Default: nil

Environment variable: TT_SHARDING_ROLES

sharding.sched_move_quota¶

A scheduler’s bucket move quota used by the rebalancer.

sched_move_quota defines how many bucket moves can be done in a row if there are pending storage refs. Then, bucket moves are blocked and a router continues making map-reduce requests.

See also: sharding.sched_ref_quota.

Примечание

This option should be defined at the global level.

Type: number

Default: 1

Environment variable: TT_SHARDING_SCHED_MOVE_QUOTA

sharding.sched_ref_quota¶

A scheduler’s storage ref quota used by a router’s map-reduce API. For example, the vshard.router.map_callrw() function implements consistent map-reduce over the entire cluster.

sched_ref_quota defines how many storage refs, therefore map-reduce requests, can be executed on the storage in a row if there are pending bucket moves. Then, storage refs are blocked and the rebalancer continues bucket moves.

See also: sharding.sched_move_quota.

Примечание

This option should be defined at the global level.

Type: number

Default: 300

Environment variable: TT_SHARDING_SCHED_REF_QUOTA

sharding.shard_index¶

The name or ID of a TREE index over the bucket id. Spaces without this index do not participate in a sharded Tarantool cluster and can be used as regular spaces if needed. It is necessary to specify the first part of the index, other parts are optional.

Примечание

This option should be defined at the global level.

Type: string

Default: „bucket_id“

Environment variable: TT_SHARDING_SHARD_INDEX

sharding.sync_timeout¶

The timeout to wait for synchronization of the old master with replicas before demotion. Used when switching a master or when manually calling the sync() function.

Примечание

This option should be defined at the global level.

Type: number

Default: 1

Environment variable: TT_SHARDING_SYNC_TIMEOUT

sharding.zone¶

A zone that can be set for routers and replicas. This allows sending read-only requests not only to a master instance but to any available replica that is the nearest to the router.

Примечание

sharding.zone can be specified at any level.

Type: integer

Default: nil

Environment variable: TT_SHARDING_ZONE

snapshot

The snapshot section defines configuration parameters related to the snapshot files. To learn more about the snapshots“ configuration, check the Persistence page.

• snapshot.dir
• snapshot.snap_io_rate_limit
• snapshot.count
• snapshot.by.*
  • snapshot.by.interval
  • snapshot.by.wal_size

snapshot.dir¶

A directory where memtx stores snapshot (.snap) files. A relative path in this option is interpreted as relative to process.work_dir.

By default, snapshots and WAL files are stored in the same directory. However, you can set different values for the snapshot.dir and wal.dir options to store them on different physical disks for performance matters.

Type: string

Default: „var/lib/{{ instance_name }}“

Environment variable: TT_SNAPSHOT_DIR

snapshot.snap_io_rate_limit¶

Reduce the throttling effect of box.snapshot() on INSERT/UPDATE/DELETE performance by setting a limit on how many megabytes per second it can write to disk. The same can be achieved by splitting wal.dir and snapshot.dir locations and moving snapshots to a separate disk. The limit also affects what box.stat.vinyl().regulator may show for the write rate of dumps to .run and .index files.

Type: number

Default: box.NULL

Environment variable: TT_SNAPSHOT_SNAP_IO_RATE_LIMIT

snapshot.count¶

The maximum number of snapshots that are stored in the snapshot.dir directory. If the number of snapshots after creating a new one exceeds this value, the Tarantool garbage collector deletes old snapshots. If snapshot.count is set to zero, the garbage collector does not delete old snapshots.

Example

In the example, the checkpoint daemon creates a snapshot every two hours until it has created three snapshots. After creating a new snapshot (the fourth one), the oldest snapshot and any associated write-ahead-log files are deleted.

snapshot:
  by:
    interval: 7200
  count: 3

Примечание

Snapshots will not be deleted if replication is ongoing and the file has not been relayed to a replica. Therefore, snapshot.count has no effect unless all replicas are alive.

Type: integer

Default: 2

Environment variable: TT_SNAPSHOT_COUNT

wal

The wal section defines configuration parameters related to write-ahead log. To learn more about the WAL configuration, check the Persistence page.

• wal.cleanup_delay
• wal.dir
• wal.dir_rescan_delay
• wal.max_size
• wal.mode
• wal.queue_max_size
• wal.ext.*
  • wal.ext.new
  • wal.ext.old
  • wal.ext.spaces

wal.cleanup_delay¶

The delay in seconds used to prevent the Tarantool garbage collector from immediately removing write-ahead log files after a node restart. This delay eliminates possible erroneous situations when the master deletes WALs needed by replicas after restart. As a consequence, replicas sync with the master faster after its restart and don’t need to download all the data again. Once all the nodes in the replica set are up and running, a scheduled garbage collection is started again even if wal.cleanup_delay has not expired.

Примечание

The option has no effect on nodes running as anonymous replicas.

Type: number

Default: 14400

Environment variable: TT_WAL_CLEANUP_DELAY

wal.dir¶

A directory where write-ahead log (.xlog) files are stored. A relative path in this option is interpreted as relative to process.work_dir.

By default, WAL files and snapshots are stored in the same directory. However, you can set different values for the wal.dir and snapshot.dir options to store them on different physical disks for performance matters.

Type: string

Default: „var/lib/{{ instance_name }}“

Environment variable: TT_WAL_DIR

wal.dir_rescan_delay¶

The time interval in seconds between periodic scans of the write-ahead-log file directory, when checking for changes to write-ahead-log files for the sake of replication or hot standby.

Type: number

Default: 2

Environment variable: TT_WAL_DIR_RESCAN_DELAY

wal.max_size¶

The maximum number of bytes in a single write-ahead log file. When a request would cause an .xlog file to become larger than wal.max_size, Tarantool creates a new WAL file.

Type: integer

Default: 268435456

Environment variable: TT_WAL_MAX_SIZE

wal.mode¶

Specify fiber-WAL-disk synchronization mode as:

• none: write-ahead log is not maintained. A node with wal.mode set to none can’t be a replication master.
• write: fibers wait for their data to be written to the write-ahead log (no fsync(2)).
• fsync: fibers wait for their data, fsync(2) follows each write(2).

Type: string

Default: „write“

Environment variable: TT_WAL_MODE

wal.queue_max_size¶

The size of the queue in bytes used by a replica to submit new transactions to a write-ahead log (WAL). This option helps limit the rate at which a replica submits transactions to the WAL.

Limiting the queue size might be useful when a replica is trying to sync with a master and reads new transactions faster than writing them to the WAL.

Примечание

You might consider increasing the wal.queue_max_size value in case of large tuples (approximately one megabyte or larger).

Type: integer

Default: 16777216

Environment variable: TT_WAL_QUEUE_MAX_SIZE