Audit module

Example on GitHub: audit_log

The audit module allows you to record various events occurred in Tarantool. Each event is an action related to authorization and authentication, data manipulation, administrator activity, or system events.

The module provides detailed reports of these activities and helps you find and fix breaches to protect your business. For example, you can see who created a new user and when.

It is up to each company to decide exactly what activities to audit and what actions to take. System administrators, security engineers, and people in charge of the company may want to audit different events for different reasons. Tarantool provides such an option for each of them.

The section describes how to enable and configure audit logging and write logs to a selected destination – a file, a pipe, or a system logger.

Read more: Audit log configuration reference.

To enable audit logging, define the log location using the audit_log.to option in the configuration file. Possible log locations:

• a file
• a pipe
• a system logger

In the configuration below, the audit_log.to option is set to file. It means that the logs are written to a file. By default, audit logs are saved in the var/log/{{ instance_name }}/audit.log file. To specify the path to an audit log file explicitly, use the audit_log.file option.

 
audit_log:  to: file  file: 'audit_tarantool.log'

If you log to a file, Tarantool reopens the audit log at SIGHUP.

To disable audit logging, set the audit_log.to option to devnull.

Tarantool's extensive filtering options help you write only the events you need to the audit log. To select the recorded events, use the audit_log.filter option. Its value can be a list of events and event groups. You can customize the filters and use different combinations of them for your purposes. Possible filtering options:

• Filter by event. You can set a list of events to be recorded. For example, select password_change to monitor the users who have changed their passwords:

   
  audit_log:  filter: [ password_change ]

• Filter by group. You can specify a list of event groups to be recorded. For example, select auth and priv to see the events related to authorization and granted privileges:

   
  audit_log:  filter: [ auth,priv ]

• Filter by group and event. You can specify a group and a certain event depending on the purpose. In the configuration below, user_create, data_operations, ddl, and custom are selected to see the events related to:

  • user creation
  • space creation, altering, and dropping
  • data modification or selection from spaces
  • custom events (any events added manually using the audit module API)

 
filter: [ user_create,data_operations,ddl,custom ]  format: json  spaces: [ bands ]  extract_key: truegroups:  group001:    replicasets:      replicaset001:        instances:          instance001:            iproto:              listen:              - uri: '127.0.0.1:3301'

Use the audit_log.format option to choose the format of audit log events – plain text, CSV, or JSON.

 
format: json  spaces: [ bands ]  extract_key: truegroups:  group001:    replicasets:      replicaset001:        instances:          instance001:            iproto:              listen:              - uri: '127.0.0.1:3301'

JSON is used by default. It is more convenient to receive log events, analyze them, and integrate them with other systems if needed. The plain format can be efficiently compressed. The CSV format allows you to view audit log events in tabular form.

The audit_log.spaces option is used to specify a list of space names for which data operation events should be logged.

In the configuration below, only the events from the bands space are logged:

 
spaces: [ bands ]  extract_key: truegroups:  group001:    replicasets:      replicaset001:        instances:          instance001:            iproto:              listen:              - uri: '127.0.0.1:3301'

If set to true, the audit_log.extract_key option forces the audit subsystem to log the primary key instead of a full tuple in DML operations.

 
extract_key: truegroups:  group001:    replicasets:      replicaset001:        instances:          instance001:            iproto:              listen:              - uri: '127.0.0.1:3301'

In this example, the following audit log configuration is used:

 
audit_log:  to: file  file: 'audit_tarantool.log'  filter: [ user_create,data_operations,ddl,custom ]  format: json  spaces: [ bands ]  extract_key: true

Create a space bands and check the logs in the file after the creation:

 
box.schema.space.create('bands')

The audit log entry for the space_create event might look as follows:

 
{  "time": "2024-01-24T11:43:21.566+0300",  "uuid": "26af0a7d-1052-490a-9946-e19eacc822c9",  "severity": "INFO",  "remote": "unix/:(socket)",  "session_type": "console",  "module": "tarantool",  "user": "admin",  "type": "space_create",  "tag": "",  "description": "Create space Bands"}

Then insert one tuple to space:

 
box.space.bands:insert { 1, 'Roxette', 1986 }

If the extract_key option is set to true, the audit system prints the primary key instead of the full tuple:

 
{  "time": "2024-01-24T11:45:42.358+0300",  "uuid": "b437934d-62a7-419a-8d59-e3b33c688d7a",  "severity": "VERBOSE",  "remote": "unix/:(socket)",  "session_type": "console",  "module": "tarantool",  "user": "admin",  "type": "space_insert",  "tag": "",  "description": "Insert key [1] into space bands"}

If the extract_key option is set to false, the audit system prints the full tuple like this:

 
{  "time": "2024-01-24T11:45:42.358+0300",  "uuid": "b437934d-62a7-419a-8d59-e3b33c688d7a",  "severity": "VERBOSE",  "remote": "unix/:(socket)",  "session_type": "console",  "module": "tarantool",  "user": "admin",  "type": "space_insert",  "tag": "",  "description": "Insert tuple [1, "Roxette", 1986] into space bands"}

The Tarantool audit log module can record various events that you can monitor and decide whether you need to take actions:

• Administrator activity – events related to actions performed by the administrator. For example, such logs record the creation of a user.
• Access events – events related to authorization and authentication of users. For example, such logs record failed attempts to access secure data.
• Data access and modification – events of data manipulation in the storage.
• System events – events related to modification or configuration of resources. For example, such logs record the replacement of a space.
• Custom events – any events added manually using the audit module API.

The full list of available audit log events is provided in the table below:

Each audit log event contains a number of fields that can be used to filter and aggregate the resulting logs. An example of a Tarantool audit log entry in JSON:

 
{    "time": "2024-01-15T13:39:36.046+0300",    "uuid": "cb44fb2b-5c1f-4c4b-8f93-1dd02a76cec0",    "severity": "VERBOSE",    "remote": "unix/:(socket)",    "session_type": "console",    "module": "tarantool",    "user": "admin",    "type": "auth_ok",    "tag": "",    "description": "Authenticate user Admin"}

Each event consists of the following fields:

Built-in event groups are used to filter the event types that you want to audit. For example, you can set to record only authorization events or only events related to a space.

Tarantool provides the following event groups:

• all – all events.

• audit – audit_enable event.

• auth – authorization events: auth_ok, auth_fail.

• priv – events related to authentication, authorization, users, and roles: user_create, user_drop, role_create, role_drop, user_enable, user_disable, user_grant_rights, user_revoke_rights, role_grant_rights, role_revoke_rights.

• ddl – events of space creation, altering, and dropping: space_create, space_alter, space_drop.

• dml – events of data modification in spaces: space_insert, space_replace, space_delete.

• data_operations – events of data modification or selection from spaces: space_select, space_insert, space_replace, space_delete.

• compatibility – events available in Tarantool before the version 2.10.0. auth_ok, auth_fail, disconnect, user_create, user_drop, role_create, role_drop, user_enable, user_disable, user_grant_rights, user_revoke_rights, role_grant_rights. role_revoke_rights, password_change, access_denied. This group enables the compatibility with earlier Tarantool versions.

Tarantool provides an API for writing custom audit log events. To enable these events, specify the custom value in the audit_log.filter option:

 
filter: [ user_create,data_operations,ddl,custom ]  format: json  spaces: [ bands ]  extract_key: truegroups:  group001:    replicasets:      replicaset001:        instances:          instance001:            iproto:              listen:              - uri: '127.0.0.1:3301'

To log an event, use the audit.log() function that takes one of the following values:

• Message string. Printed to the audit log with type message:

 
audit.log('Hello, Alice!')

• Format string and arguments. Passed to string format and then output to the audit log with type message:

 
audit.log('Hello, %s!', 'Bob')

• Table with audit log field values. The table must contain at least one field – description.

Alternatively, you can use audit.new() to create a new log module. This allows you to avoid passing all custom audit log fields each time audit.log() is called. The audit.new() function takes a table of audit log field values (same as audit.log()). The type of the log module for writing custom events must either be message or have the custom_ prefix.

 
local my_audit = audit.new({ type = 'custom_hello', module = 'my_module' })my_audit:log('Hello, Alice!')my_audit:log({ tag = 'admin', description = 'Hello, Bob!' })

It is possible to overwrite most of the custom audit log fields using audit.new() or audit.log(). The only audit log field that cannot be overwritten is time.

If omitted, the session_type is set to the current session type, remote is set to the remote peer address.

By default, custom events have the INFO severity level. To override the level, you can:

• specify the severity field
• use a shortcut function

The following shortcuts are available:

Example

If you write to a file, the size of the Tarantool audit log is limited by the disk space. If you write to a system logger, the size of the Tarantool audit log is limited by the system logger. If you write to a pipe, the size of the Tarantool audit message is limited by the system buffer. If the audit_log.nonblock = false, if audit_log.nonblock = true, there is no limit.

Consider setting up a schedule in your company. It is recommended to review audit logs at least every 3 months.

It is recommended to store audit logs for at least one year.

It is recommended to use SIEM systems for this issue.