Storage API

Storage public API

vshard.storage.cfg(cfg, instance_uuid)¶

Configure the database and start sharding for the specified storage instance.

Parameters:	cfg – a storage configuration instance_uuid – UUID of the instance

vshard.storage.info({options})¶

Return information about the storage instance. Since vshard v.0.1.22, the function also accepts options, which can be used to get additional information.

Parameters:	options – with_services — a bool value. If set to true, the function returns information about the background services (such as garbage collector, rebalancer, recovery, or applier of the routes) that are working on the current instance. See vshard.router.info for detailed reference.

Example:

tarantool> vshard.storage.info()
---
- replicasets:
    c862545d-d966-45ff-93ad-763dce4a9723:
      uuid: c862545d-d966-45ff-93ad-763dce4a9723
      master:
        uri: admin@localhost:3302
    1990be71-f06e-4d9a-bcf9-4514c4e0c889:
      uuid: 1990be71-f06e-4d9a-bcf9-4514c4e0c889
      master:
        uri: admin@localhost:3304
  bucket:
    receiving: 0
    active: 15000
    total: 15000
    garbage: 0
    pinned: 0
    sending: 0
  status: 0
  replication:
    status: master
  alerts: []
...

vshard.storage.call(bucket_id, mode, function_name, {argument_list})¶

Call the specified function on the current storage instance.

Parameters:	bucket_id – a bucket identifier mode – a type of the function: ‘read’ or ‘write’ function_name – function to execute argument_list – array of the function’s arguments
Return:

The original return value of the executed function, or nil and error object.

vshard.storage.sync(timeout)¶

Wait until the dataset is synchronized on replicas.

Parameters:	timeout – a timeout, in seconds
Return:	true if the dataset was synchronized successfully; or nil and err explaining why the dataset cannot be synchronized.

vshard.storage.bucket_pin(bucket_id)¶

Pin a bucket to a replica set. A pinned bucket cannot be moved even if it breaks the cluster balance.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket is pinned successfully; or nil and err explaining why the bucket cannot be pinned

vshard.storage.bucket_unpin(bucket_id)¶

Return a pinned bucket back into the active state.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket is unpinned successfully; or nil and err explaining why the bucket cannot be unpinned

vshard.storage.bucket_ref(bucket_id, mode)¶

Create an RO or RW ref.

Parameters:	bucket_id – a bucket identifier mode – ‘read’ or ‘write’
Return:	true if the bucket ref is created successfully; or nil and err explaining why the ref cannot be created

vshard.storage.bucket_refro(bucket_id)¶

An alias for vshard.storage.bucket_ref in the RO mode.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket ref is created successfully; or nil and err explaining why the ref cannot be created

vshard.storage.bucket_refrw(bucket_id)¶

An alias for vshard.storage.bucket_ref in the RW mode.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket ref is created successfully; or nil and err explaining why the ref cannot be created

vshard.storage.bucket_unref(bucket_id, mode)¶

Remove a RO/RW ref.

Parameters:	bucket_id – a bucket identifier mode – ‘read’ or ‘write’
Return:	true if the bucket ref is removed successfully; or nil and err explaining why the ref cannot be removed

vshard.storage.bucket_unrefro(bucket_id)¶

An alias for vshard.storage.bucket_unref in the RO mode.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket ref is removed successfully; or nil and err explaining why the ref cannot be removed

vshard.storage.bucket_unrefrw(bucket_id)¶

An alias for vshard.storage.bucket_unref in the RW mode.

Parameters:	bucket_id – a bucket identifier
Return:	true if the bucket ref is removed successfully; or nil and err explaining why the ref cannot be removed

vshard.storage.find_garbage_bucket(bucket_index, control)¶

Find a bucket which has data in a space but is not stored in a _bucket space; or is in a GARBAGE state.

Parameters:	bucket_index – index of a space with the part of a bucket id control – a garbage collector controller. If there is an increased buckets generation, then the search should be interrupted.
Return:	an identifier of the bucket in the garbage state, if found; otherwise, nil

vshard.storage.buckets_info()¶

Return information about each bucket located in storage. For example:

tarantool> vshard.storage.buckets_info(1)
---
- 1:
    status: active
    ref_rw: 1
    ref_ro: 1
    ro_lock: true
    rw_lock: true
    id: 1

vshard.storage.buckets_count()¶

Return the number of buckets located in storage.

vshard.storage.recovery_wakeup()¶

Immediately wake up a recovery fiber, if it exists.

vshard.storage.rebalancing_is_in_progress()¶

Return a flag indicating whether rebalancing is in progress. The result is true if the node is currently applying routes received from a rebalancer node in the special fiber.

vshard.storage.is_locked()¶

Return a flag indicating whether storage is invisible to the rebalancer.

vshard.storage.rebalancer_disable()¶

Disable rebalancing. A disabled rebalancer sleeps until it is enabled again with vshard.storage.rebalancer_enable().

vshard.storage.rebalancer_enable()¶

Enable rebalancing.

vshard.storage.sharded_spaces()¶

Show the spaces that are visible to rebalancer and garbage collector fibers.

tarantool> vshard.storage.sharded_spaces()
---
- 513:
    engine: memtx
    before_replace: 'function: 0x010e50e738'
    field_count: 0
    id: 513
    on_replace: 'function: 0x010e50e700'
    temporary: false
    index:
      0: &0
        unique: true
        parts:
        - type: number
          fieldno: 1
          is_nullable: false
        id: 0
        type: TREE
        name: primary
        space_id: 513
      1: &1
        unique: false
        parts:
        - type: number
          fieldno: 2
          is_nullable: false
        id: 1
        type: TREE
        name: bucket_id
        space_id: 513
      primary: *0
      bucket_id: *1
    is_local: false
    enabled: true
    name: actors
    ck_constraint: []
...

vshard.storage.on_bucket_event([trigger-function[, old-trigger-function]])¶

Since vshard v.0.1.22. Define a trigger for execution when the data from the user spaces is changed (deleted or inserted) due to the rebalancing process. The trigger is invoked each time the data batch changes.

Parameters:	trigger-function (function) – function which will become the trigger function. old-trigger-function (function) – existing trigger function which will be replaced by trigger-function.
Return:	nil or function pointer

The trigger-function can have up to three parameters:

• event_type (string) – in order to distinguish event, you can compare this argument with the supported event types, bucket_data_recv_txn and bucket_data_gc_txn.
• bucket_id (unsigned) – bucket id.
• data (table) – additional information about data change transaction. Currently it only includes an array of all spaces (data.spaces), affected by a transaction in which trigger-function is executed.

Example:

vshard.storage.on_bucket_event(function(event, bucket_id, data)
    if event == 'bucket_data_recv_txn' then
        -- Handle it.
        for idx, space in ipairs(data.spaces) do
            ...
        end
    elseif event == 'bucket_data_gc_txn' then
        -- Handle it.
        ...
    end
end)

Note

As everything executed inside triggers is already in a transaction, you shouldn’t use transactions, yield-operations (explicit or not), changes to different space engines (see rule #2).

If the parameters are (nil, old-trigger-function), then the old trigger is deleted. If both parameters are omitted, then the response is a list of existing trigger functions.

Details about trigger characteristics are in the triggers section.

Storage internal API

vshard.storage.bucket_recv(bucket_id, from, data)¶

Receive a bucket identified by bucket id from a remote replica set.

Parameters:	bucket_id – a bucket identifier from – UUID of source replica set data – data logically stored in a bucket identified by bucket_id, in the same format as the return value from bucket_collect() <storage_api-bucket_collect>

vshard.storage.bucket_stat(bucket_id)¶

Return information about the bucket id:

tarantool> vshard.storage.bucket_stat(1)
---
- 0
- status: active
  id: 1
...

Parameters:	bucket_id – a bucket identifier

vshard.storage.bucket_delete_garbage(bucket_id)¶

Force garbage collection for the bucket identified by bucket_id in case the bucket was transferred to a different replica set.

Parameters:	bucket_id – a bucket identifier

vshard.storage.bucket_collect(bucket_id)¶

Collect all the data that is logically stored in the bucket identified by bucket_id:

tarantool> vshard.storage.bucket_collect(1)
---
- 0
- - - 514
    - - [10, 1, 1, 100, 'Account 10']
      - [11, 1, 1, 100, 'Account 11']
      - [12, 1, 1, 100, 'Account 12']
      - [50, 5, 1, 100, 'Account 50']
      - [51, 5, 1, 100, 'Account 51']
      - [52, 5, 1, 100, 'Account 52']
  - - 513
    - - [1, 1, 'Customer 1']
      - [5, 1, 'Customer 5']
...

Parameters:	bucket_id – a bucket identifier

vshard.storage.bucket_force_create(first_bucket_id, count)¶

Force creation of the buckets (single or multiple) on the current replica set. Use only for manual emergency recovery or for initial bootstrap.

Parameters:	first_bucket_id – an identifier of the first bucket in a range count – the number of buckets to insert (default = 1)

vshard.storage.bucket_force_drop(bucket_id)¶

Drop a bucket manually for tests or emergency cases.

Parameters:	bucket_id – a bucket identifier

vshard.storage.bucket_send(bucket_id, to)¶

Send a specified bucket from the current replica set to a remote replica set.

Parameters:	bucket_id – bucket identifier to – UUID of a remote replica set

vshard.storage.rebalancer_request_state()¶

Check all buckets of the host storage that have the SENT or ACTIVE state, return the number of active buckets.

Return:	the number of buckets in the active state, if found; otherwise, nil

vshard.storage.buckets_discovery()¶

Collect an array of active bucket identifiers for discovery.