Native Backup¶

Status¶

This section contains the list of stages in the backup procedure now managed by ScyllaDB. All other stages are still performed by the rclone server (e.g., deduplication of SSTables, retention of old backups). The ScyllaDB Version column describes the ScyllaDB version from which the functionality is considered production ready, even though the functionality might be available in earlier versions as well.

Configuration¶

To configure native backup, perform the following steps for each ScyllaDB node:

1. Make sure that ScyllaDB Manager Agent is configured as usual.

2. Configure object_storage_endpoints in scylla.yaml.

   This configures the backup location access for the ScyllaDB node itself. The backup location access for ScyllaDB Manager Agent needs to be configured separately, in the same way as it is done for the regular backup purposes (see Setup Amazon S3). The configurations in scylla.yaml and scylla-manager-agent.yaml should match. An example of matching configurations is shown below:

   scylla-manager-agent.yaml:

   s3:
     provider: AWS
     region: us-east-1
     endpoint: https://s3.us-east-1.amazonaws.com:443
   

   scylla.yaml:

   object_storage_endpoints:
     - name: s3.us-east-1.amazonaws
       port: 443
       https: true
       aws_region: us-east-1
   

3. Throttle stream_io_throughput_mb_per_sec in scylla.yaml.

   To ensure that native SSTable upload does not interfere with ScyllaDB performance, it should be throttled by configuring stream_io_throughput_mb_per_sec in scylla.yaml.

Usage¶

The native backup usage is controlled with the sctool backup –method flag. It supports three values: rclone (default), native, and auto:

• native: Uses all native backup functionalities listed in Status section. Use this value for native backup configuration validation and testing. Note that this will fail when:

  • ScyllaDB is not configured properly (see object_storage_endpoints in Configuration)

  • Provider other than S3 is used (see limitations in Status)

  • The upload of snapshot directories would result in creation of versioned SSTables (see limitations in Status)

• auto: Uses native backup functionalities when possible, otherwise falls back to rclone backup. Use this value for production backups. It will use native backup functionality only when it is considered production ready (see version support in Status). The fallback works on per snapshot directory basis, so it allows for utilizing native backup functionalities for most snapshot directories, even if a small subset of them contains versioned SSTables.

• rclone: Uses rclone backup functionalities only. This effectively disables all native backup functionalities.

Note that sctool backup –rate-limit –transfers flags do not take effect when using native backup, as the upload performance is controlled directly by ScyllaDB itself. To control upload performance on the ScyllaDB side, configure stream_io_throughput_mb_per_sec in scylla.yaml.

You can create a new backup task with the desired method:

sctool backup -c <cluster ID> -L <backup locations> --method native

You can also update existing backup task to use a different method:

sctool backup update -c <cluster ID> <backup task ID> -L <backup locations> --method auto