ScyllaDB Documentation Logo Documentation
  • Server
    • ScyllaDB Open Source
    • ScyllaDB Enterprise
    • ScyllaDB Alternator
  • Cloud
  • Tools
    • ScyllaDB Manager
    • ScyllaDB Monitoring Stack
    • ScyllaDB Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
Download
Menu
Scylla Manager Restore

Restore¶

The following procedure lets you restore data to a new cluster to get a cluster clone from snapshots stored in a backup location.

Prerequisites

  1. New cluster with the same number of nodes as the source cluster.

  2. Scylla Manager Agent installed on all the nodes.

  3. Access to the backup location from all the nodes.

  4. Ansible installed on your local machine.

  5. Access to all the nodes over SSH from your local machine.

  6. The new cluster time (wall clock on each node) is later than the backup time, so table creation time will not override.

If you do not know the exact number of nodes of the source cluster, you can create a single node cluster and discover the number in the process. In fact you can use any Scylla node with Scylla Manager Agent installed that has access to the backup location for getting the parameters. Note that there is no need for Scylla Manager Server installation prior to restore.

This procedure can only restore complete cluster data, not a subset of keyspace or tables.

Ansible¶

Cloning the cluster is automated with Ansible playbook. The playbook has a readme that explains how to use it. For reader’s convenience it’s repeated here.

Getting the playbook¶

Procedure

  1. Clone repository from GitHub git clone git@github.com:scylladb/scylla-manager.git

  2. Go to playbook directory cd scylla-manager/ansible/restore

Playbook parameters¶

All restore parameters shall be put to vars.yaml file. Copy vars.yaml.example as vars.yaml and change parameters to match your clusters.

Note

make sure to update your var.yaml so the IPs of the destination cluster map to UUIDs of nodes of the source cluster.

For example, for a source cluster

> nodetool status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens       Owns    Host ID                               Rack
UN  10.240.0.6   119.6 MB   256          ?       9b6b5206-63c8-4cdf-9b4f-23e37977fd14  rack1
UN  10.240.0.93  122.69 MB  256          ?       2d8045e7-51a4-42c4-bb8f-5e8aa77a8228  rack1

and a Destination cluster:

> nodetool status

Datacenter: us-east1
====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address        Load       Tokens       Owns    Host ID                               Rack
UN  75.196.92.133  122.14 MB  256          ?       a21b38b4-c0b1-49f0-89f2-931fe027d6b0  b
UN  75.237.179.49  121.22 MB  256          ?       a0a63682-95e3-4cf3-9f46-4d551bd7eba2  b

vars.yaml file would look like:

# backup_location specifies the location parameter used in Scylla Manager
# when scheduling a backup of a cluster.
backup_location: gcs:manager-bucket

# snapshot_tag specifies the Scylla Manager snapshot tag you want to restore.
snapshot_tag: sm_20220204002134UTC

# host_id specifies a mapping from the clone cluster node IP to the source
# cluster host IDs.
host_id:
  75.237.179.49: 9b6b5206-63c8-4cdf-9b4f-23e37977fd14
  75.196.92.133: 2d8045e7-51a4-42c4-bb8f-5e8aa77a8228
# destination IP: source ID

The hosts file would look like:

75.237.179.49
75.196.92.133

IP to host ID mapping¶

SSH to one of the nodes and execute the following command:

scylla-manager-agent download-files -L <backup-location> --list-nodes

it gives you information about all the clusters and nodes available in the backup location.

Example output:

Cluster: prod (a9dcc6e1-17dc-4520-9e03-0a92011c823c)
AWS_EU_CENTRAL_1:
- 18.198.164.180 (7e68421b-acb1-44a7-a1a8-af7eaf1bb482)
- 3.122.35.120 (adc0a3ce-dade-4672-981e-26f91a3d35cb)
- 3.127.175.215 (2a575244-3e3c-44a1-a526-da4394f9525e)
- 3.65.85.108 (82f0f486-370d-4cfd-90ac-46464c8012cb)
- 3.66.209.145 (9ee92c19-5f78-4865-a287-980218963d96)
- 3.66.44.239 (aff05f79-7c69-4ecf-a827-5ea790a0fdc6)

Cluster: test (da5721cd-e2eb-4d10-a3a7-f729b8f72abf)
AWS_EU_CENTRAL_1:
- 3.123.55.215 (4001206a-3377-40cb-abd4-d38aad5dec41)
- 3.127.1.189 (c6466011-02f9-49dd-8951-c32028dfc6f1)
- 3.64.219.214 (bc39bb07-7a21-41cd-b576-51d44c1a694a)

For each node IP in the new cluster you need to assign a host ID (UUID in the listing above). The mapping must be put into host_id variable in vars.yaml file.

Snapshot tag¶

To list available snapshot tags for a node use --list-snapshots flag.

scylla-manager-agent download-files -L <backup-location> --list-snapshots -n <host-id>

You can filter snapshot tags containing a specific keyspaces or tables by using glob patterns. The parameter -K, --keyspace <list of glob patterns to find keyspaces> lets you do that. It works the same way as in scheduling backups or repairs with Scylla Manager.

The snapshot ID must be put into snapshot_tag variable in vars.yaml file.

Inventory¶

Put public IP addresses of all nodes to hosts file.

Running the playbook¶

Rut the playbook:

ansible-playbook -i hosts -e @vars.yaml restore.yaml
PREVIOUS
Specification
NEXT
Download files command
  • 3.0
    • master
    • 3.0
    • 2.6
    • 2.5
    • 2.4
    • 2.3
    • 2.2
  • Docker
  • Install Scylla Manager
  • Install Scylla Manager Agent
  • Scylla Manager Upgrade
  • Add a Cluster
  • Backup
    • Setup AWS S3
    • Setup S3 compatible storage
    • Setup Google Cloud Storage
    • Setup Azure Blob Storage
    • Examples
    • Specification
  • Restore
    • Download files command
  • Repair
    • Repair faster
    • Repair slower
    • Examples
  • Health Check
  • CLI sctool
    • Environment variables
    • Completion
    • Backup
    • Cluster
    • Info
    • Progress
    • Repair
    • Start
    • Status
    • Stop
    • Suspend & Resume
    • Tasks
    • Version
  • Configuration Files
    • Scylla Manager Config
    • Scylla Manager Agent Config
  • Swagger File
  • Scylla Monitoring
  • Troubleshooting
  • Older Versions
  • Slack
  • Create an issue
  • Edit this page

On this page

  • Restore
    • Ansible
      • Getting the playbook
      • Playbook parameters
        • IP to host ID mapping
        • Snapshot tag
        • Inventory
      • Running the playbook
Logo
Docs Contact Us About Us
Mail List Icon Slack Icon Forum Icon
© 2023, ScyllaDB. All rights reserved.
Last updated on 15 February 2023.
Powered by Sphinx 4.3.2 & ScyllaDB Theme 1.3.4