Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

...

Offline Backup/Restore of Trafodion

Trafodion full offline backup and restore operation are performed using HBase snapshot fetaure. Snapshots are a light weight way to keep the current state of a table without copying the data. After making changes to a table or in case of recovery, restoring the snapshot gives the previous state of the table. More on snapshots can be found at: http://blog.cloudera.com/blog/2013/03/introduction-to-apache-hbase-snapshots/.

Snapshots are taken for all the Trafodion tables including the Trafodion metadata tables and then exported to a HDFS location using the HBase ExportSnapshot Mapreduce job. When we need to restore the backed files from the HDFS location the snapshots are imported to the target system using the ExportSnapshot MapReduce job also and then restored. In the case of Trafodion full backup/restore the The process is as follows:

Full Offline Backup

  • To do a backup snapshots Trafodion needs to be completely down before starting offline backup
  • Snapshots are taken for all the Trafodion tables using the HBase shell snapshot command.Once snapshots are taken, they
  • Trafodion is restarted. Regular activity can resume on these tables, but they will not be included in the backup.
  • Snapshots are exported one by one to HDFS folder using the ExportSnapshot MapReduce job that is part of HBase install base. The MapReduce job can be run in parallel using –m option.If needed the TrafExportSnapshot class. For small tables this uses HDFS file copy while for large tables MapReduce is to copy HFiles. The number of mappers can vary by table. It is determined is determined by size of table and number of files to be copy for that table,
  • If necessary, backup files can be archived or moved to a different cluster using hadoop distcp or other methods.

Full Offline Restore

  • Trafodion needs to be completely down before doing the starting full offline backup

Full Offline Restore

  • To perform the restore the snapshots restore
  • There must be no Trafodion related tables in HBase 
  • Snapshots that were saved in HDFS are imported one by one to HBase using the ExportSnapshot MapReduce job. The MapReduce job can be run in parallelTrafExportSnapshot class. This is reverse of the copy that was done during backup.
  • Once the snapshots are imported, they can be are restored using ‘restore_snapshot‘ HBase shell command.

Trafodion

...

Trafodion Tables

Trafodion tables are created in HBase with names that have a format like ‘TRAFODION.<schema-name>.<table-name>’. To back up all Trafodion tables, the backup tool selects and backs up all table that have a name pattern like TRAFODION.<any-schema-name>.<any-table-name>. Besides user tables, metadata tables (TRAFODION._MD_.<table-name>) and Repository table repository tables (TRAFODION._REPOS_.<table-name> are ) and transaction related tables (TRAFODION._MD_.<table-name>) are also backed up.

Sudo Access

The backup and restore tools can be used in both the development and clustered environment with either Cloudera or Horton Works distributions. To run the backup and restore scripts in a clustered environment, the user running the scripts needs to have sudo access to be able to run commands as HBase user (user under which HBase server runs), HDFS user and Trafodion user without requiring to enter a password. However sudo access is not needed when the scripts are run in the development environment.

Backup Files Security

The backup and restore operations should be performed using a user with sudo access that can sudo to HBase user. In a clustered environment one possibility is that root or some admin type id is used to run backup and restore. An alternative is that trafodion user is given access to run any command under the HBase or HDFS user id with a password prompt. If this acceptable from a security perspective it can be achieved by adding this line to /etc/sudoers

trafodion  ALL=(hbase) NOPASSWD: ALL, (hdfs) NOPASSWD: ALL, (trafodion) NOPASSWD: ALL

Backup Files Security

Both backup and restore use the ExportSnapshot MapReduce job which TrafExportSnapshot  which requires read and write access to these locations:

...

./run_full_trafodion_backup.sh -b backup_folder -u trafodion_user -h HBase_user -d hdfs_user -m mappers -l 10 -n -o

Where:

-b backup_folder
      (Optional) HDFS path where all the Trafodion object are exported and saved
      The HDFS path needs to have a format like hdfs://<host>:<port>/<folder>/...
      If the path is not provided the script generates a path with a format like
      hdfs://<host>:<port>/trafodion-backlups/backup_<timestamp> and unless -n
      is specified the user is asked whether to confirm the use of the generated
      path.

-u trafodion user
      (Optional) The user under which Trafodion server runs. If not provided and
      if -n option is not specified the user is asked whether the default
      trafodion user 'trafodion' can be used or not. If the answer is yes then the
       default trafodion user is used otherwise the script exits.

-h hbase user
      (Optional) The user under which HBase server runs. If not provided the script
      tries to compute it and if it does not succeed it considers using the default
      HBase user 'hbase'. Unless the -n option is specified, the user is asked to
      confirm the selection afterwards.

-d hdfs user
      (Optional) The user under which HDFS server runs. If not provided the script
      tries to compute it and if it does not succeed it considers using the default
      HDFS user 'hdfs'. Unless the -n option is specified, the user is asked to
      confirm the selection afterwards.''

-m mappers
(Optional) Number of mappers. TheIf defaultunspecified valueor is0, 2.each snapshot will use a number suitable for its size.
-n
       (Optional) Non interactive mode. With this option the script does not prompt
the user to confirm the use of computed or default values when a parameter
like trafodion user, HBasehbase user, hdfs user or backup path is not provided.
-o
(Optional) offline. With this option trafodion will not be restarted after
snapshots are taken.
-l
(Optional) Snapshot size limit in MB above which map reduce is used for copy. Snapshots with size below this value 
will be copied using HDFS FileUtil.copy. Default value is 100 MB. FileUtil.copy is invoked through a class provided
by Trafodion. Use 0 for this option to use HBase' ExportSnaphot class instead.
 
Example: ./run_full_trafodion_backup.sh  -b hdfs://<host>:<port>/<hdfs-path>  -mn 4    

Checks performed before backing up the Trafodion tables

...

./run_full_trafodion_restore.sh -b backup_folder -b backup_dir -u trafodion_user -h hbase_user  -m mappers -nl 10 -n
Where:

-b  backup_folder
      (Not Optional) HDFS path where all the Trafodion object are exported and
       saved The HDFS path needs to have a format like hdfs://<host>:<port>/<folder>/...

-u trafodion user
      (Optional) The user under which Trafodion server runs. If not provided and
      if -n option is not specified the user is asked whether the default
      trafodion user 'trafodion' can be used or not. If the answer is yes then the
       default trafodion user is used otherwise the script exits.

-h hbase user
      (Optional) The user under which HBase server runs. If not provided the script
      tries to compute it and if it does not succeed it considers using the default
      hbase user 'hbase'. Unless the -n option is specified, the user is asked to
       confirm the selection afterwards.

-d hdfs user
      (Optional) The user under which HDFS server runs. If not provided the script
      tries to compute it and if it does not succeed it considers using the default
      HDFS user 'hdfs'. Unless the -n option is specified, the user is asked to
       confirm the selection afterwards.

-m mappers
(Optional) Number of mappers. TheIf default unspecified or 0, each snapshot will use a number suitable for its size.
-l
(Optional) Snapshot size limit in MB above which map reduce is used for copy. Snapshots with size below this value 
will be copied using HDFS FileUtil.copy. Default value is 2100 MB. FileUtil.copy is invoked through a class provided
by Trafodion. Use 0 for this option to use HBase' ExportSnaphot class instead.
-n
      (Optional) Non interactive mode. With this option the script does not prompt
      the user to confirm the use of computed or default values when a parameter
       like trafodion user, HBase user, or hdfs user or backup path is not provided.
Example: ./run_full_trafodion_restore.sh  -b hdfs://<host>:<port>/<hdfs-path>  -mn 4  

Checks performed prior to starting restore

...

./run_full_trafodion_backup.sh -b hdfs://<host>:8020/bulkload/backup -mn
16

Where hdfs://<host>:8020/bulkload/backup is the hdfs location where we want the backed up snapshot to reside in this example.

...

./run_full_trafodion_restore.sh  -b /bulkload/backup -mn
4

Backup/Restore on development workstation

For the ExportSnapshot TrafExportSnapshot to work in the development environment, we need to copy few jar files under $MY_SQROOT/sql/local_hadoop/hbase/lib directory. Otherwise we get exceptions errors when running the ExportSnapshot MapReduce job. These workaround were figured out while testing on the workstations and there could be other ways to avoid the exceptions.

...

./run_full_trafodion_backup.sh -b hdfs://localhost:<port-number>/bulkload/backup -mn
4

Restore

To run the restore:

run_full_trafodion_restore.sh -b hdfs://localhost:<port-number>/bulkload/backup -mn
4

Testing

The test cases that were performed so far include:

...