Redis数据导入导出方法

@admin  May 10, 2020

redis数据导入导出常见的大致有redis-dump、aof、rdb文件迁移三种方案。以下是根据各路大神、官方的资料,再结合实际场景中的操作整理而来的。

redis-dump

安装

[root@lnmp software]# yum install ruby rubygems ruby-devel -y # 安装依赖
[root@lnmp software]# gem sources --add http://gems.ruby-china.com/ --remove https://rubygems.org/ # 修改ruby源为国内源
http://gems.ruby-china.com/ added to sources
https://rubygems.org/ removed from sources
[root@lnmp software]# curl -L get.rvm.io | bash -s stable
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   194  100   194    0     0    126      0  0:00:01  0:00:01 --:--:--   126
100 24535  100 24535    0     0    642      0  0:00:38  0:00:38 --:--:--  2192
Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: Signature made Thu 26 Mar 2020 05:58:42 AM CST using RSA key ID 39499BDB
gpg: Can't check signature: No public key
GPG signature verification failed for '/usr/local/rvm/archives/rvm-1.29.10.tgz' - 'https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc'! Try to install GPG v2 and then fetch the public key:

    gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

or if it fails:

    command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
    command curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -

In case of further problems with validation please refer to https://rvm.io/rvm/security

[root@lnmp software]# gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 
command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
command curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -

In case of further problems with validation please refer to https://rvm.io/rvm/security
[root@lnmp software]# gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
[root@lnmp software]# curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
[root@lnmp software]# curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -
[root@lnmp software]# curl -L get.rvm.io | bash -s stable
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   194  100   194    0     0     76      0  0:00:02  0:00:02 --:--:--    76
100 24535  100 24535    0     0   7014      0  0:00:03  0:00:03 --:--:-- 2114k
Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: Signature made Thu 26 Mar 2020 05:58:42 AM CST using RSA key ID 39499BDB
gpg: Good signature from "Piotr Kuczynski <piotr.kuczynski@gmail.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7D2B AF1C F37B 13E2 069D  6956 105B D0E7 3949 9BDB
GPG verified '/usr/local/rvm/archives/rvm-1.29.10.tgz'
Creating group 'rvm'
Installing RVM to /usr/local/rvm/
Installation of RVM in /usr/local/rvm/ is almost complete:

  * First you need to add all users that will be using rvm to 'rvm' group,
    and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`.

  * To start using RVM you need to run `source /etc/profile.d/rvm.sh`
    in all your open shell windows, in rare cases you need to reopen all shell windows.
  * Please do NOT forget to add your users to the rvm group.
     The installer no longer auto-adds root or users to the rvm group. Admins must do this.
     Also, please note that group memberships are ONLY evaluated at login time.
     This means that users must log out then back in before group membership takes effect!
Thanks for installing RVM 
Please consider donating to our open collective to help us maintain RVM.

Donate: https://opencollective.com/rvm/donate


[root@lnmp software]# rvm
-bash: rvm: command not found
[root@lnmp software]# ls  /usr/local/rvm/
archives  contrib       examples   gemsets  installed.at  log      patchsets  rubies   tmp      wrappers
bin       docs          gem-cache  help     lib           man      README.md  scripts  user
config    environments  gems       hooks    LICENSE       patches  RELEASE    src      VERSION
[root@lnmp software]# ls  /usr/local/rvm/bin/
ruby-rvm-env  rvm  rvm-auto-ruby  rvm-exec  rvm-prompt  rvm-shebang-ruby  rvm-shell  rvm-smile  rvmsudo
[root@lnmp software]#  source /etc/profile.d/rvm.sh
[root@lnmp software]# rvm
Ruby enVironment Manager 1.29.10 (latest) (c) 2009-2017 Michal Papis, Piotr Kuczynski, Wayne E. Seguin

Usage:
 
    rvm [--debug][--trace][--nice] <command> <options>

  for example:

    rvm list                # list installed interpreters 
    rvm list known          # list available interpreters
    rvm install <version>   # install ruby interpreter
    rvm use <version>       # switch to specified ruby interpreter
    rvm remove <version>    # remove ruby interpreter
    rvm get <version>       # upgrade rvm: stable, master

...

redis-dump需要ruby 2.4,centos7系统自带的是ruby 2.0版本,所以我们需要把ruby版本升级到2.4.

[root@lnmp software]# rvm list known
[root@lnmp software]# rvm install 2.4
Searching for binary rubies, this might take some time.
No binary rubies available for: centos/7/x86_64/ruby-2.4.9.
Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.
Checking requirements for centos.
Installing requirements for centos.
Installing required packages: libffi-devel, sqlite-devel.......
Requirements installation successful.
Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.4.9, this may take a while depending on your cpu(s)...
ruby-2.4.9 - #downloading ruby-2.4.9, this may take a while depending on your connection...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.9M  100 11.9M    0     0  5327k      0  0:00:02  0:00:02 --:--:-- 5330k
ruby-2.4.9 - #extracting ruby-2.4.9 to /usr/local/rvm/src/ruby-2.4.9.....
ruby-2.4.9 - #configuring..................................................................
ruby-2.4.9 - #post-configuration..
ruby-2.4.9 - #compiling...................................................................................................................
ruby-2.4.9 - #installing...........................
ruby-2.4.9 - #making binaries executable..
ruby-2.4.9 - #downloading rubygems-3.0.8
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  867k  100  867k    0     0   154k      0  0:00:05  0:00:05 --:--:--  154k
No checksum for downloaded archive, recording checksum in user configuration.
ruby-2.4.9 - #extracting rubygems-3.0.8.....
ruby-2.4.9 - #removing old rubygems........
ruby-2.4.9 - #installing rubygems-3.0.8................................................................
ruby-2.4.9 - #gemset created /usr/local/rvm/gems/ruby-2.4.9@global
ruby-2.4.9 - #importing gemset /usr/local/rvm/gemsets/global.gems................................................................
ruby-2.4.9 - #generating global wrappers.......
ruby-2.4.9 - #gemset created /usr/local/rvm/gems/ruby-2.4.9
ruby-2.4.9 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.4.9 - #generating default wrappers.......
ruby-2.4.9 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.4.9 - #complete 
Ruby was built without documentation, to build it run: rvm docs generate-ri
[root@lnmp software]# ruby -v
ruby 2.4.9p362 (2019-10-02 revision 67824) [x86_64-linux]

安装redis-dump

  [root@lnmp software]# gem install redis-dump -V

### redis-dump使用

命令格式

 redis-dump -u 源库地址:端口 -a 密码 -d 库 > 备份文件.json
< 备份文.json redis-load -u 目标库地址:端口 -a 目标库密码

导出导入示例

   [root@lnmp software]# redis-dump -u :password@127.0.0.1:6379 > 127.0.0.1.json # redis有密码的导出
   [root@lnmp software]# redis-dump 127.0.0.1:6379 > 127.0.0.1.json # redis无密码的导出
   [root@lnmp software]# vim 127.0.0.1.json  # 查看导出数据
   {"db":0,"key":"test","ttl":-1,"type":"string","value":"hello","size":5}
   {"db":0,"key":"nginx","ttl":-1,"type":"string","value":"openresty","size":9}
   [root@lnmp software]#  cat 127.0.0.1.json | redis-load 127.0.0.1:6379

对比一下redis中的数据,看导出的是否有缺失。

[root@lnmp software]# redis-cli 
127.0.0.1:6379> select 0
OK
127.0.0.1:6379> get test
"hello"
127.0.0.1:6379> get nginx
"openresty"
127.0.0.1:6379> 

aof

什么是Redis的AOF?

AOF是AppendOnly File的缩写,是Redis系统提供了一种记录Redis操作的持久化方案,在AOF生成的文件中,将忠实记录发生在Redis的操作,从而达到在Redis服务器重启或者当机之后,继续恢复之前数据状态的机制。简单的理解等同于mysql的binlog。

Redis中主要的AOF设置

appendonly yes :打开aof设置,同时将快照功能置于低优先级的位置。

appendfsync no :当设置appendfsync为no的时候,Redis不会主动调用fsync去将AOF日志内容同步到磁盘,所以这一切就完全依赖于操作系统的调试了。对大多数Linux操作系统,是每30秒进行一次fsync,将缓冲区中的数据写到磁盘上。

appendfsync everysec :当设置appendfsync为everysec的时候,Redis会默认每隔一秒进行一次fsync调用,将缓冲区中的数据写到磁盘。但是当这一次的fsync调用时长超过1秒时。Redis会采取延迟fsync的策略,再等一秒钟。也就是在两秒后再进行fsync,这一次的fsync就不管会执行多 长时间都会进行。这时候由于在fsync时文件描述符会被阻塞,所以当前的写操作就会阻塞。
    
在绝大多数情况下,Redis会每隔一秒进行一 次fsync。在最坏的情况下,两秒钟会进行一次fsync操作。这一操作在大多数数据库系统中被称为group commit,就是组合多次写操作的数据,一次性将日志写到磁盘。
appendfsync always: 设置appendfsync为always时,每一次写操作都会调用一次fsync,这时数据是最安全的,当然,由于每次都会执行fsync,所以其性能也会受到影响。

测试AOF备份还原

[root@lnmp software]# redis-cli -h 127.0.0.1 config set appendonly yes # 源示例开启aof
OK

开启appendonly之后,就会在配置文件的地方生成一个aof文件
如果不知道配置文件在哪里,可以使用下面命令查看:

[root@lnmp software]# redis-cli 
127.0.0.1:6379> config get dir
1) "dir"
2) "/"
[root@lnmp software]# ls /
appendonly.aof  boot  dev       etc   lib    media  opt   root  sbin  sys  usr
bin             data  dump.rdb  home  lib64  mnt    proc  run   srv   tmp  var
[root@lnmp software]# redis-cli -h 127.0.0.1 --pipe < /appendonly.aof
[root@lnmp software]# redis-cli -h 127.0.0.1 config set appendonly no # 源示例关闭aof

rdb

Redis 是内存数据库,如果不将内存中的数据库状态保存到磁盘,那么一旦服务器进程退出,服务器中的数据库状态也会消失。所以 Redis 提供了持久化功能

RDB 持久化

RDB 持久化既可以手动执行,也可以根据服务器配置选项定期执行,该功能可以将某个时间点上的数据库状态保存到一个 RDB 文件中
RDB 持久化功能所生成的 RDB 文件是一个经过压缩的二进制文件,通过该文件可以还原生成 RDB 文件时的数据库状态

RDB文件的创建与载入

有两个 Redis 命令可以用于生成 RDB 文件,一个是 SAVE,另一个是 BGSAVE
SAVE 命令会阻塞 Redis 服务器进程,直到 RDB 文件创建完毕为止,在服务器进程阻塞期间,服务器不能处理任何命令请求.
BGSAVE 命令会派生出一个子进程,然后由子进程负责创建 RDB 文件,父进程继续处理命令请求

RDB 文件的载入工作是在服务启动时自动执行的,所以 Redis 并没有专门用于载入 RDB 文件的命令,只要 Redis 服务器在启动时检测到 RDB 文件存在,它就会自动载入 RDB 文件

因为 AOF 文件的更新频率通常比 RDB 文件的更新频率高,所以如果服务器开启了 AOF 持久化功能,那么服务器会优先使用 AOF 文件来还原数据库状态,只有在 AOF 持久化功能处于关闭状态时,服务器才会使用 RDB 文件来还原数据库状态

RDB 文件载入时的服务器状态:服务器在载入RDB 文件期间,会一直处于阻塞状态,直到载入工作完成为止

其实你大可以把rdb文件看成是数据文件,里面保存着redis中的数据。不太恰当的比较,相当于mysql下面的数据库文件吧。

rdb备份还原

[root@lnmp /]# redis-cli 
127.0.0.1:6379> save # save会阻塞进程,不建议在生产使用.
OK
127.0.0.1:6379> config get dir
1) "dir"
2) "/"

save完成后,我们进入到 / 下面查看会看到一个dump.rdb的文件。这样数据就保存完成了,接下来我们进行导入。
首先我们进入到目标服务器,先把当前的redis数据save一下,防止意外。然后停止掉redis的进程。

[root@lnmp /]#  systemctl stop redis

然后我们将备份的dump.rdb文件替换掉目标服务器的dump.rdb文件。之后启动redis服务即可。

因为我们在redis.conf配置文件中有定义save自动保存选项,所以当触发到任意一条规则的时候就会执行bgsave。

[root@lnmp /]# grep -vE '#|^$' /usr/local/redis/etc/redis.conf 
....
save 900 1 # 服务器在 900 秒之内,对数据库进行了至少 1 次修改
save 300 10 # 服务器在 300 秒之内,对数据库进行了至少 10 次修改
save 60 10000 # 服务器在 60 秒之内,对数据库进行了至少 10000 次修改
...

总结

综上所述,三种备份(导入导出)方法各有所长,大家可以根据自身的场景选择。我这边用的比较多的是redis-dump方式,其次是用aof。rdb方式我一般会在迁移redis的时候使用(当前服务器-> 一台全新的redis服务器)。

参考资料


添加新评论