kris/puppet-rsnapshot
1
0
Fork 0
mirror of https://github.com/krislamo/puppet-rsnapshot synced 2025-04-19 02:54:22 +00:00
Code Issues Releases Wiki Activity

Compare commits

base: kris:master
Branches Tags
kris:master
kris:pr/14
kris:1.2.4
kris:1.2.2
kris:1.2.1
kris:1.2.0
kris:1.1.1
kris:1.1.0
kris:1.0.3
kris:1.0.2
kris:1.0.1
kris:1.0.0
kris:0.5.2
kris:update_docs
kris:0.5.1
kris:0.5.0
kris:0.4.2
kris:0.4.1
kris:0.4.0
kris:0.3.2
kris:0.3.1
..
compare: kris:0.4.0
Branches Tags
kris:master
kris:pr/14
kris:1.2.4
kris:1.2.2
kris:1.2.1
kris:1.2.0
kris:1.1.1
kris:1.1.0
kris:1.0.3
kris:1.0.2
kris:1.0.1
kris:1.0.0
kris:0.5.2
kris:update_docs
kris:0.5.1
kris:0.5.0
kris:0.4.2
kris:0.4.1
kris:0.4.0
kris:0.3.2
kris:0.3.1

No commits in common. "master" and "0.4.0" have entirely different histories.
master ... 0.4.0

23 changed files with 220 additions and 671 deletions
Show Stats Expand all files Collapse all files

3
.bundle/config Normal file
View File

@ -0,0 +1,3 @@
---
BUNDLE_PATH: vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'

6
.fixtures.yml
View File

@ -1,10 +1,8 @@
fixtures:
forge_modules:
stdlib:
stdlib:
repo: "puppetlabs/stdlib"
epel:
epel:
repo: "stahnma/epel"
concat:
repo: "puppetlabs/concat"
symlinks:
"rsnapshot": "#{source_dir}"

5
.gitignore vendored
View File

@ -1,5 +0,0 @@
.bundle
Gemfile.lock
pkg/
vendor/
log/

13
.travis.yml
View File

@ -2,20 +2,11 @@
sudo: false
language: ruby
cache: bundler
bundler_args: --without system_tests
script: "bundle exec rake validate && bundle exec rake spec SPEC_OPTS='--format documentation'"
matrix:
fast_finish: true
include:
- rvm: 2.1
env: PUPPET_GEM_VERSION="~> 4.0"
script: "bundle exec rake validate"
- rvm: 2.1
env: PUPPET_GEM_VERSION="~> 4.0"
script: "bundle exec rake spec SPEC_OPTS='--format documentation'"
- rvm: 2.1
env: PUPPET_GEM_VERSION="~> 4.0"
script: "bundle exec rake beaker"
services: docker
sudo: required
notifications:
email: false

7
Gemfile
View File

@ -5,10 +5,3 @@ gem 'puppet', puppetversion
gem 'puppetlabs_spec_helper', '>= 0.8.2'
gem 'puppet-lint', '>= 1.0.0'
gem 'facter', '>= 1.7.0'
gem 'metadata-json-lint'
group :acceptance do
gem 'beaker-rspec', '5.6.0'
gem 'serverspec', require: false
gem 'specinfra', require: false
end

414
README.md
View File

@ -1,5 +1,3 @@
[![Build Status](https://travis-ci.org/loomsen/puppet-rsnapshot.svg?branch=master)](https://travis-ci.org/loomsen/puppet-rsnapshot)
# rsnapshot
#### Table of Contents
@ -42,7 +40,6 @@ This module is best used with an ENC like hiera. It will make your config much e
* This module will install the rsnapshot package on your system
* This module will manage the rsnapshot config on your system
* This module will manage cron entries for your configured nodes
* This module will manage the cron service on your system
### Setup Requirements
@ -134,14 +131,10 @@ rsnapshot::hosts:
backup:
'/var': './var'
cron:
mailto: 'bar1@example.com'
daily:
minute: '0-10'
hour: '1..5'
db1:
backup_scripts:
mysql:
misc:
'daily':
'minute': '0-10'
'hour': '1..5'
```
@ -151,13 +144,11 @@ rsnapshot::hosts:
The defaults are pretty reasonable, I hope. However, you may override pretty much anything. Available parameters are discussed below.
#### Specials
This module will generate random time entries for your hosts. The random number generator is hashed with hostname and backup_level, so the randomness will be repeatable per host.level. This is important so puppet won't override the crons with each run.
As mentioned, this module will generate random time entries for your hosts. The random number generator is hashed with hostname and backup_level, so the randomness will be repeatable per host.level. This is important so puppet won't override the crons with each run.
You may specify time ranges as follows:
* default cron syntax
* an array with allowed values
- for example, if you want the backup for a host to run between 1am and 5am, you would override the hours setting for the host in question.
In hiera this would look like: (Explanation see below)
* an array with allowed values, for example, if you want the backup for a host to run between 1am and 5am, you would override the hours setting for the host in question.
in hiera this would look like: (Explanation see below)
```yaml
rsnapshot::hosts:
@ -208,35 +199,49 @@ Takes an Integer, a String or an Array as input, and returns a random entry from
The following parameters are available in the `::rsnapshot` class:
#### `$backup_defaults`
Boolean. Backup default backup dirs or not.
(Default: true)
#### `$hosts`
Hash containing the hosts to be backed up and optional overrides per host
(Default: undef (do nothing when no host given))
#### `$conf_d`
The place where the configs will be dropped
(Default: /etc/rsnapshot (will be created if it doesn't exist))
#### `$backup_user`
The user to run the backup scripts as
(Default: root, also the user used for ssh connections, if you change this make sure you have proper key deployed and the user exists in the nodes to be backed up.)
#### `$package_name`
(Default: rsnapshot)
#### `$package_ensure`
(Default: present)
#### `$cron_dir`
Directory to drop the cron files to. Crons will be created per host.
(Default: /etc/cron.d)
#### `$backup_levels`
Array containing the backup levels (hourly, daily, weekly, monthly)
Configure the backup_levels (valid per host and global, so you may either set: rsnapshot::backup_levels for all hosts or override default backup_levels for specific hosts)
(Default: [ 'daily', 'weekly', ] )
#### `$backup_user`
The user to run the backup scripts as
(Default: root, also the user used for ssh connections, if you change this make sure you have proper key deployed and the user exists in the nodes to be backed up.)
#### `$conf_d`
The place where the configs will be dropped
(Default: /etc/rsnapshot (will be created if it doesn't exist))
#### `$backup_defaults`
Boolean. Backup default backup dirs or not.
(Default: true)
#### `$default_backup`
The default backup directories. This will apply to all hosts unless you set [backup_defaults](#backup_defaults) = false
Default is:
```puppet
$default_backup = {
'/etc' => './',
'/home' => './',
}
```
#### `$cron`
Hash. Set time ranges for different backup levels. Each item (minute, hour...) allows for cron notation, an array to pick a random time from and a range to pick a random time from.
The range notation is '$start..$end', so to pick a random hour from 8 pm to 2 am, you could set the hour of your desired backup level to
`[ '20..23','0..2' ]`
For the range feature to work, hours >0 and <10 must not have a preceding zero.
Wrong: `00.09`
Correct: `0..9`
Also, you can set a mailto for each host, or globally now. The settings will be merged bottom to top, so if you override a setting in a hosts cron, it will have precedence over the global setting,
which in turn has precedence over the default.
Example:
```puppet
$cron = {
mailto => 'admin@example.com',
hourly => {
minute => '0..59',
hour => [ '20..23','0..2' ],
@ -249,7 +254,6 @@ Or in hiera:
```yaml
rsnapshot::cron:
mailto: 'admin@example.com'
daily:
minute: '20'
weekly:
@ -261,32 +265,16 @@ rsnapshot::cron:
```yaml
rsnapshot::hosts:
webserver:
cron:
mailto: 'support@example.com'
daily:
hour: [ '20..23','0..2' ]
weekly:
hour: [ '20..23','0..2' ]
webhost:
customervm.provider.com:
backup_user: 'customer'
daily:
hour: [ '20..23','0..2' ]
weekly:
hour: [ '20..23','0..2' ]
```
`webhost`: Mails will go to `admin@example.com` (from the global override).
`webserver`: Mails will go to `support@example.com`.
`customervm.provider.com`: The backup (and thus ssh) user will be `customer@customervm.provider.com`
Hash is of the form:
```puppet
$cron =>{
mailto => param,
$cron =>{
daily => {
minute => param,
hour => param,
@ -304,7 +292,6 @@ Default is:
```puppet
$cron = {
mailto => 'admin@example.com',
hourly => {
minute => '0..59', # random from 0 to 59
hour => '*', # you could also do: ['21..23','0..4','5'],
@ -329,58 +316,13 @@ Default is:
monthly => {
minute => '0..59',
hour => '0..23', # you could also do: ['21..23','0..4','5'],
monthday => '1..28',
monthday => '0..28',
month => '*',
weekday => '*',
},
}
```
#### `$cron_dir`
Directory to drop the cron files to. Crons will be created per host.
(Default: /etc/cron.d)
#### `$cronfile_prefix_use`
Bool. Set this to true if you want your cronfiles to have a prefix.
(Default: false)
#### `$cronfile_prefix`
Optional prefix to add to the cronfiles name. Your files will be named: prefix_hostname
(Default: 'rsnapshot_' only if you set $cronfile_prefix_use = true)
#### `$default_backup`
The default backup directories. This will apply to all hosts unless you set [backup_defaults](#backup_defaults) = false
Default is:
```puppet
$default_backup = {
'/etc' => './',
'/home' => './',
}
```
#### `$hosts`
Hash containing the hosts to be backed up and optional overrides per host
(Default: undef (do nothing when no host given))
#### `$interval`
How many backups of each level to keep.
Default is:
```puppet
$interval = {
'daily' => '7',
'weekly' => '4',
'monthly' => '6',
}
```
#### `$package_ensure`
(Default: present)
#### `$package_name`
(Default: rsnapshot)
#### `$snapshot_root`
global. the directory holding your backups.
(Default: /backup)
@ -404,68 +346,35 @@ You will end up with a structure like:
└── weekly.0
```
#### `$interval`
How many backups of each level to keep.
Default is:
```puppet
$interval = {
'daily' => '7',
'weekly' => '4',
'monthly' => '6',
}
```
#### `$backup_scripts`
Additional scripts to create, possible values are: mysql, psql, misc
`mysql`: used for mysql backups
`psql`: used for postgresql backups
`misc`: custom commands to run on the node
You can set
`$dbbackup_user`: backup user
`$dbbackup_password`: password for the backup user
`$dumper`: path to the dump bin you wish to use
`$dump_flags`: flags for your dump bin
`$ignore_dbs`: databases to be ignored (the psql script ignores template and postgres databases by default)
`$commands`: array of commands to run on the host (this has no effect on psql and mysql scripts and is intended for your custom needs, see misc script section)
See below for defaults
NOTE: the psql and mysql scripts will SSH into your host and try and use $dumper.
Make sure you have those tools installed on your DB hosts.
Also, this module will try and use pbzip to compress your databases. You can install pbzip2 (and additional packages you might need) by passing an array to [$rsnapshot::package_name](#package_name)
Default is:
```puppet
$backup_scripts = {
mysql => {
mysql => {
dbbackup_user => 'root',
dbbackup_password => '',
dumper => 'mysqldump',
dump_flags => '--single-transaction --quick --routines --ignore-table=mysql.event',
ignore_dbs => [ 'information_schema', 'performance_schema' ],
dbbackup_password => 'myFancyPassWord',
},
psql => {
dbbackup_user => 'postgres',
dbbackup_password => '',
dumper => 'pg_dump',
dump_flags => '-Fc',
ignore_dbs => [ 'postgres' ],
},
misc => {
commands => $::osfamily ? {
'RedHat' => [
'rpm -qa --qf="%{name}," > packages.txt',
],
'Debian' => [
'dpkg --get-selections > packages.txt',
],
default => [],
},
}
misc => {},
}
```
Configuration example:
@ -484,134 +393,16 @@ rsnapshot::hosts:
backup_scripts:
mysql:
psql:
dumper: '/usr/local/bin/pg_dump'
dump_flags: '-Fc'
ignore_dbs: [ 'db1', 'tmp_db' ]
misc:
bazqux:de:
backup_scripts:
mysql:
dbbackup_user: 'myuser'
dbbackup_password: 'mypassword'
misc:
commands:
- 'cat /etc/hostname > hostname.txt'
- 'date > date.txt'
```
This creates
- a mysql and a psql backup script for `foobar.com` using the credentials `dbbackup:hunter2` for mysql and `dbbackup:yeshorsebatterystaple` for psql
- the psql script will use `/usr/local/bin/pg_dump` as the dump program with flags `-Fc`
- it will ignore the postgres databases `db1` and `tmp_db` for postgres
- a mysql backup script for `bazqux.de` using the credentials `myuser:mypassword`
- a misc script for bazqux.de containing two commands to run on the node. the output will be redirected to hostname.txt and date.txt in the misc/ subfolder of the hosts backup directory (i.e. /snapshot_root/bazqux.de/daily.0/misc/hostname.txt)
The scripts look like this:
##### `bazqux.de`
```bash
#!/bin/bash
host=bazqux.de
user=myuser
pass=mypassword
dbs=(
$(ssh -l root "$host" "mysql -u ${user} -p${pass} -e 'show databases' | sed '1d;/information_schema/d;/performance_schema/d'")
)
for db in "${dbs[@]}"; do
ssh -l root "$host" "mysqldump --user=${user} --password=${pass} --single-transaction --quick --routines --ignore-table=mysql.event ${db}" > "${db}.sql"
wait
pbzip2 "$db".sql
done
```
```bash
#!/bin/bash
ssh bazqux.de 'cat /etc/hostname > hostname.txt'
ssh bazqux.de 'date > date.txt'
```
##### `foobar.com`
psql:
```bash
#!/bin/bash
host=foobar.com
user=dbbackup
pass=yeshorsebatterystaple
PGPASSWORD="$pass"
dbs=(
$(ssh -l root "$host" "psql -U ${user} -Atc \"SELECT datname FROM pg_database WHERE NOT datistemplate AND datname ~ 'postgres|db1|tmp_db'\"" )
)
for db in "${dbs[@]}"; do
ssh -l root "$host" "pg_dump -U ${user} -Fc ${db}" > "$db".sql
wait
pbzip2 "$db".sql
done
```
mysql:
```bash
#!/bin/bash
host=foobar.com
user=dbbackup
pass=hunter2
dbs=(
$(ssh -l root "$host" "mysql -u ${user} -p${pass} -e 'show databases' | sed '1d;/information_schema/d;/performance_schema/d'")
)
for db in "${dbs[@]}"; do
ssh -l root "$host" "mysqldump --user=${user} --password=${pass} --single-transaction --quick --routines --ignore-table=mysql.event ${db}" > "${db}.sql"
wait
pbzip2 "$db".sql
done
```
misc (assuming foobar.com is a RedHat node):
```bash
#!/bin/bash
ssh foobar.com 'rpm -qa --qf "%{name}," > packages.txt'
```
##### another example with root user and empty password
mysql with root user:
```bash
#!/bin/bash
host=bazqux.de
user=root
password=
dbs=(
$(ssh -l root "$host" "mysql -e 'show databases' | sed '1d;/information_schema/d;/performance_schema/d'")
)
for db in "${dbs[@]}"; do
ssh -l root "$host" "mysqldump --single-transaction --quick --routines --ignore-table=mysql.event ${db}" > "${db}.sql"
wait
pbzip2 "$db".sql
done
```
### rsnapshot configuration variables
@ -647,25 +438,9 @@ Default is: undef
#### `$cmd_postexec`
Default is: undef
#### `$du_args`
#### `$use_lvm`
Default is: undef
#### `$exclude`
Default is: []
#### `$exclude_file`
Other than this might suggest, the default behavior is to create an exclude file per host.
Default is: undef
#### `$include`
Default is: []
#### `$include_file`
Default is: undef
#### `$link_dest`
Default is: false
#### `$linux_lvm_cmd_lvcreate`
Default is: undef # '/sbin/lvcreate'
@ -691,9 +466,6 @@ Default is: undef
#### `$linux_lvm_mountpath`
Default is: undef
#### `$lockpath`
Default is: '/var/run/rsnapshot'
#### `$logpath`
Default is: '/var/log/rsnapshot'
@ -701,22 +473,26 @@ Default is: '/var/log/rsnapshot'
unused, we are logging to $logpath/$host.log
Default is: '/var/log/rsnapshot.log'
#### `$loglevel`
Default is: '4'
#### `$lockpath`
Default is: '/var/run/rsnapshot'
#### `$manage_cron`
Should this module manage the cron service?
Default is: true
#### `$snapshot_root`
Default is: '/backup/'
#### `$no_create_root`
Boolean: true or false
Default is: undef
#### `$one_fs`
Default is: undef
#### `$verbose`
Default is: '2'
#### `$retain`
Default is: { }
#### `$loglevel`
Default is: '4'
#### `$stop_on_stale_lockfile`
Boolean: true or false
Default is: undef
#### `$rsync_short_args`
Default is: '-az'
@ -725,31 +501,45 @@ Default is: '-az'
rsync defaults are: --delete --numeric-ids --relative --delete-excluded
Default is: undef
#### `$rsync_numtries`
Default is: 1
#### `$snapshot_root`
Default is: '/backup/'
#### `$ssh_args`
Default is: undef
#### `$stop_on_stale_lockfile`
Boolean: true or false
#### `$du_args`
Default is: undef
#### `$one_fs`
Default is: undef
#### `$retain`
Default is: { }
#### `$include`
Default is: []
#### `$exclude`
Default is: []
#### `$include_file`
Default is: undef
#### `$exclude_file`
Other than this might suggest, the default behavior is to create an exclude file per host.
Default is: undef
#### `$link_dest`
Default is: false
#### `$sync_first`
Default is: false
#### `$use_lvm`
Default is: undef
#### `$rsync_numtries`
Default is: 1
#### `$use_lazy_deletes`
Default is: false
#### `$verbose`
Default is: '2'
## Limitations
Currently, this module support CentOS, Fedora, Ubuntu and Debian.
@ -761,5 +551,5 @@ want it off the face of the planet, feel free to get in touch with me.
Norbert Varzariu (loomsen)
## Contributors
Please see the [list of contributors.](https://github.com/loomsen/puppet-rsnapshot/graphs/contributors)
A big thank you to Hendrik Horeis <hendrik.horeis@gmail.com> for all his input and testing of this module.
Please see the [list of contributors.](https://github.com/loomsen/puppet-bloonix_agent/graphs/contributors)

155
manifests/config.pp
View File

@ -2,18 +2,15 @@
#
# manage host configs
class rsnapshot::config (
$hosts = $rsnapshot::hosts,
$cron_dir = $rsnapshot::cron_dir,
$hosts = $rsnapshot::hosts,
$cron_dir = $rsnapshot::cron_dir,
) {
# these are global settings, no point in setting them per host
$config_version = $rsnapshot::params::config_version
$lockpath = pick($rsnapshot::lockpath, $rsnapshot::params::config_lockpath, '/var/run/rsnapshot')
$conf_d = pick($rsnapshot::conf_d, $rsnapshot::params::conf_d, '/etc/rsnapshot')
$snapshot_root = pick($hosts['snapshot_root'], $rsnapshot::snapshot_root, '/backup')
$logpath = pick($rsnapshot::logpath, $rsnapshot::params::config_logpath)
$cronfile_prefix_use = pick($rsnapshot::cronfile_prefix_use, $rsnapshot::params::config_cronfile_prefix_use, false)
$cronfile_prefix = pick($rsnapshot::cronfile_prefix, $rsnapshot::params::config_cronfile_prefix, '')
$default_cron = assert_empty_hash($::rsnapshot::cron)
# make sure lock path and conf path exist
file { $conf_d:
ensure => 'directory',
@ -27,20 +24,16 @@ class rsnapshot::config (
file { $snapshot_root:
ensure => 'directory',
}
file { $logpath:
ensure => directory,
}
# custom function, if only a hostname is given as a param, this is an empty hash
# the next loop would break as puppet does not allow to reassign variables
# the function checks $hosts for elements like:
# the function checks $hosts for elements like:
# { foo => } and converts those to { foo => {} }
$hosts_clean = assert_empty_hash($hosts)
$hosts_clean.each |String $host, $hash | {
$backup_user = pick($hash['backup_user'], $rsnapshot::backup_user, $rsnapshot::params::config_backup_user, 'root')
$backup_user = pick($hash['backup_user'], $rsnapshot::params::config_backup_user)
$default_backup_dirs = pick($rsnapshot::default_backup, $rsnapshot::params::config_default_backup)
$backup_levels = pick($hash['backup_levels'], $rsnapshot::backup_levels, 'weekly')
$backup_levels = pick($hash['backup_levels'], $rsnapshot::params::config_backup_levels, 'weekly')
$backup = $hash['backup']
$backup_defaults = pick($hash['backup_defaults'], $rsnapshot::params::config_backup_defaults)
$cmd_cp = pick($hash['cmd_cp'], $rsnapshot::params::config_cmd_cp)
@ -62,6 +55,7 @@ class rsnapshot::config (
$linux_lvm_vgpath = pick_undef($hash['linux_lvm_vgpath'], $rsnapshot::params::config_linux_lvm_vgpath)
$linux_lvm_mountpath = pick_undef($hash['linux_lvm_mountpath'], $rsnapshot::params::config_linux_lvm_mountpath)
$no_create_root = pick_undef($hash['no_create_root'], $rsnapshot::params::config_no_create_root)
$logpath = pick($hash['logpath'], $rsnapshot::logpath, $rsnapshot::params::config_logpath)
$verbose = pick($hash['verbose'], $rsnapshot::params::config_verbose)
$loglevel = pick($hash['loglevel'], $rsnapshot::params::config_loglevel)
$stop_on_stale_lockfile = pick_undef($hash['stop_on_stale_lockfile'], $rsnapshot::params::config_stop_on_stale_lockfile)
@ -72,9 +66,9 @@ class rsnapshot::config (
$one_fs = pick_undef($hash['one_fs'], $rsnapshot::params::config_one_fs)
$interval = pick($hash['interval'], $rsnapshot::params::config_interval)
$retain = pick_undef($hash['retain'], $rsnapshot::params::config_retain)
$include = pick($hash['include'], [])
$exclude = pick($hash['exclude'], [])
$include_file = pick($hash['include_file'], $rsnapshot::params::config_include_file, "${conf_d}/${host}.include")
$include = pick_undef($hash['include'], $rsnapshot::params::config_include)
$exclude = pick_undef($hash['exclude'], $rsnapshot::params::config_exclude)
$include_file = pick_undef($hash['include_file'], $rsnapshot::params::config_include_file)
$exclude_file = pick($hash['exclude_file'], $rsnapshot::params::config_exclude_file, "${conf_d}/${host}.exclude")
$link_dest = pick_undef($hash['link_dest'], $rsnapshot::params::config_link_dest)
$sync_first = pick_undef($hash['sync_first'], $rsnapshot::params::config_sync_first)
@ -82,7 +76,7 @@ class rsnapshot::config (
$rsync_numtries = pick_undef($hash['rsync_numtries'], $rsnapshot::params::config_rsync_numtries)
#$backup_scripts = pick_undef($hash['backup_scripts'], $rsnapshot::params::config_backup_scripts)
$snapshot_dir = "${snapshot_root}/${host}"
$snapshot_dir = "${config_snapshot_root}/${host}"
$config = "${conf_d}/${host}.rsnapshot.conf"
$lockfile = "${lockpath}/${host}.pid"
$logfile = "${logpath}/${host}.log"
@ -103,7 +97,6 @@ class rsnapshot::config (
if ! ( $interval and $retain ) {
$interval = pick($hash['interval'], $rsnapshot::params::config_interval)
}
# rsnapshot wants numeric values
if $link_dest {
$link_dest_num = bool2num($link_dest)
@ -114,26 +107,20 @@ class rsnapshot::config (
if $use_lazy_deletes {
$use_lazy_deletes_num = bool2num($use_lazy_deletes)
}
if $one_fs {
$one_fs_num = bool2num($one_fs)
}
$real_include = $rsnapshot::include + $include
unless empty($real_include) {
if $include != '' {
file { $include_file:
ensure => 'file',
content => template('rsnapshot/include.erb'),
}
}
$real_exclude = $rsnapshot::exclude + $exclude
unless empty($real_exclude) {
if $exclude != '' {
file { $exclude_file:
ensure => 'file',
content => template('rsnapshot/exclude.erb'),
}
}
concat { $config:
}
concat::fragment { "${config} for ${host}":
@ -141,22 +128,21 @@ class rsnapshot::config (
content => template('rsnapshot/rsnapshot.erb'),
}
if has_key($hash, backup_scripts) {
$hash[backup_scripts].each |$script, $scriptconf| {
$real_script = deep_merge($rsnapshot::params::backup_scripts[$script], $rsnapshot::backup_scripts[$script], $hash[backup_scripts][$script])
$dbbackup_user = $real_script[dbbackup_user]
$dbbackup_password = $real_script[dbbackup_password]
$dumper = $real_script[dumper]
$dump_flags = $real_script[dump_flags]
$ignore_dbs = $real_script[ignore_dbs]
$compress = $real_script[compress]
$commands = $real_script[commands]
$hash[backup_scripts].each |$script, $credentials| {
if is_hash($credentials) {
$dbbackup_user = $credentials['dbbackup_user']
$dbbackup_password = $credentials['dbbackup_password']
} else {
$dbbackup_user = $rsnapshot::default_backup_scripts[$script]['dbbackup_user']
$dbbackup_password = $rsnapshot::default_backup_scripts[$script]['dbbackup_password']
}
concat::fragment { "${host}_${script}_backup":
target => $config,
content => "backup_script ${conf_d}/${host}.${script}.sh ./${script}\n",
target => $config,
content => "backup_script ${conf_d}/${host}.${script}.sh ./${script}\n",
}
file { "${conf_d}/${host}.${script}.sh":
@ -164,55 +150,70 @@ class rsnapshot::config (
content => template("rsnapshot/${script}.sh.erb"),
mode => '0755',
}
}
}
if $cronfile_prefix_use {
$rsnapshot_prefix = $rsnapshot::cronfile_prefix
} else {
$rsnapshot_prefix = ''
}
# cron on Debian seems to ignore files that have dots in their name; replace
# them with underscores (issue #2)
case $::osfamily {
'Debian': {
$cron_name = regsubst($host, '\.', '_', 'G')
$cronfile = "${cron_dir}/${rsnapshot_prefix}${cron_name}"
}
'RedHat': {
$cronfile = "${cron_dir}/${rsnapshot_prefix}${host}"
}
default: {
$cronfile = "${cron_dir}/${rsnapshot_prefix}${host}"
}
}
$cronfile = "${cron_dir}/${host}"
concat { $cronfile:
}
# create cron files for each backup level
# merge possible cron definitions to one
$real_cron = deep_merge($rsnapshot::params::cron, $rsnapshot::cron, $hash[cron])
concat::fragment { "mailto for ${host}":
content => "#This file is managed by puppet\nMAILTO=${real_cron[mailto]}\n\n",
target => $cronfile,
order => 1,
}
$backup_levels.each |String $level| {
$mailto = $real_cron[mailto]
$minute = rand_from_array($real_cron[$level][minute], "${host}.${level}.minute")
$hour = rand_from_array($real_cron[$level][hour], "${host}.${level}.hour")
$monthday = rand_from_array($real_cron[$level][monthday], "${host}.${level}.monthday")
$month = rand_from_array($real_cron[$level][month], "${host}.${level}.month")
$weekday = rand_from_array($real_cron[$level][weekday], "${host}.${level}.weekday")
# allow to globally override ranges, create random numbers for backup_levels daily, weekly, monthly
if has_key($hash,cron){
if has_key($hash[cron], $level) {
$cron = $hash[cron][$level]
} else {
if has_key($default_cron, $level) {
$cron = $default_cron[$level]
} else {
$cron = $rsnapshot::params::cron[$level]
}
}
} else {
if has_key($default_cron, $level) {
$cron = $default_cron[$level]
} else {
$cron = $rsnapshot::params::cron[$level]
}
}
if has_key($cron, minute) {
$c_min = $cron[minute]
} else {
$c_min = $rsnapshot::params::cron[$level][minute]
}
if has_key($cron, hour) {
$c_hour = $cron[hour]
} else {
$c_hour = $rsnapshot::params::cron[$level][hour]
}
if has_key($cron, monthday) {
$c_monthday = $cron[monthday]
} else {
$c_monthday = $rsnapshot::params::cron[$level][monthday]
}
if has_key($cron, month) {
$c_month = $cron[month]
} else {
$c_month = $rsnapshot::params::cron[$level][month]
}
if has_key($cron, weekday) {
$c_weekday = $cron[weekday]
} else {
$c_weekday = $rsnapshot::params::cron[$level][weekday]
}
$minute = rand_from_array($c_min, "${host}.${level}.minute")
$hour = rand_from_array($c_hour, "${host}.${level}.hour")
$monthday = rand_from_array($c_monthday, "${host}.${level}.monthday")
$month = rand_from_array($c_month, "${host}.${level}.month")
$weekday = rand_from_array($c_weekday, "${host}.${level}.weekday")
concat::fragment { "${host}.${level}":
target => $cronfile,
content => template('rsnapshot/cron.erb'),
order => 2,
}
}
}
}

36
manifests/init.pp
View File

@ -5,36 +5,26 @@
# === Parameters
#
class rsnapshot (
$hosts = $rsnapshot::params::hosts,
$conf_d = $rsnapshot::params::conf_d,
$logpath = $rsnapshot::params::config_logpath,
$lockpath = $rsnapshot::params::config_lockpath,
$default_backup = $rsnapshot::params::config_default_backup,
$package_name = $rsnapshot::params::package_name,
$package_ensure = $rsnapshot::params::package_ensure,
$cron = $rsnapshot::params::cron,
$backup_scripts = $rsnapshot::params::backup_scripts,
$include = $rsnapshot::params::config_include,
$exclude = $rsnapshot::params::config_exclude,
$snapshot_root = $rsnapshot::params::config_snapshot_root,
$backup_levels = $rsnapshot::params::config_backup_levels,
$backup_user = $rsnapshot::params::config_backup_user,
$cron_service_name = $rsnapshot::params::cron_service_name,
$manage_cron = $rsnapshot::params::manage_cron,
$cronfile_prefix = $rsnapshot::params::config_cronfile_prefix,
$cronfile_prefix_use = $rsnapshot::params::config_cronfile_prefix_use,
$check_mk_job = $rsnapshot::params::config_check_mk_job,
$hosts = $rsnapshot::params::hosts,
$conf_d = $rsnapshot::params::conf_d,
$logpath = $rsnapshot::params::config_logpath,
$lockpath = $rsnapshot::params::config_lockpath,
$default_backup = $rsnapshot::params::config_default_backup,
$package_name = $rsnapshot::params::package_name,
$package_ensure = $rsnapshot::params::package_ensure,
$cron = $rsnapshot::params::cron,
$backup_scripts = $rsnapshot::params::backup_scripts,
$exclude = $rsnapshot::params::config_exclude,
$snapshot_root = $rsnapshot::params::config_snapshot_root,
) inherits rsnapshot::params {
$default_backup_scripts = $rsnapshot::params::backup_scripts + $backup_scripts
$default_exclude = $rsnapshot::params::config_exclude + $exclude
if $hosts {
class { '::rsnapshot::install': }->
class { '::rsnapshot::config': }~>
class { '::rsnapshot::service': }
class { '::rsnapshot::config': }
contain '::rsnapshot::install'
contain '::rsnapshot::config'
contain '::rsnapshot::service'
}
}

13
manifests/install.pp
View File

@ -10,18 +10,5 @@ class rsnapshot::install {
ensure => $rsnapshot::package_ensure,
}
# ensure run directory exists (systemd clears /var/run, so rsnapshot can't create PID files after reboot - issue#12)
$lockpath = pick($rsnapshot::lockpath, $rsnapshot::params::config_lockpath, '/var/run/rsnapshot')
$tmpfiles_d = '/etc/tmpfiles.d'
file { "${tmpfiles_d}":
ensure => directory,
}
file { "${tmpfiles_d}/rsnapshot.conf":
ensure => present,
content => "D ${lockpath} 0755 root root -",
}
}

40
manifests/params.pp
View File

@ -7,17 +7,10 @@ class rsnapshot::params {
$config_backup_user = 'root'
$package_name = 'rsnapshot'
$package_ensure = 'present'
$cron_service_name = $::osfamily ? {
'RedHat' => 'crond',
'Debian' => 'cron',
default => '',
}
$manage_cron = true
$cron_dir = '/etc/cron.d'
$config_backup_levels = [ 'daily', 'weekly', 'monthly' ]
$config_backup_defaults = true
$config_version = '1.2'
$config_check_mk_job = false
$config_cmd_cp = '/bin/cp'
$config_cmd_rm = '/bin/rm'
$config_cmd_rsync = '/usr/bin/rsync'
@ -27,8 +20,6 @@ class rsnapshot::params {
$config_cmd_rsnapshot_diff = '/usr/bin/rsnapshot-diff'
$config_cmd_preexec = undef
$config_cmd_postexec = undef
$config_cronfile_prefix = 'rsnapshot_'
$config_cronfile_prefix_use = false
$config_use_lvm = undef
$config_linux_lvm_cmd_lvcreate = undef # '/sbin/lvcreate'
$config_linux_lvm_cmd_lvremove = undef # '/sbin/lvremove'
@ -41,13 +32,13 @@ class rsnapshot::params {
$config_logpath = '/var/log/rsnapshot'
$config_logfile = '/var/log/rsnapshot.log' # unused, we are logging to $logpath/$host.log
$config_lockpath = '/var/run/rsnapshot'
$config_snapshot_root = '/backup'
$config_snapshot_root = '/backup/'
$config_no_create_root = undef # bool, true or false
$config_verbose = '2'
$config_loglevel = '4'
$config_stop_on_stale_lockfile = undef # bool
$config_rsync_short_args = '-az'
$config_rsync_long_args = undef # defaults are --delete --numeric-ids --relative --delete-excluded
$config_rsync_long_args = undef # defaults are --delete --numeric-ids --relative --delete-excluded
$config_ssh_args = undef
$config_du_args = undef
$config_one_fs = undef
@ -71,7 +62,6 @@ class rsnapshot::params {
}
$config_backup_scripts = {}
$cron = {
mailto => 'admin@example.com',
hourly => {
minute => '0..59',
hour => '*', # you could also do: ['21..23','0..4','5'],
@ -96,38 +86,20 @@ class rsnapshot::params {
monthly => {
minute => '0..59',
hour => '0..23', # you could also do: ['21..23','0..4','5'],
monthday => '1..28',
monthday => '0..28',
month => '*',
weekday => '*',
},
}
$backup_scripts = {
mysql => {
mysql => {
dbbackup_user => 'root',
dbbackup_password => '',
dumper => 'mysqldump',
dump_flags => '--single-transaction --quick --routines --ignore-table=mysql.event',
ignore_dbs => [ 'information_schema', 'performance_schema' ],
compress => 'pbzip2',
dbbackup_password => 'myFancyPassWord',
},
psql => {
dbbackup_user => 'postgres',
dbbackup_password => '',
dumper => 'pg_dump',
dump_flags => '-Fc',
ignore_dbs => [ 'postgres' ],
compress => 'pbzip2',
},
misc => {
commands => $::osfamily ? {
'RedHat' => [
'rpm -qa --qf="%{name}," > packages.txt',
],
'Debian' => [
'dpkg --get-selections > packages.txt',
],
default => [],
},
},
misc => {},
}
}

13
manifests/service.pp
View File

@ -1,13 +0,0 @@
# == Class: rsnapshot::service
#
# Reloads cron
class rsnapshot::service (
$manage_cron = $rsnapshot::manage_cron
) {
if $manage_cron {
service { $rsnapshot::cron_service_name:
ensure => running,
}
}
}

15
metadata.json
View File

@ -1,6 +1,6 @@
{
"name": "loomsen-rsnapshot",
"version": "1.2.5",
"version": "0.4.0",
"author": "loomsen",
"summary": "Configures rsnapshot.",
"license": "Apache-2.0",
@ -12,10 +12,13 @@
"backup",
"rsync"
],
"requirements": [
{ "name": "pe", "version_requirement": ">= 4.0.0" },
{ "name": "puppet", "version_requirement": ">= 4.0.0" }
],
"dependencies": [
{ "name": "puppetlabs-stdlib","version_requirement": ">= 1.0.0" },
{ "name": "stahnma-epel","version_requirement": ">= 1.0.0" },
{ "name": "puppetlabs-concat","version_requirement": ">= 2.0.0 < 3.0.0" }
{ "name": "puppetlabs-stdlib","version_range": ">= 1.0.0" },
{ "name": "stahnma-epel","version_range": ">= 1.0.0" }
],
"operatingsystem_support": [
{
@ -28,11 +31,11 @@
},
{
"operatingsystem": "Ubuntu",
"operatingsystemrelease": [ "12.04", "12.10", "13.04", "13.10", "14.04", "14.10", "15.04", "15.10", "16.04", "16.10", "17.04" ]
"operatingsystemrelease": [ "12.04", "10.04", "14.04" ]
},
{
"operatingsystem": "Fedora",
"operatingsystemrelease": [ "17", "18", "19", "20", "21", "22", "23", "24", "25" ]
"operatingsystemrelease": [ "17", "18", "19", "20", "21", "22", "23", "24" ]
}
]
}

63
spec/acceptance/default_spec.rb
View File

@ -1,63 +0,0 @@
require 'spec_helper_acceptance'
describe 'rsnapshot' do
context 'with defaults' do
it 'run idempotently' do
pp = <<-EOS
class { 'rsnapshot':
hosts => {
'localhost' => {},
'example.com' => {
backup_defaults => false,
backup => {
'/var/' => './'
}
}
}
}
EOS
apply_manifest(pp, catch_failures: true)
apply_manifest(pp, catch_changes: true)
end
end
context 'packages installed' do
describe package('rsnapshot') do
it { is_expected.to be_installed }
end
end
context 'files provisioned' do
describe file('/etc/tmpfiles.d') do
it { should be_directory }
end
describe file('/etc/tmpfiles.d/rsnapshot.conf') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'D /var/run/rsnapshot 0755 root root -' }
end
describe file('/var/run/rsnapshot') do
it { should be_directory }
end
describe file('/etc/rsnapshot/localhost.rsnapshot.conf') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'backup' }
end
describe file('/etc/rsnapshot/example.com.rsnapshot.conf') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'backup\troot@example.com:/var/\t./' }
end
describe file('/etc/rsnapshot.conf') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'localhost' }
its(:content) { is_expected.to match 'example.com' }
end
describe file('/etc/rsnapshot/localhost.rsnapshot.conf') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'backup' }
end
describe file('/etc/cron.d/example_com') do
it { is_expected.to exist }
its(:content) { is_expected.to match 'example.com' }
end
end
end

15
spec/acceptance/nodesets/default.yml
View File

@ -1,15 +0,0 @@
HOSTS:
debian-8-x64:
platform: debian-8-amd64
hypervisor: docker
image: debian:8
docker_container_name: puppet_rsnapshot
docker_preserve_image: true
docker_cmd: '["/sbin/init"]'
docker_image_commands:
- 'echo deb http://ftp.debian.org/debian jessie-backports main >> /etc/apt/sources.list'
- 'apt-get update && apt-get install -y cron locales-all net-tools wget'
- 'rm -f /usr/sbin/policy-rc.d'
CONFIG:
trace_limit: 200
masterless: true

2
spec/classes/init_spec.rb
View File

@ -1,7 +1,5 @@
require 'spec_helper'
describe 'rsnapshot' do
let(:facts) {{ :osfamily => 'RedHat' }}
it { should contain_class('rsnapshot::params') }
it { is_expected.to compile }
end

26
spec/spec_helper_acceptance.rb
View File

@ -1,26 +0,0 @@
require 'puppet'
require 'beaker-rspec'
require 'yaml'
install_puppet_agent_on hosts, {}
RSpec.configure do |c|
# Module root and settings
module_root = File.expand_path(File.join(File.dirname(__FILE__), '..'))
module_name = module_root.split('/').last.sub('-', '_').split('_').last()
c.formatter = :documentation
c.max_displayed_failure_line_count = 5
c.before :suite do
puts 'Install module'
puppet_module_install(source: module_root, module_name: module_name)
hosts.each do |host|
puts 'Install fixtures'
on host, puppet('module','install','stahnma/epel')
on host, puppet('module','install','puppetlabs/stdlib')
on host, puppet('module','install','puppetlabs/concat')
end
end
end

2
templates/cron.erb
View File

@ -1 +1 @@
<%= @minute %> <%= @hour %> <%= @monthday %> <%= @month %> <%= @weekday %> root<% if scope['rsnapshot::check_mk_job'] == true -%> mk-job rsnapshot_<%= @host %>_<%= @level %><% end -%> /usr/bin/rsnapshot -c <%= @config %> <%= @level %>
<%= @minute %> <%= @hour %> <%= @month %> <%= @monthday %> <%= @weekday %> root /usr/bin/rsnapshot -c <%= @config %> <%= @level %>

2
templates/exclude.erb
View File

@ -1,3 +1,3 @@
<% @real_exclude.each do |exc| -%>
<% @exclude.each do |exc| -%>
<%= exc %>
<% end %>

2
templates/include.erb
View File

@ -1,3 +1,3 @@
<% @real_include.each do |inc| -%>
<% @include.each do |inc| -%>
<%= inc %>
<% end %>

7
templates/misc.sh.erb
View File

@ -1,7 +0,0 @@
#!/bin/bash
#This file is managed by puppet
#
<%@commands.each do |command| -%>
ssh -l <%=@backup_user-%> <%=@host-%> <%=command-%>
<%end-%>

29
templates/mysql.sh.erb
View File

@ -1,29 +0,0 @@
#!/bin/bash
# This file is managed by puppet
# written by Norbert Varzariu <loomsen@gmail.com>
# This Script is triggered by rsnapshot and dumps the specified dbs on the configured host.
#
host=<%=@host%>
user=<%=@dbbackup_user%>
<% if (@dbbackup_user == 'root' && @dbbackup_password == '') -%>
dbs=(
$(ssh -l <%=@backup_user-%> "$host" "mysql -e 'show databases' | sed '1d;<%@ignore_dbs.to_a.each do |db|-%>/<%=db-%>/d;<%end-%>'" )
)
<%else-%>
dbs=(
$(ssh -l <%=@backup_user-%> "$host" "mysql -u <%=@dbbackup_user-%> -p'<%=@dbbackup_password-%>' -e 'show databases' | sed '1d;<%@ignore_dbs.each do |db|-%>/<%=db-%>/d;<%end-%>'" )
)
<%end-%>
for db in "${dbs[@]}"; do
<% if (@dbbackup_user == 'root' && @dbbackup_password == '' )-%>
ssh -l <%=@backup_user-%> "$host" "<%=@dumper-%> <%=@dump_flags-%> ${db}" > "$db".sql
<%else-%>
ssh -l <%=@backup_user-%> "$host" "<%=@dumper-%> --user=<%=@dbbackup_user-%> --password='<%=@dbbackup_password-%>' <%=@dump_flags-%> ${db}" > "$db".sql
<%end-%>
wait
<% if @compress != '' -%>
<%=@compress-%> "$db".sql
<%end-%>
done

19
templates/psql.sh.erb
View File

@ -1,19 +0,0 @@
#!/bin/bash
# This file is managed by puppet
# written by Norbert Varzariu <loomsen@gmail.com>
# This Script is triggered by rsnapshot and dumps the specified dbs on the configured host.
#
host=<%=@host%>
user=<%=@dbbackup_user%>
dbs=(
$(ssh -l <%=@backup_user-%> "$host" "PGPASSWORD='<%=@dbbackup_password%>' psql -U ${user} -Atc \"SELECT datname FROM pg_database WHERE NOT datistemplate AND NOT datname ~ '<%@ignore_dbs.each do |db|-%><%if db == @ignore_dbs.last-%><%=db-%><%else-%><%=db-%>|<%end-%><%end-%>'\"")
)
for db in "${dbs[@]}"; do
ssh -l <%=@backup_user-%> "$host" "PGPASSWORD='<%=@dbbackup_password%>' <%=@dumper-%> -U ${user} <%=@dump_flags-%> ${db}" > "$db".sql
wait
<% if @compress != '' -%>
<%=@compress-%> "$db".sql
<%end-%>
done

4
templates/rsnapshot.erb
View File

@ -85,12 +85,12 @@ du_args <%= @du_args %>
<% if @one_fs != '' -%>
one_fs <%= @one_fs_num %>
<% end -%>
<% unless @real_include.empty? -%>
<% if @include != '' -%>
<% if @include_file != '' -%>
include_file <%= @include_file %>
<% end -%>
<% end -%>
<% unless @real_exclude.empty? -%>
<% if @exclude != '' -%>
<% if @exclude_file != '' -%>
exclude_file <%= @exclude_file %>
<% end -%>