Bug fixes:
- Final release, no changes since previous alpha. [maurits]
Breaking changes:
- Removed
additional_filestorages
option. Removed support inalternative_restore_sources
for multiple storages: only the standard single filestorage plus blobstorage is supported. Renamedalternative_restore_sources
toalternative_restore_source
, keeping the old alias for now. [maurits] (Issue #64) - Removed
gzip
option. From now on, we always call repozo with the gzip option, you can no longer turn this off. [maurits] (Issue #67) - Removed option
gzip_blob
. This was a backwards compatibility alias forarchive_blob
. You should use that now it you want to create a tar archive. Additionally use thecompress_blob
option if you are really sure you want to compress the archive. [maurits] (Issue #68) - Removed option
quick
. From now on, we always call repozo with the quick option for normal backups, you can no longer turn this off. For full backups (snapshotbackups and zipbackups), the quick option is not used. [maurits] (Issue #69)
Bug fixes:
Bug fixes:
- Add missing changelog entries for release 5.0.0a1. [maurits]
Breaking changes:
- Drop support for Python 2.7-3.7, add support for 3.10. [maurits] (Issue #62)
- No longer support the
enable_fullbackup
option and thefullbackup
script. A full backup is automatically made by the first call ofbin/backup
after you have done a zeopack. If you still want such a script, look atbin/snapshotbackup
script or use the full option, optionally in a second buildout section. [maurits] (Issue #66)
New features:
- Added an option to use hard links for the first backup copy [ale-rt] (Issue #57)
- Add support for Python 3.11. [maurits] (Issue #311)
Bug fixes:
- Drop code-analysis, add other QA checks: isort, black, flake8. [maurits] (Issue #63)
New features:
- Changed default of
blob_timestamps
to true. We might remove the previous behavior (creatingblobstorage.0
,blobstorage.1
etcetera) completely in version 5. [maurits] (Issue #61)
Bug fixes:
- Switch the test infrastructure to GitHub Actions and tox. Also test Python 3.8 and 3.9. [ale-rt, maurits] (Issue #58)
- Fixed
latest
symlink when using optionsblob_timestamps
andno_rsync
. Previously it pointed to for exampleblobstorage.2021-01-02-03-04-05/blobstorage
instead of the directory above it. [maurits] (Issue #61) - When creating a zipbackup, do not create a blob timestamp. We always only keep one zipbackup, so there is no need for timestamps to know which filestorage backup it belongs to. [maurits] (Issue #61)
Bug fixes:
- When using timestamps, use correct filestorage backup location for blob timestamp of snapshot and zip backup. [maurits] (Issue #55)
New features:
- Create symlink to latest timestamped blobstorage backup.
When
blob_timestamps
is false,blobstorage.0
is a stable filename, but withblob_timestamps
true, such a stable name was missing. [maurits] (Issue #48) - Dropped official Python 2.6 support, added Python 3.7 support. No code change, but we are no longer testing 2.6. [maurits] (Issue #49)
Bug fixes:
- Updated package versions in test buildout. Improved code quality according to flake8 and with black. [maurits] (Issue #49)
- Removed already disabled support for sitepackage_safe_scripts. [maurits] (Issue #51)
Bug fixes:
- With archive_blob true, still allow restoring non archives. We prefer archives then, but if no proper archive is found, we look for non archives. [maurits] (Issue #44)
- Sort os.listdir, for improved functioning on Mac. [maurits] (Issue #45)
- Updated readme and added sponsorship note. [maurits]
- Added
incremental_blobs
option. This creates tarballs with only the changes compared to the previous blob backups. This option is ignored when thearchive_blob
option is false. [maurits] - Improved code quality, reducing complexity. [maurits]
- Refactored the main functions to not have so much code duplication. The normal, full, snapshot and zip backups had almost the same code. This made it hard to add new options. [maurits]
- Test Python 3.6 (and 2.6 and 2.7) on Travis from now on. [maurits]
- Ignore the zope2instance recipe integration tests on Python 3.
They would need a compatible
mailinglogger
package. See issue #31. [maurits] - Tests: use cleaner way to check the mock repozo output. Share this setup between tests. This makes the output order the same on Python 2 and 3. See issue #31. [maurits]
- Added basic Python 3 support. We do not test with it yet, but you should not get NameErrors anymore. See issue #31. [maurits]
- No longer create the
fullbackup
script by default. You can still enable it by settingenable_fullbackup
totrue
. [maurits] - Without explicit
blob-storage
option, default tovar/blobstorage
. Take thevar
option from zeoserver/client recipes into account. Fixes issue #27. [maurits] - Do not create hidden backup
.0
when blob_storage ends with a slash. Fixes issue #26. [maurits]
- Make custom backup locations relative to the
locationprefix
option or thevar
directory. Until now, thelocationprefix
option was only used if you did not set custom locations. Custom location would be relative to the buildout directory. Now they are relative to thelocationprefix
option, with thevar
directory as default. So if you used a relative path, your backups may end up in a different path. Absolute paths are not affected: they ignore the locationprefix. [maurits] - When log level is DEBUG, show time stamps in the log. [maurits]
- Added
compress_blob
option. Default is false. This is only used when thearchive_blob
option is true. When switched on, it will compress the archive, resulting in a.tar.gz
instead of atar
file. When restoring, we always look for both compressed and normal archives. We used to always compress them, but in most cases it hardly decreases the size and it takes a long time anyway. I have seen archiving take 15 seconds, and compressing take an additional 45 seconds. The result was an archive of 5.0 GB instead of 5.1 GB. [maurits] - Renamed
gzip_blob
option toarchive_blob
. Kept the old name as alias for backwards compatibility. This makes room for letting this create an archive without zipping it. [maurits] - Automatically remove old blobs backups that have no corresponding filestorage backup.
We compare the timestamp of the oldest filestorage backup with the timestamps of the
blob backups. This can be the name, if you use
blob_timestamps = true
, or the modification date of the blob backup. This means that thekeep_blob_days
option is ignored, unless you useonly_blobs = true
. [maurits] - When backing up a blobstorage, use the timestamp of the latest filestorage backup.
If a blob backup with that name is already there, then there were no database changes,
so we do not make a backup.
This is only done when you use the new
blob_timestamps = true
option. [maurits] - When restoring to a specific date, find the first blob backup at or before
the specified date. Otherwise fail. The repozo script does the same.
We used to pick the first blob backup after the specified date,
because we assumed that the user would specify the exact date that is
in the filestorage backup.
Note that the timestamp of the filestorage and blobstorage backups may be
a few seconds apart, unless you use the
blob_timestamps == true
option. In the new situation, the user should pick the date of the blob backup or slightly later. [maurits] - Added
blob_timestamps
option. Default is false. By default we createblobstorage.0
. The next time, we rotate this toblobstorage.1
and create a newblobstorage.0
. Withblob_timestamps = true
, we create stable directories that we do not rotate. They get a timestamp, the same timestamp that the ZODB filestorage backup gets. For example:blobstorage.1972-12-25-01-02-03
. [maurits] - When restoring, first run checks for all filestorages and blobstorages. When one of the backups is missing, we quit with an error. This avoids restoring a filestorage and then getting into trouble due to a missing blobstorage backup. [maurits]
- Add a
locationprefix
option to configure a folder where all other backup and snapshot folders will be created [erral] - Only claim compatibility with Python 2.6 and 2.7. [maurits]
- Updated test buildout to use most recent versions. [maurits]
- Refactored the init and install methods of this recipe. During the init phase we were reading the buildout configuration, but during this phase the configuration is still being build. So differences could occur, especially in the order of execution of parts. This was not good. Most code is now moved from the init to the install (and update) method. This has less possible problems. Downside: some configuration errors are caught later. [maurits]
- Read
zeo-var
,var
,file-storage
from buildout sections. Update default backup and Data.fs locations based on this. [maurits]
- Do not accept
backup_blobs
false andenable_zipbackup
true. The zipbackup script is useless without blobs. [maurits] - Set default
backup_blobs
to true on Python 2.6 (Plone 4) and higher. Otherwise false. If noblob_storage
can be found, we quit with an error. [maurits] - Accept
true
,yes
,on
,1
, in lower, upper or mixed case as true value. Treat all other values in the buildout options as false. [maurits] - Find plone.recipe.zope2instance recipes also when they are not completely lower case. The zope2instance recipe itself works fine when it has mixed case, so we should accept this too. [maurits]
- When restoring, create
var/filestorage
if needed. Fixes #23. [maurits]
- Add
enable_fullbackup
option. Default: true, so no change compared to previous version. [maurits] - Create backup/snapshot/zipbackup directories only when needed. Running the backup script should not create the snapshot directories. [maurits]
- Add zipbackup and ziprestore scripts when
enable_zipbackup = true
. [maurits]
- Call repozo with
--quick
when making an incremental backup. This is a lot faster. Theoretically it lead to inconsistency if someone is messing in your backup directory. You can return to the previous behavior by specifyingquick = false
in the backup recipe part in your buildout config. [maurits] - check and create folders now happens after pre_commands is run [@djay]
- Add
rsync_options
option. These are added to the defaultrsync -a
command. Default is no extra parameters. This can be useful for example when you want to restore a backup from a symlinked directory, in which casersync_options = --no-l -k
does the trick. [fiterbek]
- Add
alternative_restore_sources
option. This creates abin/altrestore
script that restores from an alternative backup location, specified by that option. You can use this to restore a backup of the production data to your testing or staging server. [maurits] - When checking if the backup script will be able to create a path, remove all created directories. Until now, only the final directory was removed, and not any created parent directories. [maurits]
- Testing: split the single big doctest file into multiple files, to make the automated tests less dependent on one another, making it easier to change them and add new ones. [maurits]
- No longer test with Python 2.4, because Travis does not support it out of the box. Should still work fine. [maurits]
- Do not create blob backup dirs when not backing up blobs. Do not create filestorage backup dirs when not backing up filestorage. Fixes #17 [maurits]
- Restore compatibility with Python 2.4 (Plone 3). [maurits]
- Archive blob backups with buildout option
gzip_blob
. [matejc]
- When printing that we halt the execution due to an error running repozo, actually halt the execution. [maurits]
- Backup directories are now created when we launch
backup
orfullbackup
orsnapshotbackup
scripts, no more during initialization. [bsuttor]
- Print the names of filestorages and blobstorages that will be restored. Issue #8. [maurits]
- Added a new command-line argument :
--no-prompt
disables user input when restoring a backup or snapshot. Useful for shell scripts. [bouchardsyl] - Fixed command-line behavior with many arguments and not only a date. [bouchardsyl]
- Added
fullbackup
script that defaults tofull=true
. This could have been handled by making a new part, but it seemed like overkill to have to generate a complete new set of backup scripts, just to get one for full. [spanky]
- Fixed possible KeyError:
blob_snapshot_location
. [gforcada]
- Fixed possible KeyError:
blob_backup_location
. #3 [maurits]
- additional_filestorages improved: blob support and custom location. [mamico]
- Added pre_command and post_command options. See the documentation. [maurits]
- Moved code to github: https://github.com/collective/collective.recipe.backup [maurits]
- Fixed silly indentation error that prevented old blob backups from
being deleted when older than
keep_blob_days
days. [maurits]
- Quit the rest of the backup or restore when a repozo call gives an error. Main use case: when restoring to a specific date repozo will quit with an error when no files can be found, so we should also not try to restore blobs then. [maurits]
- Allow restoring the blobs to the specified date as well. [maurits]
- Refactored script generation to make a split between initialization code and script arguments. This restores compatibility with zc.buildout 1.5 for system pythons. Actually we no longer create so called 'site package safe scripts' but just normal scripts that work for all zc.buildout versions. [maurits]
- Added option
keep_blob_days
, which by default specifies that only for partial backups we keep 14 days of backups. See the documentation. [maurits] - Remove old blob backups when doing a snapshot backup. [maurits]
- Raise an error when the four backup location options (blobbackuplocation, blobsnapshotlocation, location and snapshotlocation) are not four distinct locations (or empty strings). [maurits]
- Fixed possible TypeError: 'Option values must be strings'. Found by Alex Clark, thanks. [maurits]
- Backup and restore blobs, using rsync. [maurits]
- Ask if the user is sure before doing a restore. [maurits]
- Fix generated repozo commands to work also when recipe is configured to have a non Data.fs main db plus additional filestorages. e.g.: datafs= var/filestorage/main.fs additional = catalog [hplocher]
- Added the option enable_snapshotrestore so that the creation of the script can be removed. Backwards compatible, if you don't specify it the script will still be created. Rationale: you may not want this script in a production buildout where mistakenly using snapshotrestore instead of snapshotbackup could hurt. [fredvd]
- Fix: when running buildout with a config in a separate directory
(like
bin/buildout -c conf/prod.cfg
) the default backup directories are no longer created inside that separate directory. If you previously manually specified one of the location, snapshotlocation, or datafs parameters to work around this, you can probably remove those lines. So: slightly saner defaults. [maurits]
- Added documentation about how to get the required bin/repozo script in your buildout if for some reason you do not have it yet (like on Plone 4 when you do not have a zeo setup). Thanks to Vincent Fretin for the extra buildout lines. [maurits]
- Added snapshotrestore script. [Nejc Zupan]
- The part name is now reflected in the created scripts and var/ directories.
Originally bin/backup, bin/snapshotbackup, bin/restore and var/backups
plus var/snapshotbackups were hardcoded. Those are still there when you
name your part
[backup]
. With a part named[NAME]
, you get bin/NAME, bin/NAME-snapshot, bin/NAME-restore and var/NAMEs plus var/NAME-snapshots. Request by aclark for plone.org. [reinout]
- Run the cleanup script (removing too old backups that we no longer want to keep) for additional file storages as well. Fixes https://bugs.launchpad.net/collective.buildout/+bug/408224 [maurits]
- Moved everything into a src/ subdirectory to ease testing on buildbot (which would grab all egss in the eggs/ dir that buildbot's mechanism creates. [reinout]
- Quote all paths and arguments so that it works on paths that contain spaces (specially on Windows). [sidnei]
- Windows path compatibility fix. [Juan A. Diaz]
- Changed the default for gzipping to True. Adding
gzip = true
to all our server deployment configs gets tired pretty quickly, so doing it by default is the best default. Stuff like this needs to be changed before a 1.0 release :-) [reinout] - Backup of additional databases (if you have configured them) now takes place before the backup of the main database (same with restore). [reinout]
- Added $BACKUP-style enviroment variable subsitution in addition to the tilde expansion offered by 0.6. [reinout, idea by Fred van Dijk]
- Fixed the test setup so both bin/test and python setup.py test work. [reinout+maurits]
- Added support for ~ in path names. And fixed a bug at the same time that would occur if you call the backup script from a different location than your buildout directory in combination with a non-absolute backup location. [reinout]
- Added support for additional_filestorages option, needed for for instance a split-out catalog.fs. [reinout]
- Test setup fixes. [reinout+maurits]
- Allowed the user to make the script more quiet (say in a cronjob) by using 'bin/backup -q' (or --quiet). [maurits]
- Refactored initialization template so it is easier to change. [maurits]
- Added 'gzip' option, including changes to the cleanup functionality that treats .fsz also as a full backup like .fs. [reinout]
- Fixed typo: repoze is now repozo everywhere... [reinout]
- Extra tests and documentation change for 'keep': the default is to keep 2 backups instead of all backups. [reinout]
- If debug=true, then repozo is also run in --verbose mode. [reinout]
- Added bin/restore. [reinout]
- Added snapshot backups. [reinout]
- Enabled cleaning up of older backups. [reinout]
- First working version that runs repozo and that creates a backup dir if needed. [reinout]
- Started project based on zopeskel template. [reinout]