Retention Policy

Discuss any other CloudBacko Pro's issues here.

Retention Policy

Postby wilton » Tue Sep 26, 2017 1:56 am

We use CloudBacko on some servers, on these servers we create a retention policy to keep the backup for 7 days, and the retention policy is waived after the backup is finished (as per photos attached), but the policy is not running and instead of keeping the backup of only 7 days there are backups of more than 3 months, I checked the logs and there is no error information, as I can verify why the policy is not being performed.

CloudBacko Version: 3.1.01
Attachments
1.png
1.png (71.01 KiB) Viewed 1349 times
2.png
2.png (60.67 KiB) Viewed 1349 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Tue Sep 26, 2017 3:35 pm

Hi Wilton,

To help us looking into this issue, could you please provide us with the Cloudbacko client log files for our investigation?

You can copy the log files to Dropbox, OneDrive, Google drive etc and share the link on this thread for us to download?

Please refer to our FAQ "Where are the CloudBacko backup and restore logs located?" viewtopic.php?f=58&t=233
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Fri Sep 29, 2017 4:20 am

Hello jeff.tang,

Sorry for the delay, as requested follows the download link for the logs, this server had CloudBacko enabled on 2017-03-24, and from the beginning has the 7 day retention enabled, as previously informed, but it is not applied and there have been backups since the mentioned date.

Link:
https://drive.google.com/open?id=0B89jK ... Xd0YkQ4WVE

I await your comments.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Fri Sep 29, 2017 3:34 pm

Hi Wilton,

Thank you for the log files. From the backup log, the retention appears to work fine with clearing data from retention 7 days ago.

[2017/09/28 01:17:52] [info] [1490373624601] Removing BackupSet="Backup Emails GQDNVG (1490373240933)" Files uploaded before backup job="2017-09-20-01-00-00"
[2017/09/28 01:17:52] [info] [1490373624601] Removing files uploaded before backup job "2017-09-20-01-00-00"

On 28th Sept, the retention in backup 20th Sep was removed by this backup job. Please be reminded that there should have live data in very old snapshots. Therefore, you should be able to see all previous snapshots and CBK will check on each snapshots to remove the files exceeding retention period.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Fri Sep 29, 2017 7:55 pm

Actually in the log is presented the information that the backup was removed, however as I print attached the content continues inside my FTP, that is, it says that it is removing but does not actually delete.

If you oberserve print, are there backups prior to the 20th, which should no longer appear, or am I mistaken?
Attachments
Seleção_033.png
Seleção_033.png (271.23 KiB) Viewed 1337 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby stephen.chow » Tue Oct 03, 2017 4:29 pm

Hi,

To clarify that the retention policy only applies on files that has been modified. If there were "very old" files and existing on an older snapshot, it is very likely that the files was never being modified / changed from the backup source. Can you please confirm if this is the case?
Stephen Chow
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
stephen.chow
 
Posts: 25
Joined: Wed Jul 06, 2016 4:10 pm

Re: Retention Policy

Postby wilton » Wed Oct 04, 2017 2:44 am

Hi,

Yes, we have folders that are changed daily and others that do not, as we back up the server's email structure.

Some doubts:

I made the first backup and in my FTP CloudBacko created a folder called 2017-10-03-01-00-00, with all the content that I selected. Inside this folder I have folders and subfolders that do not change and others that undergo changes.

So when CloudBacko will run the backup again, it will create the folder 2017-10-04-01-00-00, in this new folder only the content that has been changed will be saved and the rest (which has not changed) will be kept in the 2017-10-03-01-00-00 folder?

Now with regards to the removal, when the seven days set in the policy pass CloudBacko will remove only the content that has changed, that is, it will identify which files have changed more than seven days and remove and keep those that have not changed .

Example:

Folder 2017-10-03-01-00-00 has subfolders B and C, subfolder C has changed, after seven days the folder C will be removed (according to policy) from the folder 2017-10-03-01-00- 00 and will it be recreated in the next backup folder?
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby fabricio » Thu Oct 05, 2017 10:16 pm

Hi,

I'm watching this topic and I have the same question.

In my case, the cloudbacko was configured to retain seven days of backups but in the location there are a lot of folders and files. Do you have some documentation of backup/retain?

For example, in my mind, that could be:

- day 01: backup full
- day 02: backup incremental
- day 03 until day 07: backup incremental
- day 08: backup full again
- day 09 until day 14: backup incremental
- day 10: delete of backups day 01 until day 07 and make a full backup again
User avatar
fabricio
 
Posts: 2
Joined: Thu Oct 05, 2017 10:11 pm

Re: Retention Policy

Postby jeff.tang » Mon Oct 09, 2017 11:47 am

Hello,

For your information, all snapshots (date folder e.g. 2017-10-04-01-00-00) will not be removed from backup destination by design. Only the files/folders within the snapshot will be removed by retention policy when they over the retention period.

For your example:
Folder 2017-10-03-01-00-00 has subfolders B and C, subfolder C has changed, after seven days the folder C will be removed (according to policy) from the folder 2017-10-03-01-00- 00 and will it be recreated in the next backup folder?

The subfolder C will be kept in folder "2017-10-03-01-00-00" if the only the contents (files) in subfolder C were changed. Otherwise, the subfolder C should be removed by retention when the subfolder C has been removed from the source machine.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby fabricio » Fri Oct 20, 2017 5:34 am

Hi,

I'll post another real example, because i doesn't undertood.

I have a server with 7 days of retention. If i take a look at ftp server, i can see folders and cloudbacko's files since the configuration of software, that be more than a month. So, the backup can be restore a larger than 7 days, that i set in the retention config.

I want only seven days of backup, with one full and sixt incrementals. At eighth day, the software have to make a new backup and delete the first day. How can i configure this in the cloudbacko software?
User avatar
fabricio
 
Posts: 2
Joined: Thu Oct 05, 2017 10:11 pm

Re: Retention Policy

Postby jeff.tang » Mon Oct 23, 2017 10:21 am

Hello,

To configure a backup uploading full after 7 days incremental, you can do this with the "Advanced Settings" of the In-File Delta option. Please see attached screenshot.
in-file delta.png
in-file delta.png (22.86 KiB) Viewed 1250 times
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Tue Dec 26, 2017 6:31 pm

Good Morning,

I made these settings on all servers that have CloudBacko (more than 20 servers) and the problem persists, cloudbacko keeps files longer than seven days, and I realized that when I use the tool to free up space it does not clean old files.

I need a solution because the FTP space is filling and the backup is not performed due to lack of space.

If the problem is not resolved we will have to stop using the tool and request a refund of the licenses.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby wilton » Thu Dec 28, 2017 9:21 pm

Good Morning,

Any news about our problem?
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby stephen.chow » Fri Dec 29, 2017 5:04 pm

Hi Witon,

Regarding to your issue, could you please send us the client log files from one of your machine for our investigation first? Furthermore, please let us know which in-file delta configuration you are using. (provide us a screenshot in possible)

If the log is too large you can copy the backup log to Dropbox, OneDrive, Google drive etc and share the link on this thread?

Please refer to the article as follow for the log location on your machine:

FAQ "Where are the CloudBacko backup and restore logs located?" viewtopic.php?f=58&t=233
Stephen Chow
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
stephen.chow
 
Posts: 25
Joined: Wed Jul 06, 2016 4:10 pm

Re: Retention Policy

Postby wilton » Tue Jan 02, 2018 8:10 pm

Hello Stephen,

No problem, I'm sending the logs via Google Driver (link), and attaching to the screenshot your request, but I inform I already have sent this logs to you guys, where I was informed that in the logs there were no errors of application of the policy and I had to send print's showing that backups were still inside FTP.

As I said, I need a solution because I have 40 servers in this same situation that are having a high cost due to the occupation of disk space of my storage, in case the solution is not found as soon as possible my manager has already requested the search for a another software.

Download Logs:
https://drive.google.com/open?id=0B89jK ... Xd0YkQ4WVE
Attachments
ConfRetention.png
ConfRetention.png (62.68 KiB) Viewed 1096 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Wed Jan 03, 2018 2:58 pm

Hi Wilton,

According to your In-file delta setting, CBK will only upload full when either the number of delta is over 7 or the delta ratio is over 50%. It could be the case where the delta ratio is never over 50% and therefore, no full file is uploaded.

If you need to ensure that a full backup will be uploaded after 7 days, please uncheck the "Delta ratio" option.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Thu Jan 04, 2018 12:22 am

Hi Jeff,
So, the configuration that we are currently using was recommended by you guys, according to a previous post.

I understand your suggestion and I will put the Delta Ratio at 10% for testing and keep the settings as stated previously posted, once you have a position, working or not, post the result.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby wilton » Thu Jan 11, 2018 4:15 am

Hello,

As requested in the last post, the delta in-file decreases to 10% and even then the retention has not been applied.

I have disabled the in-file delta function, and I configured the Retention Policy for 2 Jobs and it still does not remove the files, but if I check the log it says it has been removed as below:

[2018/01/10 18:06:31] [info] [-] Start [ Linux 2.6.32-696.6.3.el6.x86_64 (iudmjf.hospedagemweb.net), CloudBacko Pro v3.3.0.1 ]
[2018/01/10 18:06:31] [info] [1514997700954] Start Backup ... [In-File Delta: Full]
[2018/01/10 18:06:31] [info] [1514997700954] Using Temporary Directory /tmp/CloudBacko Pro/1514997505282/FTP@1514997700954
[2018/01/10 18:06:31] [info] [-] Start running pre-commands
[2018/01/10 18:06:31] [info] [-] [Backup Mysql] /etc/admin/cloudflex_backup_mysql.pl
[2018/01/10 18:06:31] [info] [-] Finished running pre-commands
[2018/01/10 18:06:31] [info] [1514997700954] Downloading server file list...
[2018/01/10 18:06:32] [info] [1514997700954] Downloading server file list... Completed
[2018/01/10 18:06:32] [info] [1514997700954] Reading backup source from hard disk...
[2018/01/10 18:06:36] [info] [1514997700954] Getting all files which have been moved...
[2018/01/10 18:06:36] [info] [1514997700954] Getting all files which have been moved... Completed
[2018/01/10 18:06:36] [info] [1514997700954] Reading backup source from hard disk... Completed
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 52% of "/var/vpopmail/backup/mysql.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 91% of "/var/vpopmail/backup/mysql.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/vpopmail/backup/mysql.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/vpopmail/backup/mail_admin.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/vpopmail/backup/vpopmail.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/vpopmail/backup/roundcubemail.sql"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/log/httpd/access_log"
[2018/01/10 18:06:36] [info] [1514997700954] [Update File]... 100% of "/var/log/httpd/error_log"
[2018/01/10 18:06:37] [info] [1514997700954] Total New Files = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total New Directories = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total New Links = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total Updated Files = 6
[2018/01/10 18:06:37] [info] [1514997700954] Total Attributes Changed Files = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total Deleted Files = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total Deleted Directories = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total Deleted Links = 0
[2018/01/10 18:06:37] [info] [1514997700954] Total Moved Files = 0
[2018/01/10 18:06:37] [info] [1514997700954] Start running retention policy on backup set "Backup emails iudmjfbr(1514997505282)", "ftpbackup(1514997700954)"
[2018/01/10 18:06:37] [info] [1514997700954] Start processing space freeing up on backup set= "Backup emails iudmjfbr (1514997505282)" destination= "ftpbackup (1514997700954)"
[2018/01/10 18:06:37] [info] [1514997700954] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-10-17-36-09"
[2018/01/10 18:06:37] [info] [1514997700954] Removing files uploaded before backup job "2018-01-10-17-36-09"
[2018/01/10 18:06:37] [info] [1514997700954] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-10-17-37-19"
[2018/01/10 18:06:37] [info] [1514997700954] Removing files uploaded before backup job "2018-01-10-17-37-19"
[2018/01/10 18:06:37] [info] [1514997700954] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-10-17-38-57"
[2018/01/10 18:06:37] [info] [1514997700954] Removing files uploaded before backup job "2018-01-10-17-38-57"
[2018/01/10 18:06:37] [info] [1514997700954] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-10-17-41-04"
[2018/01/10 18:06:37] [info] [1514997700954] Removing files uploaded before backup job "2018-01-10-17-41-04"
[2018/01/10 18:06:37] [info] [1514997700954] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-10-18-02-14"
[2018/01/10 18:06:37] [info] [1514997700954] Removing files uploaded before backup job "2018-01-10-18-02-14"
[2018/01/10 18:06:37] [info] [1514997700954] File= "/"
[2018/01/10 18:06:37] [info] [1514997700954] Space freeing up on backup set= "Backup emails iudmjfbr (1514997505282)" destination= "ftpbackup (1514997700954)" is completed
[2018/01/10 18:06:37] [info] [1514997700954] Finished running retention policy on backup set "Backup emails iudmjfbr(1514997505282)", "ftpbackup(1514997700954)"
[2018/01/10 18:06:38] [info] [1514997700954] Saving encrypted backup file index to 1514997505282/blocks at destination ftpbackup...
[2018/01/10 18:06:38] [info] [1514997700954] Saving encrypted backup file index to 1514997505282/blocks/2018-01-10-18-06-13 at destination ftpbackup...
[2018/01/10 18:06:38] [info] [1514997700954] Saving encrypted backup settings to settings/iudmjf.hospedagemweb.net-2018-01-10.xml at destination ftpbackup...
[2018/01/10 18:06:39] [info] [-] Start running post-commands
[2018/01/10 18:06:39] [info] [-] Finished running post-commands
[2018/01/10 18:06:42] [info] [1514997700954] Deleting temporary file /tmp/CloudBacko Pro/1514997505282/FTP@1514997700954
[2018/01/10 18:06:42] [info] [1514997700954] Backup Completed Successfully

Before this I need some information so that the retention removes the previous files to 7 days, because as already informed I am running out of disk space for this reason. Additionally I reported that I performed tests manually running the backup with the retention at close times, but I already tested it by letting the backup be done automatically as scheduled and also did not work.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby stephen.chow » Thu Jan 11, 2018 5:50 pm

Hi Wilton,

I have created a new support ticket for you on #KLZ-969-67384. Kindly check on your mailbox and we shall continue the case on the ticket.
Stephen Chow
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
stephen.chow
 
Posts: 25
Joined: Wed Jul 06, 2016 4:10 pm

Re: Retention Policy

Postby wilton » Thu Jan 11, 2018 9:06 pm

Hello Stephen,

I ran new tests, doing backup locally and the situation, in the log says it was removed but the folders continue, I am sending the configuration and folder print attached.

Log:

[2018/01/11 10:45:13] [info] [-] Start [ Linux 2.6.32-696.6.3.el6.x86_64 (iudmjf.hospedagemweb.net), CloudBacko Pro v3.3.0.1 ]
[2018/01/11 10:45:13] [info] [1515674366808] Start Backup ... [In-File Delta: Full]
[2018/01/11 10:45:13] [info] [1515674366808] Using Temporary Directory /tmp/CloudBacko Pro/1514997505282/Local@1515674366808
[2018/01/11 10:45:13] [info] [-] Start running pre-commands
[2018/01/11 10:45:13] [info] [-] [Backup Mysql] /etc/admin/cloudflex_backup_mysql.pl
[2018/01/11 10:45:13] [info] [-] Finished running pre-commands
[2018/01/11 10:45:13] [info] [1515674366808] Downloading server file list...
[2018/01/11 10:45:13] [info] [1515674366808] Downloading server file list... Completed
[2018/01/11 10:45:13] [info] [1515674366808] Reading backup source from hard disk...
[2018/01/11 10:45:17] [info] [1515674366808] Getting all files which have been moved...
[2018/01/11 10:45:17] [info] [1515674366808] Getting all files which have been moved... Completed
[2018/01/11 10:45:17] [info] [1515674366808] Reading backup source from hard disk... Completed
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/vpopmail/backup/mail_admin.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 52% of "/var/vpopmail/backup/mysql.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 91% of "/var/vpopmail/backup/mysql.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/vpopmail/backup/mysql.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/vpopmail/backup/roundcubemail.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/vpopmail/backup/vpopmail.sql"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/log/httpd/access_log"
[2018/01/11 10:45:17] [info] [1515674366808] [Update File]... 100% of "/var/log/httpd/error_log"
[2018/01/11 10:45:18] [info] [1515674366808] Total New Files = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total New Directories = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total New Links = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total Updated Files = 6
[2018/01/11 10:45:18] [info] [1515674366808] Total Attributes Changed Files = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total Deleted Files = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total Deleted Directories = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total Deleted Links = 0
[2018/01/11 10:45:18] [info] [1515674366808] Total Moved Files = 0
[2018/01/11 10:45:18] [info] [1515674366808] Start running retention policy on backup set "Backup emails iudmjfbr(1514997505282)", "wilton(1515674366808)"
[2018/01/11 10:45:18] [info] [1515674366808] Start processing space freeing up on backup set= "Backup emails iudmjfbr (1514997505282)" destination= "wilton (1515674366808)"
[2018/01/11 10:45:18] [info] [1515674366808] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-11-10-39-53"
[2018/01/11 10:45:18] [info] [1515674366808] Removing files uploaded before backup job "2018-01-11-10-39-53"
[2018/01/11 10:45:19] [info] [1515674366808] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-11-10-41-10"
[2018/01/11 10:45:19] [info] [1515674366808] Removing files uploaded before backup job "2018-01-11-10-41-10"
[2018/01/11 10:45:20] [info] [1515674366808] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-11-10-42-16"
[2018/01/11 10:45:20] [info] [1515674366808] Removing files uploaded before backup job "2018-01-11-10-42-16"
[2018/01/11 10:45:21] [info] [1515674366808] Removing BackupSet="Backup emails iudmjfbr (1514997505282)" Files uploaded before backup job="2018-01-11-10-43-52"
[2018/01/11 10:45:21] [info] [1515674366808] Removing files uploaded before backup job "2018-01-11-10-43-52"
[2018/01/11 10:45:21] [info] [1515674366808] File= "/"
[2018/01/11 10:45:21] [info] [1515674366808] Space freeing up on backup set= "Backup emails iudmjfbr (1514997505282)" destination= "wilton (1515674366808)" is completed
[2018/01/11 10:45:21] [info] [1515674366808] Finished running retention policy on backup set "Backup emails iudmjfbr(1514997505282)", "wilton(1515674366808)"
[2018/01/11 10:45:21] [info] [1515674366808] Saving encrypted backup file index to 1514997505282/blocks at destination wilton...
[2018/01/11 10:45:21] [info] [1515674366808] Saving encrypted backup file index to 1514997505282/blocks/2018-01-11-10-44-55 at destination wilton...
[2018/01/11 10:45:22] [info] [1515674366808] Saving encrypted backup settings to settings/iudmjf.hospedagemweb.net-2018-01-11.xml at destination wilton...
[2018/01/11 10:45:22] [info] [-] Start running post-commands
[2018/01/11 10:45:22] [info] [-] Finished running post-commands
[2018/01/11 10:45:25] [info] [1515674366808] Deleting temporary file /tmp/CloudBacko Pro/1514997505282/Local@1515674366808
[2018/01/11 10:45:25] [info] [1515674366808] Backup Completed Successfully

Additionally, I have already tried all the settings described in previous posts, activating the In-File Delta and so far nothing works for the removal.
Attachments
InFileDelta.png
InFileDelta.png (43.79 KiB) Viewed 1056 times
Retention.png
Retention.png (39.05 KiB) Viewed 1056 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby walter » Thu Jan 11, 2018 10:50 pm

Hello Guys,

I understand your information when you say the backups is running normally in the logs because now I see what you mean with the example A,B,C.

But my problem is the same of this guys here, CloudBacko is not delete the old files (7 days for example) the tool just remove the file to old(A) and move the others files/folders (B and C) to the next backup job, until B or C turn to be a old file/folder (more than 7 days) and not enter anymore in the backup job.

So CloudBacko have or not the policy to delete this files and erase from the FTP or just remove from the backup job?
User avatar
walter
 
Posts: 1
Joined: Thu Jan 11, 2018 10:36 pm

Re: Retention Policy

Postby wilton » Thu Jan 11, 2018 11:09 pm

Hi Stephen,

So If I understand right, CloudBacko remove from the backup job the incrementals one's but is not delete from the FTP. That sentence is rigth?
Because if it is I need to delete that information not just remove from the backup job.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby stephen.chow » Fri Jan 12, 2018 3:35 pm

wilton wrote:Hi Stephen,

So If I understand right, CloudBacko remove from the backup job the incrementals one's but is not delete from the FTP. That sentence is rigth?
Because if it is I need to delete that information not just remove from the backup job.


Hi Wilton,

No, it is not correct. CloudBacko could remove the files that "marked" as retained data from its storage by the retention policy, even it is on the FTP storage.

To clarify that the retention only take effect when:
1. The files that has been modified. i.e. If "file A" uploaded since the initial backup and being never modified, the "file A" will always staying in the Data area and will never moved to the retention - even there is a retention policy configured (unless the "file A" has been modified and reuploaded to the data area)

2. No Delta chain between the FULL files and current files (deltas). Hence, the behaviour of incremental in-file delta do not remove files by retention policy.

Furthermore, to clarify that the message "Removing BackupSet="XXXXXXX" Files uploaded before backup job="XXXXXXX" would logged when every time the retention policy runs - it is a
message referring to the application attempted to scan the existing snapshot folder from the data storage area, and it is not referring a deletion of the data.

Hope this could gives a better understanding to your case.

To further work on the issue, We shall continue the discuss on the ticket that I've created on yesterday as we may require further follow up with you on the issue. (Please check your mail box for the related message)
Stephen Chow
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
stephen.chow
 
Posts: 25
Joined: Wed Jul 06, 2016 4:10 pm

Re: Retention Policy

Postby wilton » Fri Jan 12, 2018 8:57 pm

Hi Stephen,

Thanks for your reply

So, you are saying in example if I have 3 Files(a, B and C), A with 200 GB, B with 1 TB and C with 1 MB, and CloudBacko made a FULL backup of those 3 files. 1 year pass and I never modified the C file , the tool would keep the folder with 1 TB 200GB and 1 MB because one day I made one FULL backup 1 year ago??

I need CloudBacko every time I have the second Full backup of the server, the old one ( 8 days earlyer) have to be erase from my FTP. I don't understand why the files of my backups need to be keeped on the server if is not been modified anymore.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby wilton » Sat Jan 13, 2018 3:31 am

Thank you for your reply

So, my question is simple this time, did you confirm CloudBacko erase old Backups in the retention policy? If your answer is yes, how I configure this function.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby wilton » Tue Jan 16, 2018 2:04 am

Thank you for your reply

So, my question is simple this time, did you confirm CloudBacko erase old Backups in the retention policy? If your answer is yes, how I configure this function.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Tue Jan 16, 2018 3:26 pm

Hi Wilton,

For retention policy to erase old file, you can try with below example.

Firstly, configure 1 day retention policy from CBK.
Run a backup to backup a, b and c files.
Delete file a and b from the source machine and run backup again.
both a and b files should be marked as retention again.
next day, run backup again and both a and b should be removed from retention.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Thu Jan 18, 2018 12:58 am

Hi Stephen,

As you recommended, I performed the procedure:

I created a policy retention witch one day.
I created a TestRetention (folder) with 3 files Text1.txt, Text2.txt and Text3.txt
I run the backup, and the folder 2018-01-16-10-26-28 with the size of 24K was created.
I go to the TestRetention and removed (rm -rf) the Text1.txt, Text2.txt, leaving only Text3.txt
I run the backup, and the tool created the folder 2018-01-16-10-27-26, with the size of 16K
I waited more than 24 hours, ran the backup again and was created 2018-01-17-11-18-26 with the size of 16KB.


That is, the folder 2018-01-16-10-26-28 that would be backed up with the three files was not removed, CloudBakco just din't back up the deleted files, only take them out of the backup job.

Facing this I want to know when the folder 2018-01-16-10-26-28 and its contents will be erased.

Here are images of the settings and the backup folder.

In addition, we inform you that we do not change the in-File Delta settings, we leave the default settings.
Attachments
delta.png
delta.png (60.53 KiB) Viewed 988 times
retention.png
retention.png (42.71 KiB) Viewed 988 times
folder.png
folder.png (52.94 KiB) Viewed 988 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby wilton » Thu Jan 18, 2018 12:59 am

Hi Stephen,

Additionally follow the CloudBacko logs.
Attachments
logs.tar
(20 KiB) Downloaded 9 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Thu Jan 18, 2018 11:49 am

Hi Wilton,

Thank you for your update. Can you please run the backup once more to see if the files will be deleted from retention?
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Fri Jan 19, 2018 2:50 am

Hi Stephen,

As requested I waited another 24 hours and ran the backup again, and the folder 2018-01-16-10-26-28 was not deleted and neither its content in print can be seen that its size has not changed.

Follow the attached logs and print.
Attachments
logs.tar
(30 KiB) Downloaded 10 times
folder.png
folder.png (26.01 KiB) Viewed 980 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Fri Jan 19, 2018 12:06 pm

Hi Wilton,

For your information, the folder 2018-01-16-10-26-28 is backup snapshot which will not be deleted by retention job. Only the content within the folder will be deleted.

Apart from that, may ask what is the size of the test files? It appears that the size is too small to show from the list command even with the files being removed from the snapshot. I would suggest to try a file with few MB instead. Moreover, please also check if the deleted files are still available from the restore option on CBK.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Wed Jan 24, 2018 8:18 pm

Hi Stephen,

Based on your latest information I ran a new test:

1 - I created three files and performed the manual backup (Figure 1 - 19/01/2018 at 17:23).

2 - I changed the size of Texte3.txt and performed the backup manually and you can view the file with the new size (figure 2 - 19/01/2018 at 17:24).

3 - I let the scheduled backup run automatically without making any changes (figure 3 - 19/01/2018 at 18:15).

4 - I let the scheduled backup run automatically without making changes (figure 4 - on 01/20/2018 at 18:15).

5 - I let the scheduled backup run automatically without making any changes (Figure 5 - 21/01/2018 at 18:15).

6 - After this last automatic backup I performed the removal of the Text1.txt file and let the backup run automatically and no longer see the deleted file (Figure 6 - 22/01/2018 at 18:15).

7 - After this last automatic backup I performed the removal of the Text1.txt file and let the backup run automatically and no longer see the deleted file (Figure 7 - 23/01/2018 at 18:15).

Faced with this there are two doubts, regarding my retention that is configured for one day:

1 - Because the file that has undergone changes on 01/19/2018 can still be restored with the original size, the correct one would not only be the restoration of the file with the new size?

2- Because the file that was removed on 01/22/2018 can still be restored in the previous days, the correct one would not be that it had been deleted?

Attached are logs and figures mentioned.
Attachments
LogsFigure.tar
(256.76 KiB) Downloaded 11 times
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Thu Jan 25, 2018 11:41 am

Hi Wilton,

>1 - Because the file that has undergone changes on 01/19/2018 can still be restored with the original size, the correct one would not only be the restoration of the file with the new size?
Ans. You should be able to restore the file with the original size by running incremental delta backup since the original full file is required to keep with the incremental changes in order to restore the latest snapshot.

>2- Because the file that was removed on 01/22/2018 can still be restored in the previous days, the correct one would not be that it had been deleted?
The file is got deleted on 01/22/2018 which should be still available in retention configured for 1 day. Please be reminded that the retention is to keep file from the day of deletion. Therefore, the file should be removed from retention if you run backup again on 01/24/2018.
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am

Re: Retention Policy

Postby wilton » Thu Jan 25, 2018 8:56 pm

Hi Stephen,

With regard to file change I understood how it works, it will never perform a file consolidation, ie it will generate a new file with changes that is kept to the possibility of restoring this file on different dates.

With regard to file deletion, so I understand, if the retention is set to one day, it will take two days for the file to be permanently deleted, on the other hand, I imagine if you have a 7 day retention the deleted file will take in around 14 days to be definitively excluded, and this caused my doubt, because I imagined that soon on the eighth day he would be definitively excluded.

Now I have another question, I would like to know how to restore a server that has corrupted your operating system, for example:

I have a server that has CloudBacko installed and making backups of my emails on an external FTP, when I access the FTP with view the xxxx.bak files generated by CloudBacko, however that server had a problem and corrupted the operating system, how I can restore the backups of my emails in this server, because I lost access to the profile created in CloudBacko.

Well this happened this week, but when I reconfigured the server and CoudBacko (with the same username and FTP password) I was not able to access the real backups.
User avatar
wilton
 
Posts: 20
Joined: Mon Sep 25, 2017 11:31 pm

Re: Retention Policy

Postby jeff.tang » Mon Jan 29, 2018 3:34 pm

Hi Wilton,

Thank you for your reply.

For better handling and more effective, we have changed our support channel from Forum to our ticket system. For your new inquiry regarding to restore a corrupted system, please kindly submit ticket via:

https://www.cloudbacko.com/en/cloudback ... et-sla.jsp
Jeff Tang
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
jeff.tang
 
Posts: 748
Joined: Mon Feb 02, 2015 9:50 am


Return to Others

Who is online

Users browsing this forum: No registered users