Check Data Integrity has errors. Now what? (#10437)

Discuss any files backup related issues here.

Check Data Integrity has errors. Now what? (#10437)

Postby rockymountain » Sat Oct 11, 2014 10:59 pm

Hi,

Using Cloudbacko Pro (trial right now), Windows 7, File Backup, Backing up to Google Cloud Storage.

Otherwise apparently working, but... When I run "Data Integrity Check", I get this message:
"Data Integrity Check is completed with Error(s)."

The operative line from the Log appears to be:
"Failed to check integrity of destination "1412304996963". Error: [Runtime exception][Bttree.KeyRangeIterator...." <truncated>

The impression I get from this message is that rather than reporting an integrity failure, it is reporting that it is simply unable to complete the check for some reason. Nonetheless, this does not inspire confidence. How do I know that my data is safe? What do I do to fix this problem?

I see no similar problem with the backup runs themselves. They complete successfully,with no error message in the logs.

What now?
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what?

Postby CB.Support.Manager » Mon Oct 13, 2014 6:41 pm

Hello

Thank you for your interest in CloudBacko Pro.

The partial error message you post "Bttree.KeyRangeIterator" seems to indicate an index related issue with the backup set.

You mentioned there were no problems with the backups themselves.

-Do you run a backup after you encountered this error? Was the backup successful?
-Did you have any problems when you restored any of the files using CloudBacko Pro?
-Is Google Cloud Storage the only backup destination configured for this backup set?
-How many files/folders did you upload to Google Cloud Storage?

To allow us to get a better understanding of this problem. Do you mind copying the CloudBacko Pro system logs to Dropbox, OneDrive, Google drive etc and share the link on this thread?

The system logs files are located in:
- C:\users\{username}\.cbp\system
- C:\Users\Administrator\.cbp\log\System
Support Manager of CloudBacko
The Most Secure Cloud & Local Backup Software
User avatar
CB.Support.Manager
Site Admin
 
Posts: 443
Joined: Wed Aug 21, 2013 5:53 pm

Re: Check Data Integrity has errors. Now what?

Postby rockymountain » Tue Oct 14, 2014 7:20 am

-Do you run a backup after you encountered this error? Was the backup successful?

Yes, backups continue to complete successfully.
Integrity checks continue to fail.

-Did you have any problems when you restored any of the files using CloudBacko Pro?

Have not tried. Last thing I want to do is restore from a backup that has not passed integrity check!
I suppose I could back up a test file and then try. I'll let you know shortly.

-Is Google Cloud Storage the only backup destination configured for this backup set?

Yes.

-How many files/folders did you upload to Google Cloud Storage?

32GB worth backed up, at last count. Another 4GB added every night. Not sure how to tell the number of files.


Nor sure if its relevant, but I should also point out that I switched from Cloudbacko Lite to Cloudbacko Pro, after I already had some files backed up. I exported settings from lite, then imported them into pro, then uninstalled light. I have no idea if this problem started before or after that, because I only tried "integrity check" for the first time after I switched to Pro. I have never seen an Integrity Check with any other outcome.
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what?

Postby rockymountain » Tue Oct 14, 2014 7:32 am

The system logs files are located in:
- C:\users\{username}\.cbp\system
- C:\Users\Administrator\.cbp\log\System

There does not seem to be a C:\Users\Administrator\ on my system.

C:\users\{username}\.cbp\system has been uploaded here:
https://drive.google.com/file/d/0B-RzrE ... sp=sharing
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what?

Postby CB.Support.Manager » Tue Oct 14, 2014 9:51 am

Hello

Thank you for update.

My apologies the system logs should be in C:\Users\{username}\.cbp\log\system

Do you mind posting these logs as well? As the CRC integrity logs are located in this folder.
Support Manager of CloudBacko
The Most Secure Cloud & Local Backup Software
User avatar
CB.Support.Manager
Site Admin
 
Posts: 443
Joined: Wed Aug 21, 2013 5:53 pm

Re: Check Data Integrity has errors. Now what?

Postby rockymountain » Tue Oct 14, 2014 8:21 pm

-Did you have any problems when you restored any of the files using CloudBacko Pro?

Have not tried. Last thing I want to do is restore from a backup that has not passed integrity check!
I suppose I could back up a test file and then try. I'll let you know shortly.


I just tried adding a small test file, backed up, deleted the file, and then restored. Everything worked fine.
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what?

Postby CB.Support.Manager » Wed Oct 15, 2014 10:46 am

Hello

Thank you for updating us.

We are currently looking at the details errors in the CRC checking log you sent.
Support Manager of CloudBacko
The Most Secure Cloud & Local Backup Software
User avatar
CB.Support.Manager
Site Admin
 
Posts: 443
Joined: Wed Aug 21, 2013 5:53 pm

Re: Check Data Integrity has errors. Now what?

Postby CB.Support.Manager » Thu Oct 16, 2014 4:35 pm

Hello,

After analysis the CRC check logs you sent us we suspect the problem is related to the CloudBacko index files and not the actual files you uploaded to Google Cloud Storage. Specifically the indexes dealing with the files in the retention which is why you are not getting any errors during your backup jobs.

Do you mind send us a copy of your CloudBacko Pro temporary directory as a copy of the indexes are also stored in this folder?
Support Manager of CloudBacko
The Most Secure Cloud & Local Backup Software
User avatar
CB.Support.Manager
Site Admin
 
Posts: 443
Joined: Wed Aug 21, 2013 5:53 pm

Re: Check Data Integrity has errors. Now what?

Postby rockymountain » Fri Oct 17, 2014 8:28 am

- Do you mind send us a copy of your CloudBacko Pro temporary directory as a copy of the indexes are also stored in this folder?

Can you give me a directory path? I don't see anything that is obviously a temporary directory.

Also, how should I send it to you? I'm a little uncomfortable with attaching this sort of thing to a public forum. Not sure what sort of information it contains.
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what?

Postby CB.Support.Manager » Fri Oct 17, 2014 10:47 am

Hello.

The temporary folder path is usually shown at the beginning of the backup job and is the location is user defined. You can find the temporary folder path from the backup report in CloudBacko UI under [Report] or directly from the backup logs.

Please refer to our FAQ "Where are the CloudBacko backup and restore logs located?" viewtopic.php?f=58&t=233

Example
[2014/10/12 23:55:15] [info] [1413183294059] Start [ Windows Server 2008 R2 (WIN2012R@), CloudBacko Pro 1.5.2.0 ]
[2014/10/12 23:55:15] [info] [1413183294059] Start Backup ... [In-File Delta: Incremental]
[2014/10/12 23:55:15] [info] [1413183294059] Using Temporary Directory D:\Temp\CloudBacko Pro\1413183215185\Local@1413183294059
[2014/10/12 23:55:15] [info] [-] Start running pre-commands
[2014/10/12 23:55:15] [info] [-] Finished running pre-commands


In this example, you will need to zip the "D:\Temp" folder.

The indexes themselves files are actually encrypted and there is no personal files or data stored in the temporary folder as any backup related information is deleted as the "Remove temporary files after backup" option is selected by default.

Do you mind copying the contents of the temporary folder (zipped) to Dropbox, OneDrive, Google drive etc and share the link on this thread? Alternatively you can PM me the link if you are concerned about privacy.
Support Manager of CloudBacko
The Most Secure Cloud & Local Backup Software
User avatar
CB.Support.Manager
Site Admin
 
Posts: 443
Joined: Wed Aug 21, 2013 5:53 pm

Re: Check Data Integrity has errors. Now what?

Postby rockymountain » Tue Oct 21, 2014 11:56 am

OK, here's the zip
https://drive.google.com/file/d/0B-RzrE ... sp=sharing

BTW I notice that the locks are being stored in a directory named Cloudbacko Lite, even though I am using Cloudbacko Pro. I assume this is just a legacy of the fact that I started out using Lite and then switched to Pro later on?
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what? (#10437)

Postby rafael » Tue Oct 21, 2014 4:34 pm

Hello

Thank you for the index files.

I will pass on the details to our developers they will try to find out if the problem is related to issues with the index files.

This investigation may take some time I will try to give you an update within the next two weeks.

After you upgraded to CloudBacko Pro if you did not changes the temporary folder path then it will be stored under Cloudbacko Lite folder.
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm

Re: Check Data Integrity has errors. Now what? (#10437)

Postby rockymountain » Tue Nov 04, 2014 8:26 am

Hi,

Just wondering if there's been any progress on this?

I know you said it'll take a while, but my trial subscription is about to expire. I like the product otherwise, but I'm reluctant to commit to cloudbacko at this point, since I've not yet seen a single integrity check pass, to date. Until I can feel confident that my data is safe, I can't cancel my other backup provider.

Derek.
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what? (#10437)

Postby rafael » Tue Nov 04, 2014 1:26 pm

Hello Derek,

My apologies our developers are still looking into the problem, one of the difficulties is we cannot reproduce the problem based on the information and setup you provided us.

Have you tried restoring other files from this backup set? Do you encounter any problems during the restore?

Do you mind helping further by performing the following test?

1. Create a new file backup set with Google Cloud Storage as the destination
2. Select some files to backup (around 100 small files)
3. After the backup is completed successfully.
4. Run a data integrity check on the new backup set and verify if there are any errors.
5. Restore all the files back to your machine and see if there are any errors.
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm

Re: Check Data Integrity has errors. Now what? (#10437)

Postby rockymountain » Wed Nov 05, 2014 11:30 am

1. Create a new file backup set with Google Cloud Storage as the destination
2. Select some files to backup (around 100 small files)
3. After the backup is completed successfully.
4. Run a data integrity check on the new backup set and verify if there are any errors.
5. Restore all the files back to your machine and see if there are any errors.


OK, I did the experiment you proposed with a smallish file set.
Everything worked flawlessly, including integrity check (with CRC) and subsequent recovery after I deleted the local files.

This is encouraging, but my main PC backup still fails integrity check every time.

It is tempting to just make a clean start. Erase the online backup, and start with a clean slate -- see if its better the second time around. Problem is, my data set is 500+GB, and I can only back up about 5GB per day. As is, it'll probably be 10 more weeks before I have a complete GCS backup and can cancel my other backup service (Carbonite). Making a fresh start will just push that out even longer.

And now my Cloudbacko trial is about to run out any day now, without ever once having passed integrity check on my PC backup.

I don't want to give up on Cloudbacko. It seems like a good solution. But what am I supposed to do, if it fails its own integrity check every time?
User avatar
rockymountain
 
Posts: 30
Joined: Sat Oct 11, 2014 10:01 pm

Re: Check Data Integrity has errors. Now what? (#10437)

Postby rafael » Wed Nov 05, 2014 5:02 pm

Hello Derek,

Thank you for sharing your results with us. Your results are identical to our own testing.

We suspect it could be related issues encountered during migration of the file backup set from CloudBacko Lite to CloudBacko Pro. Although when our developers ran through the same scenario no errors where found.

We understand you are dealing with a large amount of data and you have limited bandwidth, but as you mentioned it may be better in this particular instance to make a clean start.
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm


Return to Files backup and restore

Who is online

Users browsing this forum: Bing [Bot]