Error Message - Backups not running

Discuss any Exchange Database backup and restore issues here.

Error Message - Backups not running

Postby barrettem@mika.com » Mon Jan 05, 2015 12:28 am

I am getting the following error, i have rebooted, recreated the backup set, cleared the temp files and updated both java and CloudBacko - any ideas?

Server 2008 R2 - Exchange 2010, two servers with clustering enabled. Running only on local databases. i get error withing 13 secs of starting backup

[2015/01/04 11:15:01] [erro] [BackupSetCmd.uploadBackupFiles][java.lang.RuntimeException] Error=[RuntimeException] [BackupFileIterator.Combined.hasNext] Throwable=[BackupFileIterator.hasNext] Throwable=[Bptree.KeyRangeIterator.hasNext] Error=null. [BackupFileIterator.toString] bfCachedNext=null bfLastReturn=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] bClose=false bfs=BackupSetIndex{fIndexDir=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index, sBackupSetHome='1413468546215', sBackupSetName='USEV Exchange'} itr=[RowIterator.toString] bdb=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index\index.b2b ibptkeyMin=[IBptree.SimpleKey] Key='null' ibptkeyMax=[IBptree.SimpleKey] Key='null' bExcludeSameMaxKey=true ibptkeyCurrent=[IBptree.SimpleKey] Key='null' ibptvalCurrent=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] itr=[RowIterator.Disk.toString] itrKeys=com.ahsay.afc.db.bdb2.Bptree$KeyRangeIterator@66a33414 tvn=[TreeValueNode.toString]@64b041b0 lBlockStart='0xc6c800' lNextBlockOffset='0xffffffffffffffff' lFirstBlockOffset='0xc6c800' lLastBlockOffset='0xc6c800' ibptval='[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d]' byPrevOffsetLength='8' lSameKeyPrevOffset='0xffffffffffffffff' byNextOffsetLength='8' lSameKeyNextOffset='0xffffffffffffffff' iRefCount='1' ff=null. [BackupFileIterator.Combined.toString] bfCurrentItr=[BackupFileIterator.toString] bfCachedNext=null bfLastReturn=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] bClose=false bfs=BackupSetIndex{fIndexDir=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index, sBackupSetHome='1413468546215', sBackupSetName='USEV Exchange'} itr=[RowIterator.toString] bdb=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index\index.b2b ibptkeyMin=[IBptree.SimpleKey] Key='null' ibptkeyMax=[IBptree.SimpleKey] Key='null' bExcludeSameMaxKey=true ibptkeyCurrent=[IBptree.SimpleKey] Key='null' ibptvalCurrent=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] itr=[RowIterator.Disk.toString] itrKeys=com.ahsay.afc.db.bdb2.Bptree$KeyRangeIterator@66a33414 tvn=[TreeValueNode.toString]@64b041b0 lBlockStart='0xc6c800' lNextBlockOffset='0xffffffffffffffff' lFirstBlockOffset='0xc6c800' lLastBlockOffset='0xc6c800' ibptval='[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d]' byPrevOffsetLength='8' lSameKeyPrevOffset='0xffffffffffffffff' byNextOffsetLength='8' lSameKeyNextOffset='0xffffffffffffffff' iRefCount='1' ff=null, caused by [RuntimeException] [BackupFileIterator.hasNext] Throwable=[Bptree.KeyRangeIterator.hasNext] Error=null. [BackupFileIterator.toString] bfCachedNext=null bfLastReturn=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] bClose=false bfs=BackupSetIndex{fIndexDir=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index, sBackupSetHome='1413468546215', sBackupSetName='USEV Exchange'} itr=[RowIterator.toString] bdb=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index\index.b2b ibptkeyMin=[IBptree.SimpleKey] Key='null' ibptkeyMax=[IBptree.SimpleKey] Key='null' bExcludeSameMaxKey=true ibptkeyCurrent=[IBptree.SimpleKey] Key='null' ibptvalCurrent=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] itr=[RowIterator.Disk.toString] itrKeys=com.ahsay.afc.db.bdb2.Bptree$KeyRangeIterator@66a33414 tvn=[TreeValueNode.toString]@64b041b0 lBlockStart='0xc6c800' lNextBlockOffset='0xffffffffffffffff' lFirstBlockOffset='0xc6c800' lLastBlockOffset='0xc6c800' ibptval='[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935E.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d]' byPrevOffsetLength='8' lSameKeyPrevOffset='0xffffffffffffffff' byNextOffsetLength='8' lSameKeyNextOffset='0xffffffffffffffff' iRefCount='1' ff=null, caused by [RuntimeException] [Bptree.KeyRangeIterator.hasNext] Error=null, caused by [StackOverflowError] null
User avatar
barrettem@mika.com
 
Posts: 4
Joined: Mon Jan 05, 2015 12:21 am

Re: Error Message - Backups not running

Postby rafael » Mon Jan 05, 2015 9:55 am

Hello,

Although you have posted some error messages from your Exchange backup job which seems to indicate some index related issues. there is insufficient information for us to locate the source of the problem.

-Is the E:\Temp\CloudBacko Pro\1413468546215 located on a local drive on the same machine as your CloudBacko Pro installation?
-Apart from this local Exchange database backup are there any other backup sets configured, i.e. file, Windows System backup, Exchange mail level? Are there any problem with these backup jobs?
-Is there any anti-virus software installed on this machine? If so is the "E:\Temp" folder excluded from the ant-virus scan?
-Is the storage destination on a local, network, or Cloud drive?

To help to get clearer picture of your Hyper-V backup problem. Do you mind copying the backup logs to Dropbox, OneDrive, Google drive etc and share the link on this thread?

Please refer to our FAQ "Where are the CloudBacko backup and restore logs located?" viewtopic.php?f=58&t=233
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm

Re: Error Message - Backups not running

Postby barrettem@mika.com » Tue Jan 06, 2015 10:58 pm

- E:\Temp\CloudBacko Pro\1413468546215 is on a local drive on the same machine.
- No other backups configured.
- E:\Temp is not excluded from the anti-virus scan, but we only have the scan scheduled once a week.
- Storage destination is on a network drive.

Here's the link to the logs: https://www.dropbox.com/sh/4fg5nwa6ucej ... xk2Da?dl=0
User avatar
barrettem@mika.com
 
Posts: 4
Joined: Mon Jan 05, 2015 12:21 am

Re: Error Message - Backups not running

Postby rafael » Wed Jan 07, 2015 12:15 pm

Hello,

The errors appear to indicate a local index related problem but you have already stated that you manually cleared the temporary folder E:\Temp\CloudBacko Pro. Did you perform this action after each backup job encountered this error?

The logs you sent us cover backups from Jan 2nd 2015 to Jan 6th 2015, we noticed on each of the backup jobs the error refers to [BackupByJob=2014-12-05-05-00-00] (backup job on Dec 5th 2014), so we suspect the current problem is likely caused by a issue in a previous backup job(s). Do you mind sending us all the backup logs for this Exchange backup set even if they are successful?

BackupSetCmd.uploadBackupFiles][java.lang.RuntimeException] Error=[RuntimeException] [BackupFileIterator.Combined.hasNext] Throwable=[BackupFileIterator.hasNext] Throwable=[Bptree.KeyRangeIterator.hasNext] Error=null. [BackupFileIterator.toString] bfCachedNext=null bfLastReturn=[BackupFile][ClientPath=Microsoft Exchange Server\Microsoft Information Store\USEVWESMAIL04\9d4a6194-8660-4dd3-bed3-33b747a41d93\E00001A935D.log][Type=F][BackupByJob=2014-12-05-05-00-00][FileSystemObjectType=2][ServerPath=1413468546215\files\0xb3\0x68\0x06\0x6d] bClose=false bfs=BackupSetIndex{fIndexDir=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index, sBackupSetHome='1413468546215', sBackupSetName='USEV Exchange'} itr=[RowIterator.toString] bdb=E:\Temp\CloudBacko Pro\1413468546215\Local@1413468684491\index\index.b2b ibptkeyMin=

In order to completely rule out any issue with anti-virus software interfering with index files by mistake. Do you mind permanently excluding the E:\Temp folder from any manual and automatic anti-virus scans?
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm

Re: Error Message - Backups not running

Postby barrettem@mika.com » Fri Jan 09, 2015 4:27 am

All of the logs are uploaded. Included some new ones that ran since my last post.
The temp folder has been excluded from antivirus.
I tried running the backup set manually today after excluding it; backup failed.
User avatar
barrettem@mika.com
 
Posts: 4
Joined: Mon Jan 05, 2015 12:21 am

Re: Error Message - Backups not running

Postby rafael » Fri Jan 09, 2015 10:31 am

Hello,

Although you already mentioned the backup set has been recreated but given the circumstances I think it would be better if we just started off fresh instead of digging into the problem.

1. Delete the current backup set and data in the storage destination.
2. Delete the E:\Temp\CloudBacko Pro folder.
3. Create a new Exchange database backup set using CloudBack Pro v1.9.0.0 and run a full backup
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm

Re: Error Message - Backups not running

Postby barrettem@mika.com » Wed Jan 21, 2015 9:41 pm

Did as you suggested, and it failed with the following log:

[2015/01/21 08:36:20] [info] [-] Changing backup type from "" to "DATABASE" because backup has never been run against destination "Bean" before
[2015/01/21 08:36:21] [info] [1421847358012] Start [ Windows Server 2008 R2 (USEVwesMail04), CloudBacko Pro 1.9.0.0 ]
[2015/01/21 08:36:33] [info] [1421847358012] Start Backup ... Database [In-File Delta: Full]
[2015/01/21 08:36:33] [info] [1421847358012] Using Temporary Directory E:\Temp\CloudBacko Pro\1421787415953\Local@1421847358012
[2015/01/21 08:36:33] [info] [1421847358012] Microsoft Exchange Server 2010 (14.1.218.15)
[2015/01/21 08:36:33] [info] [-] Start running pre-commands
[2015/01/21 08:36:33] [info] [-] Finished running pre-commands
[2015/01/21 08:36:33] [info] [1421847358012] Downloading server file list...
[2015/01/21 08:36:33] [info] [1421847358012] Downloading server file list... Completed
[2015/01/21 08:36:33] [info] [-] Start Microsoft Exchange backup
[2015/01/21 08:36:33] [info] [-] Start creating Shadow Copy Set...
[2015/01/21 08:36:35] [erro] [-] Cannot start shadow copy, reason =
[2015/01/21 08:36:35] [info] [1421847358012] Saving encrypted backup file index to 1421787415953/blocks at destination Bean...
[2015/01/21 08:36:35] [info] [1421847358012] Saving encrypted backup settings to settings/USEVwesMail04-2015-01-21.xml at destination Bean...
[2015/01/21 08:36:35] [info] [1421847358012] Saving encrypted backup log files to 1421787415953/logs at destination Bean...
[2015/01/21 08:36:35] [info] [-] Start running post-commands
[2015/01/21 08:36:35] [info] [-] Finished running post-commands
[2015/01/21 08:36:35] [info] [1421847358012] Deleting temporary file E:\Temp\CloudBacko Pro\1421787415953\Local@1421847358012
[2015/01/21 08:36:35] [erro] [1421847358012]
[2015/01/21 08:36:35] [erro] [1421847358012] Backup completed with error(s)
User avatar
barrettem@mika.com
 
Posts: 4
Joined: Mon Jan 05, 2015 12:21 am

Re: Error Message - Backups not running

Postby rafael » Fri Jan 23, 2015 10:18 am

Hello,

It looks like CloudBacko Pro encountered some type of Microsoft Volume Shadow Copy error when it tried to take a snapshot of the Exchange 2010 database in preparation for the backup job. This is causing you backup job to failed.

[2015/01/21 08:36:33] [info] [-] Start creating Shadow Copy Set...
[2015/01/21 08:36:35] [erro] [-] Cannot start shadow copy, reason =

To resolve this issue:

1. Register the Volume Shadow Copy by running the C:\Program Files\CloudBacko Pro\bin\RegisterVSS.bat file
2. After re-registering VSS on your Exchange server, please reboot the machine for the settings to take effect.
3. After the machine is rebooted, run the following command "vssadmin list writers" and verify if the Exchange VSS writers are stable and are not reporting errors.
Rafael Ho
CloudBacko Forum Support
The Most Secure Cloud & Local Backup Software
User avatar
rafael
Site Admin
 
Posts: 998
Joined: Thu Jun 12, 2014 4:41 pm


Return to Microsoft Exchange Server Database

Who is online

Users browsing this forum: No registered users