Integrity Check

The function of the Data Integrity Check is to perform a health check for the index file and backed up data by %edition_name% to ensure the data integrity and restorability. After each data integrity check, the storage statistics will be refreshed.

Please kindly note that:

The Data Integrity Check CANNOT fix or repair files that are already corrupted. Data Integrity Check feature is used to identify and remove any corrupted file(s) on the backup destination(s), so that they can be backed up again to ensure the restorability.
A data integrity check can only be started when there is no manual, scheduled or continuous backup job running (of the corresponding backup set), and vice versa. It is highly recommended to temporarily disable the backup schedule to ensure that no scheduled backup is started while the data integrity check is running.
After each data integrity check, the storage statistics will be refreshed.

There are two (2) options available:

Unselected "Run Cyclic Redundancy Check (CRC) during data integrity check" option: For checking the index only.
Selected "Run Cyclic Redundancy Check (CRC) during data integrity check" option: For checking index and integrity of the files against the checksum file generated at the time of the backup job.

When you start the data integrity check, %edition_name% will do the following operation:

Unselected "Run Cyclic Redundancy Check (CRC) during data integrity check"

%edition_name% will perform a comparison of the files/folders on the backup destination(s) with the list of files/folders recorded in the current index file. If the data integrity check finds:

There are files/folders in the backup destination(s) which do not exist in the current index file, then the extra files/folders will be deleted from the backup destination(s). For example: If the backup process is interrupted before completion, there will be some incomplete uploaded files or partially uploaded files.
There are files/folders listed in the current index file which do not exist in the backup destination(s), then the entries for the extra files/folders will be deleted from the current index file.
The current index file is corrupted, then the corrupted index file will be deleted and %edition_name% will replace it with the index files from the previous backup job or snapshot, therefore the file(s)/folder(s) backed up in the current backup job or current snapshot will be deleted from the backup destination and no longer be recoverable. %edition_name% will try to upload these files again in the subsequent backup job if they still exist on the clients machine.

Selected "Run Cyclic Redundancy Check (CRC) during data integrity check"

%edition_name% will perform check on the integrity of the files on the backup destination(s) against the checksum file generated at the time of the backup job. If there is a discrepancy, this indicates the file(s) on backup destination(s) are corrupted, and then %edition_name% will remove these files from the backup destination(s). If these file(s) still exist on the client machine on the next backup job, %edition_name% will upload the latest copy.

Please kindly note that:

If the CRC (Cycle Redundancy Check) option is enabled, backup data will be streamed from the backup destination (e.g. the cloud storage location or FTP location for example), to the client computer during the CRC check. For user with metered Internet connection, pay close attention to the data charge fee during a data integrity check if CRC is enabled.
The time required to complete a data integrity check depends on a number of factors, such as the number of files / folders in the backup set(s), bandwidth available on the client computer, hardware specifications of the client computer such as the disk I/O and CPU performance, and if there are other resource intensive job running. So during a data integrity check with CRC enabled, pay attention to the resource usage on the client computer.

To perform the data integrity check:

By default, the Data Integrity Check Test Mode screen will only prompt for confirmation of applying the changes if one of the following requirement below matches:

When the deleted backup files are 1000 and above.
When the deleted backup file size is 512MB and above.
When the deleted backup files are 10% above of the total backup data.
When the deleted backup file size is 10% above of the total backup data.

Select a backup set from the drop down list. You can choose a specific backup set or All (the default selection).
!

Please kindly note that:
- If you select "All", all backup sets except for the backup sets which are Run on Server (For example: Office 365 and Cloud File Backup Sets) and all destinations will be checked but this will take longer to complete depending on the number of backup sets and destinations.
- If you select a particular backup set, you can select a particular destination or "All" backup destination to check.
Click on the "Run Cyclic Redundancy Check (CRC) during data integrity check" option if required to verify the integrity of the data in the backup destination. This will require more time to complete.
Click [Start] to begin.
In case you need to stop the progress, press the [Stop] button to quit.
When the data integrity check is completed, the following TEST MODE page (preview mode) will be shown:
- If the "Statistics" status indicates "Correct", it means "Data Integrity Check is completed successfully" and there are no corrupted index or data found in the backup set(s), you can click on the [View log] button to check on the logs for details, or click on [Close] to quit.
- If the "Statistics" status indicates "Incorrect", it means there is discrepancy between backed up files and the checksum file generated at the time of the backup job and or a issue with the index files. You can also check the value of "Items found in index" and "Data corrupted items" to see details of the discrepancy between backed up files and the index file.
  - For the situation that there are files/folders in the backup destination(s) which do not exist in the current index file, such as some incomplete uploaded files or partially uploaded files remaining in the backup destination(s) caused by interruption of the backup process before completion:
    - You can click on [Yes] to delete the corrupted data items, checksum incorrect items and index broken data blocks. If the deleted file(s) still exist on the client machine on the next backup job, %edition_name% will upload the latest copy.
    - You can click on [No] so no action will be taken, and the corrupted items, backup or restore problems will NOT be fixed.
    - You can click on the [View log] button to check the according log, such as "1519821703318, 2018-02-28-12-41-43,INFO,"Removing file \"1518797253825/blocks/2018-02-23-09-05-00/0/000982.bak\" from destination because it does not exist in index"" .
  - For the situation that there are files/folders listed in the current index file which do not exist in the backup destination(s):
    - You can click on [Yes] to delete the additional index entries. If the deleted file(s) still exist on the client machine on the next backup job, %edition_name% will upload the latest copy.
    - You can click on [No] so no action will be taken, and the corrupted items, backup or restore problems will NOT be fixed.
    - You can click on the [View log] button to check the according log, such as "1520578431424,2018-03-09-14-53-51,INFO,Utilities,"Removing backup file \"C:\\Users\\Administrator\\Desktop\\in case backup of ahsaycbs\\build\\engine-framework\\custom-obm\\app\\common\\bin\\cbCoreRes_sl.properties\" from index because it does not exist in destination"".
  - For the situation that the current index file is corrupted:
    - You can click on [Yes] to delete the corrupted files. %edition_name% will replace it with the index files from the previous backup job or snapshot, therefore the file(s)/folder(s) backed up in the current backup job or current snapshot will be deleted from the backup destination and no longer be recoverable. If the deleted file(s) still exist on the client machine on the next backup job, %edition_name% will upload the latest copy.
    - You can click on [No] so no action will be taken, and the backup and restore problem will NOT be fixed.
    - You can click on the [View log] button to check the according log, such as "1520837556138,2018-03-12-14-52-36,INFO,Utilities,"Index files are corrupted. Download valid index files from backup job \"2018-03-09-18-30-35\"" and "1520837563454,2018-03-12-14-52-43,INFO,Utilities,"Removing backup file \"C:\\Users\\Administrator\\Documents\\en\\BS_Create_VMware.html\" from index because it does not exist in destination"".
Click [Close] to quit.

It is strongly recommended to perform the Cyclic Redundancy Check (CRC) regularly to ensure the data integrity of the backup data files and clear out the incomplete files from backup destination.