Introduction
Delta Watch (DW) uses a local database (DB) to store things like imported and processed data. After importing a data file, the original file will also be stored as a zip archive in the local archive folder of the relevant project for save keeping. Having a local DB allows DW to easily make DB backups and manage these on a rotational basis. But over time, the DB size does grow and requires more disk-space. For example, the DB can grow by adding new data or reprocessing existing data into smaller intervals. When data is deleted from within DW then it is not actually removed but rather marked to not be used. To permanently remove this deleted data and shrink the DB, a special utility will need to be used. Additionally, it is also possible to clean up the DB by removing raw data and only keeping the calculated results. This article covers some methods of managing the DB, these include making and creating manual backups as well as size reduction options.
What is in this article:
- Introduction to DB management utility.
- Configure automatic DB backups.
- Manually creating a backup.
- Restoring a backup.
- Reducing DB size.
- Space calculation example.
TUNIS and SCHEDULER files Whenever an action is performed on the TUNIS file, either making or restoring, the same action should be performed on the SCHEDULER file. The SCHEDULER file contains critical information about things like automation and reporting. Failing to keep this file might result in loss of configuration of your project! |
DB management utility
Delta Watch comes with a DB management utility called Tunis.Utils.exe. It should be used to create or restore a backup of the Delta Watch DB. When it is making a backup, it can also reduce the DB size by permanently removing deleted data. This utility can be found in the DW program folder. On a default installation, this is:
- [drive]:\Topcon Delta Watch\Tunis.Utils.exe
For security reasons, to be able to use the utility, the following login details are to be used:
User: | dbmaintenance | |
Password: | uriGel45 |
These details are for DW versions 3.10 and higher, please contact support if you have an earlier version of Delta Watch.
The main interface, as seen on the right, gives access to the various functions. The ones used in this article are Create backup and Restore backup.
If Delta Watch is configured to make regular backups of the DB on a user configured location, then, when required, any of these DB's could be used for restoring purposes. By default, these backup files will be in a subfolder of the Delta Watch program called \Backup:
[drive]:\Topcon Delta Watch\Backup
Depending on how the DB backup was created, the file name and extension might differ. But all can be restored via the same methods as discussed later on. The DB could have a filename extension of either .bak or .bku, and the filename could look something like one of the below, depending on how the backup was created:
- tunisdb_20211205183059.bku
- tunisdb_firebird_20211205181529.bku
- tunisdb_manual_20211205181013.bku
Configure Delta Watch for automated backups
Delta Watch can create regular DB backups on a rotational basis at a user specified interval and start time. It also allows creating a second copy of the backup, which can be placed on an off-site location for extra security. Backup intervals of less than 24 hours will put additional workload to Delta Watch and should be avoided, considering it is very fast to re-process just the last 24 hours of data again in case a backup does need to be restored.
To configure the automated backup, open the System Configuration module in the Administration group and locate the item Automatic backup in the tree. Configure the below minimum fields.
Field | Description |
---|---|
Automatic backup | Set to Enable |
Start time | The date and time when the first backup should happen. Set the time-value to a period of the day when little access to DW is required. |
Interval | At what interval, starting from Start time, new backups should be created. Value is in minutes, and a good starting point is 1440, which is every 24 hours. |
Directory | Location, on local drive, where the backups will be stored. |
Number of old backups to keep | If the number of backups specified here is reached, the oldest backup will be deleted to make space for the next backup. |
Field | Description |
---|---|
Additional copy | Set to Enable if a second copy should be made, this is usually to an off-site or physical different drive. |
Directory | Same as per above |
Number of old backups to keep | Same as per above |
Once configured, click on Apply in the toolbar. If required, a manual backup could be started. How, see the section Create backup using Delta Watch, and study the section on Space requirements.
Manually creating a DB backup
There are 2 ways of manually creating a DB backup.
- Using Delta Watch
- Using Tunis.Utils.exe program
When using Tunis.Utils utility, there is the option to reduce the DB size by permanently removing deleted records, and well as being able to specify a custom source DB.
Create backup using Delta Watch
The easy way to create a manual backup is to use Delta Watch. Even though automatic backups are configured, there might sometimes be the need to trigger a manual backup. Maybe big changes are going to be made to a project, and for safety reasons a backup is required.
To trigger the backup process, select the Help item in the menu bar and then select Database backup…, see on the left in below illustration. Depending on the size of the DB this could take a while. When the process is completed, a message will be written to the Message archive and also shows in the message bar in the bottom right of Delta Watch, as per example in the below illustration on the right.
Beware that when creating a backup from within Delta Watch, as per above, only the TUNIF.FDB database is backed up. The user would still manually have to create a backup from the SCHEDULER.FDB file by either:
|
Create backup using Tunis.Utils program
The DB management utility, as described above, should be used when additional actions have to be performed. Such as permanently removing deleted records from the DB or deleting messages from the message archive.
|
Restoring a backup
Restoring a DB backup might need to be done for reasons such as:
- Accidental loss of data; User deleted wrong data.
- Computer operating system issues, requiring a re-installation of complete system.
- Management action like DB clean-up and size reduction.
The backup might have been created by any of the below methods:
- Automated procedure as configured by the user in DW.
- Manual backup triggered from within DW via the menu About.Database backup....
- Manual backup performed using Tunis.Utils program.
The following steps will explain the process of restoring an existing DB backup using the Tunis.Utils program.
- Close Delta Watch and start Tunis.Utils utility.
- Select Restore backup, see illustration on the right.
- Select the DB to restore and click on Next.
- Select where to restore the DB and specify the target name. Beware to not overwriting an existing file, a small icon with a warning triangle will show if that might happen. The default location is the same folder as the source file, but with the file extension now being FDB. Accept this and click Next, see also the note further down.
- Progress of restoring the DB will be shown on the screen and when done click Finish.
- If there was a matching backup of the SCHEDULER DB, then repeat steps 2 to 5 for this file.
- Close the program.
If at step 4 above the DB was not saved in the \Database folder with the correct name for Delta Watch, either TUNIS.FDB or SCHEDULER.FDB, then it needs to be manually moved and renamed before it can be used.
- Using Windows file explorer, navigate to the \Database folder of Delta Watch. On a default installation, this can be found on: [drive]:\Topcon Delta Watch\Database
- Rename; move or delete the current TUNIS.FDB and SCHEDULER.FDB files. These are the current Delta Watch DB files.
- Move the restored DB, from step 4 above, into the \Database folder. The filename might look something like tunisdb_firebird_20210302113853.fdb.
- Once moved, rename the file to: TUNIS.FDB
- Repeat step 10 and 11 but for the SCHEDULER file if applicable
- Start Delta Watch
When restoring a DB, it is possible to restore directly to the \Database folder of Delta Watch. When doing this, consider what need to happen with this existing DB before starting the restore process and accidentally overwrite any files. |
Reducing DB size
Databases are great, but they tend to grow the more we put into it, and when you delete something it is not always really deleted. The same is true for a Delta Watch DB. It is possible to reduce the size of the DB, by how much, depends on the information it contains.
We can reduce the Delta Watch DB with 2 different actions:
- Removing raw observation data from epochs
- Permanently removing delete data
When DW processes raw geodetic data and converts that to coordinates, it does keep the raw data in the DB but also stores things like the network adjustment results. On a long-running project, over time, the storing of these raw data values will add to the growing size of the DB. If there is no need to reprocess older data then, at regular intervals, removing the raw data from the DB will help in keeping the required disk space to a minimum. This is called: cleanup data in Delta Watch.
When performing a cleanup data action in DW, it marks all unnecessary information of the selected time range that it can be deleted. This does not remove anything yet, that happens when Tunis.Utils is used to permanently delete the data. Follow the below steps to perform the cleanup data action, there is a slight difference depending on which version of Delta Watch is used.
> |
|
Tunis.Utils steps (all versions)
- Using Tunis.Utils, create a manual backup as described above in section Create backup using Tunis.Utils program. Make sure to Enable the option to Skip garbage collection, as per illustration.
- After completion of the backup process, restore the created backup as per instructions under Restoring a backup. The new TUNIS.FDB will have decreased in size.
Space requirements
It is important to make regular assessments of the space requirements on the monitoring server. The monitoring DB is growing on a daily basis and Delta Watch needs a certain amount of free disk space to be able to continue processing, but also to be able to create and maintain the backups of the DB.
When Delta Watch creates a DB, it first creates it on a temporary location. Once created, the DB is copied to the backup folder and then an integrity check is performed before the temporary DB file is deleted.
Further, we need to take into account the number of DB backups to keep as configured in the System Configuration module. All-in-all, this requires a certain disk space on top of the space needed for the active DB.
It is advised to have a minimum of 150GB free space, and for many of the projects this should be enough. But it is good practice to keep an eye on the available space and perform regular DB maintenance to help further reduce the space requirement.
To calculate the total space needed to create and maintain the backups, don’t forget to take into account any space already used by existing backups, as this will be freed when they are too old.
Working DB size |
4020 MB |
Check size of TUNIS.FDB |
Backup DB size |
3200 MB |
Check size of last backup in ‘Backup’ folder |
Number of backups |
10 |
Found in DW under section ‘Automatic backup’ in system configuration. |
Required space |
~44 GB |
(3 x Working_DB) + (NumOfBackups x Backup_DB) |