So I was less than thrilled with the method of using ftp to pull the backups. Mostly because it had a ton of extra configuration, but more importantly the implementation of FTP was less then stellar. I have my K1000 in my DMZ and I would have had to open a lot of ports just to allow data transfer. On top of that there was no method of retaining files.
My solution was to write a power shell script that I could schedule using windows task scheduler and have the script clean up my files I no longer needed. The script copies the files from the K1000 to a location on my NAS where I use commvault to back it up at the file level to disk and tape.
This method relies on using the direct http links and using powershell to retrieve the files from those urls.
The script has two components: a .BAT file which calls the PS1 file. This is because you cannot call a powershell script directly from the task schedule so I just the BAT file to load powershell and execute the script.
The script is heavily commented and you should only need to edit files in the Variable - EDIT section. Also to note, there are two back up files, one for the Database which is fairly small and the Files which are fairly large if you are using a lot of deployments etc.
If you have any questions please feel free to comment and I will do my best to help you out.
1. - The Script: K1000Backup.ps1 - Copy this into a txt file and save it as a .ps1 in whichever location you use. Copy and paste into notepad before editing etc. Formatting does not display properly on the blog but it shows up properly in notepad.
#####################################################
#This script will allow you to set a scheduled #
#task and automatically backup your k1000 #
#please feel free to distribute but just include #
#the copyright info and url to the original forum #
#post on itninja,com #
# Copyright 2012 - Andrew Raia #
################################################################################
# Variables - EDIT #
############################ This is the FQDN of your kbox
$myKbox = "kbox.domain.com"
# This is the place you wish to store your backups. Use a UNC if you plan on running when no user is logged on.
# Also Make sure account running this script has permissions to this location
$myBackupPath = "UNC Path"# This defines how many days I want to keep on disk of each backup
$myFilesRetention = "7"
$myDBRetention = "30"# Set overwrite (set to "enabled" if you want to overwrite backups with the same name, otherwise "disabled")
$myOverwriteOption = "disabled"##########################################
# Execute Backup Retention - DO NOT EDIT #
########################################### Create useable var for the path to all file backups
$FileRetentionPath = "$myBackupPath" + "*.tgz"# Create useable var for the path to all DB backups
$DBRetentionPath = "$myBackupPath" + "*.gz"# Removes backup files in the backup path based on the number of days specified in the retention variable
echo "Please be patient while your backup directory is being cleaned..."# File Retention
echo "Files older than $myFilesRetention day(s) are being removed"
Get-ChildItem "$FileRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myFilesRetention)} | Remove-Item -Force# DB Retention
echo "Files older than $myDBRetention day(s) are being removed"
Get-ChildItem "$DBRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myDBRetention)} | Remove-Item -Force###########################
# Variables - DO NOT EDIT #
############################ This gets the current date and formats it the way the kbox does in the backup file
$myDate = Get-Date -format "yyyyMMdd"# This is the url for the database files
$urldb = "http://$myKbox/common/download_file.php?FILENAME=/kbackup/$myDate" + "_k1_dbdata.gz"# This is the path that it will store your backups in
$pathdb = "$myBackupPath" + "$myDate" + "_k1_dbdata.gz"# This is the url for the Files backup
$urlfile = "http://$myKbox/common/download_file.php?FILENAME=/kbackup/$myDate" + "_kbox_file.tgz"# This is the path that it will store your files backup in
$pathfile = "$myBackupPath" + "$myDate" + "_kbox_file.tgz"$myDBExist = Test-Path $pathdb
$myFileExist = Test-Path $pathfile###################################
# Execute DB Backup - DO NOT EDIT #
###################################echo "Please be patient while your K1000 files are being copied...."
# Check to see if the DB backup already exists, if not go ahead and copy it
if ($myDBExist -eq $False) {
echo "DB Backup is being copied"
$clientDB = new-object System.Net.WebClient
$clientDB.DownloadFile( $urldb, $pathdb)
}
# If the database already exists and overwrite is enabled then copy overwrite the existing file
elseif ($myDBExist -eq $True -And $myOverwriteOption -eq "enabled"){
echo "DB Backup already exists but is being overwritten because overwrite is enabled"
$clientDB = new-object System.Net.WebClient
$clientDB.DownloadFile( $urldb, $pathdb)
}
# If the database backup already exists and overwrite is disabled, then do nothing
else {
echo "DB Backup is being skipped because overwite is disabled and the file already exists"
}#####################################
# Execute File Backup - DO NOT EDIT #
###################################### Check to see if the DB backup already exists, if not go ahead and copy it
if ($myFileExist -eq $False) {
echo "File Backup is being copied"
$clientfile = new-object System.Net.WebClient
$clientfile.DownloadFile( $urlfile, $pathfile)
}
# If the database already exists and overwrite is enabled then copy overwrite the existing file
elseif ($myFileExist -eq $True -And $myOverwriteOption -eq "enabled"){
echo "File Backup already exists but is being overwritten because overwrite is enabled"
$clientfile = new-object System.Net.WebClient
$clientfile.DownloadFile( $urlfile, $pathfile)
}
# If the database backup already exists and overwrite is disabled, then do nothing
else {
echo "File Backup is being skipped because overwite is disabled and the file already exists"
}
##############################
# Debugging #
############################### Uncomment the following to verify your variables. You may also want to comment out execute backup sections above to skip copying.
# echo "DB URL: $urldb"
# echo "DB Path: $pathdb"
# echo "File URL: $urlfile"
# echo "File Path: $pathfile"
# echo "DB Backup Already Exists: $myDBExist"
# echo "File Backup Already Exists: $myFileExist"
# echo "FileRetentionPath: $FileRetentionPath"
# echo "DB Retention Path: $DBRetentionPath"
2. The .BAT file - K1000Backup .bat - copy this into a txt file and save it as a .bat in the same location as your powershell file. Make sure the location in this bat file points to the path of your powershell file as well as the path for the log file.
@echo off
PowerShell.exe "C:\Scripts\k1000backup\k1000Backup.ps1" >c:\Scripts\K1000backup\log.txt
3. Schedule a task to run the .BAT file according to your desired schedule. I set mine to run daily, and set it to run hidden even when a user is not logged on.
Also how do I add the set-execution policy on the script. Thanks.
PS C:\K1000\K1000Scripts> .\K1000Backup.ps1
Please be patient while your backup directory is being cleaned...
Files older than 7 day(s) are being removed
Get-ChildItem : Illegal characters in path.
At C:\K1000\K1000Scripts\K1000Backup.ps1:43 char:14
+ Get-ChildItem <<<< "$FileRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myFilesRetention)} |
Remove-Item -Force
Files older than 30 day(s) are being removed
Get-ChildItem : Illegal characters in path.
At C:\K1000\K1000Scripts\K1000Backup.ps1:47 char:14
+ Get-ChildItem <<<< "$DBRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myDBRetention)} | Remo
ve-Item -Force
Please be patient while your K1000 files are being copied....
DB Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\K1000Backup.ps1:81 char:23
+ $clientDB.DownloadFile( <<<< $urldb, $pathdb)
File Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\K1000Backup.ps1:102 char:25
+ $clientfile.DownloadFile( <<<< $urlfile, $pathfile) - shanky1972 11 years ago
$myKbox = "kbox.domain.com"
$myBackupPath = "UNC Path"
For the execution policy you can just run this once on the machine you are running the script on
Set-ExecutionPolicy unrestricted - MoranTug 11 years ago
Please be patient while your backup directory is being cleaned...
Files older than 7 day(s) are being removed
Get-ChildItem : Illegal characters in path.
At C:\K1000\K1000Scripts\k1000Backup.ps1:43 char:14
+ Get-ChildItem <<<< "$FileRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myFilesRetention)} |
Remove-Item -Force
Files older than 30 day(s) are being removed
Get-ChildItem : Illegal characters in path.
At C:\K1000\K1000Scripts\k1000Backup.ps1:47 char:14
+ Get-ChildItem <<<< "$DBRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myDBRetention)} | Remo
ve-Item -Force
Please be patient while your K1000 files are being copied....
DB Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\k1000Backup.ps1:81 char:23
+ $clientDB.DownloadFile( <<<< $urldb, $pathdb)
File Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\k1000Backup.ps1:102 char:25
+ $clientfile.DownloadFile( <<<< $urlfile, $pathfile)
DB URL: http://sdisbkacek1/common/download_file.php?FILENAME=/kbackup/20130510_k1_dbdata.gz
DB Path: \\wdisb0111\K1000BP20130510_k1_dbdata.gz
File URL: http://sdisbkacek1/common/download_file.php?FILENAME=/kbackup/20130510_kbox_file.tgz
File Path: \\wdisb0111\K1000BP20130510_kbox_file.tgz
DB Backup Already Exists: False
File Backup Already Exists: False
FileRetentionPath: \\wdisb0111\K1000BP*.tgz
DB Retention Path: \\wdisb0111\K1000BP*.gz - shanky1972 11 years ago
Please be patient while your backup directory is being cleaned...
Files older than 7 day(s) are being removed
Files older than 30 day(s) are being removed
Please be patient while your K1000 files are being copied....
DB Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\k1000Backup.ps1:81 char:23
+ $clientDB.DownloadFile( <<<< $urldb, $pathdb)
File Backup is being copied
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At C:\K1000\K1000Scripts\k1000Backup.ps1:102 char:25
+ $clientfile.DownloadFile( <<<< $urlfile, $pathfile) - shanky1972 11 years ago
Please be patient while your backup directory is being cleaned...
Files older than 7 day(s) are being removed
Files older than 30 day(s) are being removed
Please be patient while your K1000 files are being copied....
DB Backup is being copied
File Backup is being copied - shanky1972 11 years ago
(ADD)
#######################
# Set PowerShell Policy #
#######################
Set-ExecutionPolicy unrestricted
(AFTER)
# Copyright 2012 - Andrew Raia #
#####################################################
(ADD)
########################
# Revert PowerShell Policy #
########################
Set-ExecutionPolicy Restricted (or whichever mode you want)
(AFTER)
echo "File Backup is being skipped because overwite is disabled and the file already exists"
}
hope that makes sense - MoranTug 11 years ago
File C:\K1000\K1000Scripts\k1000Backup.ps1 cannot be loaded because the executi
on of scripts is disabled on this system. Please see "get-help about_signing" f
or more details.
At line:1 char:37
+ C:\K1000\K1000Scripts\k1000Backup.ps1 <<<< - shanky1972 11 years ago
#####################################################
#This script will allow you to set a scheduled #
#task and automatically backup your k1000 #
#please feel free to distribute but just include #
#the copyright info and url to the original forum #
#post on itninja,com #
# Copyright 2012 - Andrew Raia #
#####################################################
#######################
# Set PowerShell Policy #
#######################
Set-ExecutionPolicy unrestricted
###########################
# Variables - EDIT #
###########################
# This is the FQDN of your kbox
$myKbox = "sdisbkacek1"
# This is the place you wish to store your backups. Use a UNC if you plan on running when no user is logged on.
# Also Make sure account running this script has permissions to this location
$myBackupPath = "\\wdisb0111\K1000BP\"
# This defines how many days I want to keep on disk of each backup
$myFilesRetention = "7"
$myDBRetention = "30"
# Set overwrite (set to "enabled" if you want to overwrite backups with the same name, otherwise "disabled")
$myOverwriteOption = "disabled"
##########################################
# Execute Backup Retention - DO NOT EDIT #
##########################################
# Create useable var for the path to all file backups
$FileRetentionPath = "$myBackupPath" + "*.tgz"
# Create useable var for the path to all DB backups
$DBRetentionPath = "$myBackupPath" + "*.gz"
# Removes backup files in the backup path based on the number of days specified in the retention variable
echo "Please be patient while your backup directory is being cleaned..."
# File Retention
echo "Files older than $myFilesRetention day(s) are being removed"
Get-ChildItem "$FileRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myFilesRetention)} | Remove-Item -Force
# DB Retention
echo "Files older than $myDBRetention day(s) are being removed"
Get-ChildItem "$DBRetentionPath" -recurse | where {$_.Lastwritetime -lt (date).adddays(-$myDBRetention)} | Remove-Item -Force
###########################
# Variables - DO NOT EDIT #
###########################
# This gets the current date and formats it the way the kbox does in the backup file
$myDate = Get-Date -format "yyyyMMdd"
# This is the url for the database files
$urldb = "http://$myKbox/common/download_file.php?FILENAME=/kbackup/$myDate" + "_k1_dbdata.gz"
# This is the path that it will store your backups in
$pathdb = "$myBackupPath" + "$myDate" + "_k1_dbdata.gz"
# This is the url for the Files backup
$urlfile = "http://$myKbox/common/download_file.php?FILENAME=/kbackup/$myDate" + "_kbox_file.tgz"
# This is the path that it will store your files backup in
$pathfile = "$myBackupPath" + "$myDate" + "_kbox_file.tgz"
$myDBExist = Test-Path $pathdb
$myFileExist = Test-Path $pathfile
###################################
# Execute DB Backup - DO NOT EDIT #
###################################
echo "Please be patient while your K1000 files are being copied...."
# Check to see if the DB backup already exists, if not go ahead and copy it
if ($myDBExist -eq $False) {
echo "DB Backup is being copied"
$clientDB = new-object System.Net.WebClient
$clientDB.DownloadFile( $urldb, $pathdb)
}
# If the database already exists and overwrite is enabled then copy overwrite the existing file
elseif ($myDBExist -eq $True -And $myOverwriteOption -eq "enabled"){
echo "DB Backup already exists but is being overwritten because overwrite is enabled"
$clientDB = new-object System.Net.WebClient
$clientDB.DownloadFile( $urldb, $pathdb)
}
# If the database backup already exists and overwrite is disabled, then do nothing
else {
echo "DB Backup is being skipped because overwite is disabled and the file already exists"
}
#####################################
# Execute File Backup - DO NOT EDIT #
#####################################
# Check to see if the DB backup already exists, if not go ahead and copy it
if ($myFileExist -eq $False) {
echo "File Backup is being copied"
$clientfile = new-object System.Net.WebClient
$clientfile.DownloadFile( $urlfile, $pathfile)
}
# If the database already exists and overwrite is enabled then copy overwrite the existing file
elseif ($myFileExist -eq $True -And $myOverwriteOption -eq "enabled"){
echo "File Backup already exists but is being overwritten because overwrite is enabled"
$clientfile = new-object System.Net.WebClient
$clientfile.DownloadFile( $urlfile, $pathfile)
}
# If the database backup already exists and overwrite is disabled, then do nothing
else {
echo "File Backup is being skipped because overwite is disabled and the file already exists"
}
########################
# Revert PowerShell Policy #
########################
Set-ExecutionPolicy Restricted
##############################
# Debugging #
##############################
# Uncomment the following to verify your variables. You may also want to comment out execute backup sections above to skip copying.
# echo "DB URL: $urldb"
# echo "DB Path: $pathdb"
# echo "File URL: $urlfile"
# echo "File Path: $pathfile"
# echo "DB Backup Already Exists: $myDBExist"
# echo "File Backup Already Exists: $myFileExist"
# echo "FileRetentionPath: $FileRetentionPath"
# echo "DB Retention Path: $DBRetentionPath" - shanky1972 11 years ago
powershell -noprofile Set-ExecutionPolicy Unrestricted
then after the ps1 is called
powershell -noprofile Set-ExecutionPolicy restricted
if that one doesn't work try this format
powershell -command "& {Set-ExecutionPolicy Unrestricted}" - MoranTug 11 years ago
@echooff
powershell -noprofile Set-ExecutionPolicy Unrestricted
powerShell "C:\K1000\K1000Scripts\k1000Backup.ps1" >C:\K1000\K1000Scripts\log.txt
powershell -noprofile Set-ExecutionPolicy restricted
Thanks again. you made my day. - shanky1972 11 years ago
http://$myKbox/common/download_file.php?FILENAME=/kbackup/$myDate" + "_kbox_file.tgz" - MoranTug 10 years ago
We had to change in the Preferences -> Systemcontrol? -> Security
the setting: Secure Backup Data - Karre 10 years ago
Also, the update changed the default time that the KACE runs its backup to 22:00 (in our environment anyway). The script is looking for files with the current date, so if you are scheduling this script prior to 22:00, the files won't be there. You need to have the script set to run on the same day AFTER the backup is complete. The time KACE runs the backup can be adjusted in Settings>Control Panel>Backup Settings.
Thanks MoranTug for putting this together! Maybe 7.0 will be the version where we no longer need it... - Fletchdust 10 years ago
Nope...never got a chance to revisit this. I may need to contact KACE (Dell) Support to expedite getting this setup. - joaxe 9 years ago
Joe OC - JoeOC 9 years ago