Difficulties with AdminUI becoming unavailable looks to be caused by rogue backup at 2:00AM.
Hello,
Last night and the night before last I had some difficulty with KACE's auto-update and backup features. At about 2:00 AM our web GUI became unresponsive and wouldn't load and our KBOX's HDD and CPU activity skyrocketed in our VMWare.
Today looking at the logs we noticed this:
[Thu Oct 11 02:01:38 CDT 2012] [notice] BACKUP: Begin KBOX Backup...
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Stopping webserver...
[2012-10-11 02:01:38 -0500] KBOX [info] Inventory received after 4459 seconds, but inventory interval is 7200. (MA300255-D 50.78.7.30 6CA0044F-3F1E-4087-AF97-65F1493B9EA5)
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Processing uploaded files...
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Processing Entire Database...
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: DB Maintenance Complete.
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Starting webserver...
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Complete.
Last night was a late night, we were deploying our software all over the east coast, while the logs indicate that the backup was complete at 2:01 AM AdminUI (and all KACE management services, like inventory, file syncs and scripting) was not available for ~45 minutes. Between 2:01 and 2:45 we were completely KACE-less, our file syncs were messed up and a script that was verifying the receipt of our software on 100 or so computers was interrupted.
I looked at our backup settings and they are as follows:
K1000 Backup script schedule
Run every day at: 0:00
We need our KBOX to be fully functional throughout the night when we are deploying our software, it can't be backing itself up, or downloading updates, it needs to be completely devoted to making sure that scripts are getting run and file syncs are happening.
Is there a way I can disable all auto-update and auto-backup features? Let me know if I can give you any more log information, I've been looking pretty hard for an answer and am waiting on a response from support.
And, thank you for your time :)
jobla
Answers (1)
You must be at least version 5.2 on your kbox and all RSA's correct. https://kace.uservoice.com/forums/82699-k1000/suggestions/2010513-allow-control-of-the-backup-schedules
K1000 appliance automatically runs the backup script at 2AM. Sounds like it is reverting back to defaults also, try changing your backup time to 0:01 for your appliance and all rsa's
http://www.kace.com/support/resources/kb/article/How-to-manually-run-the-backup-script-and-retrieve
Comments:
-
Hmm, I am on version 5.2, here's from my appliance:
Automatic K1000 Upgrade
Current Version: 5.2.38773
And it's still giving me problems why would changing my backup time to 0:01 make any difference? I don't want it at 0:00 or 2:00, I want it completely off when I'm deploying software. : / - jobla 12 years ago-
you said you had it set at 0:00, The software may be having problems due to a bug of using all zero's as the scheduled time so it is reverting back to factory default of 2am. Try making it run at a different time and see if the 2am problem goes away. - SMal.tmcc 12 years ago
-
During the backup process the web server is shutdown to prevent the K1000 from being accessed. There is no way to disable the backup process from running, only the ability to set the backup schedule time. Backup files are critical in the event of disaster recovery, so I not sure why you want to not have backup files that you can off load on a daily bases. - KevinG 12 years ago
-
@SMal.tmcc Ah, I see, I misunderstood your first response, will try that.
@KevinG It didn't occur to me that I'd need backup anything on our KBOX every night, but I've never had to recover from a disaster. If I miss a day of backup files could I not use the previous day's backup files? In the webgui I see this:
20121012_k1_dbdata.gz (206.58 MB)
20121011_k1_dbdata.gz (205.90 MB)
20121010_k1_dbdata.gz (204.87 MB)
20121009_k1_dbdata.gz (203.92 MB)
20120930_k1_dbdata.gz (199.46 MB)
20120831_k1_dbdata.gz (184.95 MB)
It seems like if I were missing one of those .gz files and disaster struck I'd be able to use any of the other ones.
On late nights, once-a-month (max), when I'm trying to get something done like rolling out windows updates or a new version of our software I feel like making sure nothing interrupts my ability to manage our computers would be higher priority than backing up our KBOX. But I concede that disaster recovery isn't something I've had to deal with yet. - jobla 12 years ago -
I might also add that in the log file it says:
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Starting webserver...
[Thu Oct 11 02:01:38 CDT 2012] [notice] Backup: Complete.
But we weren't able to access our adminui page until ~2:50 am - jobla 12 years ago -
my guess is this is part of the entire process kace goes thru each night. It is like when you install a programs it says it is 100% complete but takes 5 minutes before that closes. I think you are only seeing the logs for a single part of the process. since you do rare late nights you may want to make the backup happen a 6am. - SMal.tmcc 12 years ago
