Patching schedules for servers - specific reboot times
We're using K1000 patching relatively successfully for workstations, but we have been struggling to get it working for our estate of approx. 350 servers. The main issue is around scheduling restarts - we're a healthcare organisation running 24x7, and many of our systems have very narrow downtime windows. For example we may want to restart a specific server between 2am and 2.30am, but with a deploy + restart schedule, we have no control over how long the deployment takes, so can't control when the restart occurs with sufficient precision.
The ideal situation would be to install all of the updates ahead of time with restart suppressed, and then trigger the restart during the allotted time window. However this is proving difficult, and we're currently looking at scripting solutions outside of KACE to handle the restarts. Or else ditching KACE for servers and trying WSUS or another 3rd party product.
I'm wondering if there are any other organisations out there with similar requirements, and if so how you handle it?
Things we have experimented with so far:
- A deploy run with restart suppressed earlier in the day, following by a second deploy run with forced restart during the downtime window. The thinking was that the second deploy would seeing nothing further to do except the restart, and complete quickly. This doesn't seem to work, as the second deploy schedule either does nothing, or if it does run it still does a full pass (we can see the LM.detection_64.exe process running) so the timing is imprecise.
- A deploy run with restart suppressed earlier in the day, following by a kscript to restart during the downtime window. This fails because the K1000 is in its backup phase overnight, so the script seems to be skipped. Also we're having difficulties getting scripts that perform a restart to upload their logs to the K1000. I'm loathe to try it with offline scripts as that seems risky for servers.
Any suggestions please? If you have narrow downtime windows how do you handle it? 350 servers is too many for our small team to manage the restarts manually.
Thanks.
0 Comments
[ + ] Show comments
Answers (1)
Please log in to answer
Posted by:
ondrar
6 years ago
KACE scripts that have the computer reboot or otherwise terminate the agent service aren't going to return results well, because the process that runs the script is ended, and so the script execution is ended, too. I had an example like that as well.
I would suggest using the built in Windows Task Scheduler to set up a nightly reboot during your downtime window. with the patches already having been installed from the earlier one-pass deploy job. I think the Task Scheduler would be able to perform that restart more reliably than a KACE script.
https://www.prrcomputers.com/blog/automatic-reboot-in-windows-7/
You can even create the job once, then export and import it on the other servers.
https://www.askvg.com/how-to-import-export-backup-restore-tasks-using-task-scheduler-in-windows/
Or create the reboot task in Group Policy to apply it to all your servers at once, if they're all grouped in AD.
Comments:
-
Thanks, we've tried using a Task Scheduler job pushed out through Group Policy and controlled by AD groups (creating and administering 350 task scheduler jobs individually on each server sounds like an accident waiting to happen). We set up Server Group A with one restart schedule, Server Group B for a different day/time, and so on.
Unfortunately there was a fundamental flaw in our plan, which is that the server needs to be rebooted to pick up group membership, or also if we want to change its group. (There's a workaround floating around the web about purging tickets using klist.exe, which forces a refresh of the server's tokens to update its group membership, but we're a bit hesitant about the impact of using that on production servers used by thousands of users.)
We're currently looking at a simple powershell restart script reading server names from a text file, and running that in task scheduler. But it all feels a bit makeshift and I can't help wondering if KACE offers more control but we're not using it correctly... - john_u 6 years ago-
I don't believe KACE has native reboot options, but if it does, I'm missing it, too.
What if you move the KACE backup back so that it's done by the time the reboot script is supposed to run?
To get a result, since the agent won't always report it back, you could add a breadcrumb in the script, then read it with a Custom Inventory Rule, Smart Label, or report. - ondrar 6 years ago
