Slow K1000 Performance After Upgrade to 5.5
Anyone else upgraded their K1000 to the 5.5 release yet? We have a decent size environment (~4,900 clients - Windows / Mac mix) running in vSphere 5.1. System running very sluggish. First noticed when nightly maintenance went from under 1 hour to over 3 hours. Most inventory related queries are very slow. We have a support ticket open and while the team has been helpful in pointing out a few things on our side (e.g. some SmartLabel SQL queries which were taking a lot of time, trimming our asset history, etc.) we're still left with a feeling that something's not quite right.
BTW, I just noticed when updating my support ticket that the K1000 code version on https://support.kace.com/common/about.php is v5.3.47927. Perhaps I shouldn't be so gung-ho to upgrade ;-).
Answers (6)
There have been many hotfix patches supplied by KACE support. Make sure you mention the exact problems to support and have them escalate. Known patches that have been given out to resolve these issues are. k1_hotfix_5.5_K1_13891_script_quoting_20130910.kbin k1_hotfix_5.5_K1_14050_HD_slowdown_20130909.kbin k1_hotfix_5_5_K1_14047_HD_column_sort_20130912.kbin k1_hotfix_5.5_K1-13919_20130830_UI_Freeze.kbin
We are seeing a noticable improvement after KACE Support applied a 'developer patch' to our system. This patch is reported to improve us performance dealing with software inventory metadata. Our client sync rate per hour have jumped back up to where they have been prior to 5.5. Nightly maintenance and backup routine is now running in about a hours time. Received our first MIA report since the upgrade. So things are looking up.
Comments:
-
What patch did they give you? - petelanglois 11 years ago
-
KACE support did not provide it to us as a self-applied patch, I believe they made a direct code change at the request of the developers. - joncutler 11 years ago
-
you can see what they applied in the log under updates I believe - petelanglois 11 years ago
-
@petelanglois nothing recent listed under System -> K1000 Settings -> Logs -> Updates. Last 'update' was the Sizereport.txt KBIN they had me run on 9/11/2013. So whatever they did, it must have been via the SSH tether. - joncutler 11 years ago
I have applied 3-4 hotfixes to resolve my sluggish issues.
- k1_hotfix_5.5_K1_13891_script_quoting_20130910.kbin
- k1_hotfix_5.5_K1_14050_HD_slowdown_20130909.kbin
- k1_hotfix_5_5_K1_14047_HD_column_sort_20130912.kbin
- k1_hotfix_5.5_K1-13919_20130830_UI_Freeze.kbin
Those are the ones I was given to get me back to near 5.4 speed. Make sure you ask your support person about the hotfixes.
Comments:
-
I was having a lot of issues deploying anything from 5.5 after Kace L3 engineers review the logs and determined nothing was wrong. Luckily we have a snapshot of DB files backed up so i completely deleted the Kace VM itself, redeployed the template, upgraded to 5.5 and restored my back ups. Kace is now working 100% as intended.
I am assuming there was some underlying issue that no one could see and that reset fixed it. - areiner 11 years ago -
@petelanglois, so far KACE Support has only suggested k1_hotfix_5.5_K1-13919_20130830_UI_Freeze.kbin. I'll ask about the other 3 this morning. Thanks. - joncutler 11 years ago
-
@areiner, we do have a VMware snapshot as well as offline .GZ and .TGZ backups of our pre-upgrade state. I'm reluctant to call retreat just yet until we hear more from KACE, but glad to hear that you are happy with the results of going this route. - joncutler 11 years ago
-
Hi! Did you receive those packages from KACE and if you did had they any effect on the system performance? I´m having similar problems with our system. First we had the UI freeze thing and now the operational speed is just awful. Seems that the 5.5 has some issues still. - Putkonen 11 years ago
-
Yes those are hotfixes from KACE. You need to open up a support request with exact issues you are having and they will give you the hotfixes that apply to your issues. There may be more hotfixes out there but they are case by case. - petelanglois 11 years ago
Support had me install about 8 different patches and now the UI runs terrible! Any page that is accessing the db takes about 5 mins to load. This includes any computer and or software inventory and assets.
Not to mention that 5.5 broke all of our software licensing with the new software catalog. This Kace product... I love the 'capabilties' that it could have, but it's starting to become a $9000/year paper weight. I had to kill all my patch jobs with the 5.5 update as well since they broke. Dell, please just give me one 'good' update so I can get it working, but I'm never upgrading again. As of right now, this Kace appliance is becoming a security hole in our architecture and I'm about to rip it out of our organization.
Apply this update
http://www.kace.com/support/resources/kb/solutiondetail?sol=SOL116901
KACE Support still struggling with this issue. We applied update on 9/4/2013. Latest test was the tech used a tether to do a local file system copy (cp -r windows/ /tmp/windows) and the 1.3GB of data. This copy on tech's dev server took less than 1 minute. On our production host it took about 11 minutes to perform same copy. They asked us to check disk resources and we found that we still were holding on to a vSphere snapshot of the KBOX pre-5.5 upgrade. We removed that snapshot and copy speed increased to 8 minutes. Support asked us to check system resources (e.g. RAM and CPU). We were at 4GB RAM and bumped that to 6GB, copy speed improved to 5 minutes. We just completed bumping RAM to 8GB and going to do another local copy test.
Earlier reboot took over 90 minutes to complete with significant delays on DB table consistency check for tables IMMEDIATE FOLLOWING:
KBSYS.PATCHLINK_PACKAGE_LANGUAGE_JT
ORG1.MACHINE_FIELD_DEFINITION
ORG2.USER_KEYS
Anyone familiar with the mySQL internals on ver 5.5 as to what are the next tables and thoughts about why they should take so long to check during a normal reboot?
JBC
Has support had you apply any K1000 hotfixes yet? Ask about that if not.
Just to play devil's advocate, have you ruled out any ESXi issues? Like running out of space or having other resource issues like over allocation?
Comments:
-
@jknox, thanks for your suggestions. We have applied k1_hotfix_5.5_K1-13919_20130830_UI_Freeze.kbin right after upgrade and we opened the support ticket with Kace Support. Just to confirm ESXi, we are at the latest 5.1.x version of vSphere, running latest VMware tools and have 300GB out of available 500GB disk space. System performance was quite alright under 5.4SP1 with no sluggishness. - joncutler 11 years ago
We have a little over 300 computers. We use the Service Desk module pretty heavily. We have a lot of email rules that run and a lot of ticket searches that are run. But, nothing that we do should render the application unusable. And on a semi-regular basis (at least 2 times per week), our system crawls so slow that no one can hardly get any work done. - rmackin@hospiscript.com 10 years ago