Mac agent stops, Macs stop reporting in
We are a mixed MS/Mac/Linux shop (mostly MS, and I'm an MS admin) and added 18 Macs to the K1000 a few months ago. They have since stopped responding and this appears to be related to the konea process stopping intermittently, sometime between 2 and 37 minutes after starting.
Here's the timeline of events:
- Added Macs to old ds-kace01.domain.local, everything checks in consistently, no issues.
- Added DNS entries to point to kace.domain.com.
- Ran this script to change Macs over to new hostname: (http://www.itninja.com/question/script-to-change-kace-appliances-for-mac-amptools). Specifically, this (yes, my reading comprehension sucked that day; I should have read chucksteel's comment first):
#!/bin/bash
# redirects a KACE client to pcckboxdev - rgetter 070215
cd ${0%/*}
mv -f amp.conf /Library/Application\ Support/Dell/KACE/data
# restart the AMPAgent
/Library/Application\ Support/Dell/KACE/bin/AMPTools stop
/Library/Application\ Support/Dell/KACE/bin/AMPTools start
- Macs stop reporting. PCs continue to report.
- Changed hostname on K1000 to kace.domain.com.
- Research further, noticed I missed chucksteel's comment, and also received heads up from Kace support. Disabled the first script, then ran this script manually on all Macs. Verified that amp.conf and amp_auto.conf were updated to point to the new (correct) K1000.
- Macs still not reporting in; notice that the konea agent is not running. Run sudo /Library/Application\ Support/Dell/KACE/bin/AMPTools start to start the AMPAgent and konea processes and a given Mac will report in and update inventory; however, after an intermittent period of time (shortest has been 2 minutes, longest was 37), konea stops and the Mac goes "offline" again.
From the system diagnostic report, we pulled this:
Process: AMPAgent [65]
Path: /Library/Application Support/Dell/*/AMPAgent
Identifier: AMPAgent
Version: ???
Code Type: X86-64 (Native)
Parent Process: launchd [1]
Responsible: AMPAgent [65]
User ID: 0
Date/Time: 2016-06-28 14:17:43.174 -0600
OS Version: Mac OS X 10.11.4 (15E65)
Report Version: 11
Anonymous UUID: B17358A5-16F0-0418-4B07-706148E53782
Sleep/Wake UUID: 37163E51-5BC4-4F10-845D-43DF50A5A0B0
Time Awake Since Boot: 1100 seconds
Time Since Wake: 120 seconds
System Integrity Protection: enabled
Crashed Thread: 1
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: EXC_I386_GPFLT
Exception Note: EXC_CORPSE_NOTIFY
Thread 0:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x0000000100f95eca kevent + 10
1 AMPAgent 0x0000000100132b1e kq_dispatch + 478
2 AMPAgent 0x000000010012261e event_base_loop + 734
3 AMPAgent 0x000000010012233a event_base_dispatch + 26
4 AMPAgent 0x000000010000cc64 NimitzAgentProcess + 548
5 AMPAgent 0x00000001000106c9 main + 108
6 AMPAgent 0x0000000100000fe4 start + 52
Thread 1 Crashed:
0 libsystem_pthread.dylib 0x00000001010ea45c pthread_mutex_lock + 0
1 AMPAgent 0x000000010001da42 AcquireLock + 14
2 AMPAgent 0x000000010000961c kpiKmsgrMsgDoneCallback + 309
3 pluginPatching.so 0x0000000101a3095c ProcessPayloadPerform(void*) + 2125
4 AMPAgent 0x000000010002a90e WorkerThreadProcess + 106
5 libsystem_pthread.dylib 0x00000001010ec99d _pthread_body + 131
6 libsystem_pthread.dylib 0x00000001010ec91a _pthread_start + 168
7 libsystem_pthread.dylib 0x00000001010ea351 thread_start + 13
Thread 2:
0 libsystem_kernel.dylib 0x0000000100f943ca __accept + 10
1 AMPAgent 0x000000010000f6eb NetListener + 183
2 AMPAgent 0x000000010000c9d4 AgentlistenerThread + 21
3 libsystem_pthread.dylib 0x00000001010ec99d _pthread_body + 131
4 libsystem_pthread.dylib 0x00000001010ec91a _pthread_start + 168
5 libsystem_pthread.dylib 0x00000001010ea351 thread_start + 13
Thread 3:
0 libsystem_kernel.dylib 0x0000000100f94c02 __open + 10
1 AMPAgent 0x000000010000d15e BootUpEventThread + 641
2 libsystem_pthread.dylib 0x00000001010ec99d _pthread_body + 131
3 libsystem_pthread.dylib 0x00000001010ec91a _pthread_start + 168
4 libsystem_pthread.dylib 0x00000001010ea351 thread_start + 13
Thread 4:
0 libsystem_kernel.dylib 0x0000000100f94c02 __open + 10
1 AMPAgent 0x000000010000d494 LogonEventThread + 710
2 libsystem_pthread.dylib 0x00000001010ec99d _pthread_body + 131
3 libsystem_pthread.dylib 0x00000001010ec91a _pthread_start + 168
4 libsystem_pthread.dylib 0x00000001010ea351 thread_start + 13
Thread 1 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000100edc050 rcx: 0x0006980000000000 rdx: 0x0000000100ed8c18
rdi: 0xbd3aaa3a2e07ce28 rsi: 0x0006980000069800 rbp: 0x000070000007c1e0 rsp: 0x000070000007c1d8
r8: 0x00000000fffffffc r9: 0x0000000100ed8c10 r10: 0x00000000ffffffff r11: 0xffffffff00000000
r12: 0x0000000000000000 r13: 0x00000000002c27e5 r14: 0x0000000103009700 r15: 0x0000000000000000
rip: 0x00000001010ea45c rfl: 0x0000000000010286 cr2: 0x00007fd2d4014000
Logical CPU: 0
Error Code: 0x02000005
Trap Number: 133
Binary Images:
0x100000000 - 0x100340fff +AMPAgent (???) <CDDBFD1C-2E37-363E-9921-FE1383725F62> /Library/Application Support/Dell/*/AMPAgent
0x100437000 - 0x100485ff7 libstdc++.6.dylib (104.1) <76E2AFB1-07E5-3F19-B692-F6E21B7E470D> /usr/lib/libstdc++.6.dylib
0x1004ee000 - 0x10054cfff com.apple.SystemConfiguration (1.14 - 1.14) <10082F58-6190-3A7C-8B6C-C12B16DC793A> /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
0x100585000 - 0x1009fbfff com.apple.CoreFoundation (6.9 - 1258.1) <943A1383-DA6A-3DC0-ABCD-D9AEB3D0D34D> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x100c7a000 - 0x100c7bffb libSystem.B.dylib (1226.10.1) <CD307E99-FC5C-3575-BCCE-0C861AA63124> /usr/lib/libSystem.B.dylib
0x100c81000 - 0x100caafff libc++abi.dylib (125) <DCCC8177-3D09-35BC-9784-2A04FEC4C71B> /usr/lib/libc++abi.dylib
0x100cb9000 - 0x100cbdfff libcache.dylib (75) <9548AAE9-2AB7-3525-9ECE-A2A7C4688447> /usr/lib/system/libcache.dylib
0x100cc3000 - 0x100cceff7 libcommonCrypto.dylib (60075.40.2) <B9D08EB8-FB35-3F7B-8A1C-6FCE3F07B7E7> /usr/lib/system/libcommonCrypto.dylib
0x100cdc000 - 0x100ce3ff7 libcompiler_rt.dylib (62) <A13ECF69-F59F-38AE-8609-7B731450FBCD> /usr/lib/system/libcompiler_rt.dylib
0x100cf1000 - 0x100cf9fff libcopyfile.dylib (127) <A48637BC-F3F2-34F2-BB68-4C65FD012832> /usr/lib/system/libcopyfile.dylib
0x100d00000 - 0x100d77feb libcorecrypto.dylib (335.40.8) <9D300121-CAF8-3894-8774-DF38FA65F238> /usr/lib/system/libcorecrypto.dylib
0x100d90000 - 0x100dbdfff libdispatch.dylib (501.40.12) <C7499857-61A5-3D7D-A5EA-65DCC8C3DF92> /usr/lib/system/libdispatch.dylib
0x100de7000 - 0x100deaffb libdyld.dylib (360.21) <8390E026-F7DE-3C32-9486-3DFF6BD131B0> /usr/lib/system/libdyld.dylib
0x100df2000 - 0x100df2ff7 libkeymgr.dylib (28) <8371CE54-5FDD-3CE9-B3DF-E98C761B6FE0> /usr/lib/system/libkeymgr.dylib
0x100df7000 - 0x100df7ff7 liblaunch.dylib (765.40.36) <1CD7619D-AF2E-34D1-8EC6-8021CF473D9B> /usr/lib/system/liblaunch.dylib
0x100dfe000 - 0x100e03ff7 libmacho.dylib (875.1) <318264FA-58F1-39D8-8285-1F6254EE410E> /usr/lib/system/libmacho.dylib
0x100e0a000 - 0x100e0cff7 libquarantine.dylib (80) <0F4169F0-0C84-3A25-B3AE-E47B3586D908> /usr/lib/system/libquarantine.dylib
0x100e13000 - 0x100e14ffb libremovefile.dylib (41) <552EF39E-14D7-363E-9059-4565AC2F894E> /usr/lib/system/libremovefile.dylib
0x100e1a000 - 0x100e31ff7 libsystem_asl.dylib (323.40.3) <007F9094-317A-33EA-AF62-BAEAAB48C0F7> /usr/lib/system/libsystem_asl.dylib
0x100e3f000 - 0x100e40fff libsystem_blocks.dylib (65) <1244D9D5-F6AA-35BB-B307-86851C24B8E5> /usr/lib/system/libsystem_blocks.dylib
0x100e45000 - 0x100ed2fff libsystem_c.dylib (1082.20.4) <CDEBF2BB-A578-30F5-846F-96274951C3C5> /usr/lib/system/libsystem_c.dylib
0x100efe000 - 0x100f00ff7 libsystem_configuration.dylib (802.40.13) <3DEB7DF9-6804-37E1-BC83-0166882FF0FF> /usr/lib/system/libsystem_configuration.dylib
0x100f07000 - 0x100f09fff libsystem_coreservices.dylib (19.2) <1B3F5AFC-FFCD-3ECB-8B9A-5538366FB20D> /usr/lib/system/libsystem_coreservices.dylib
0x100f0f000 - 0x100f26ff7 libsystem_coretls.dylib (83.40.5) <C90DAE38-4082-381C-A185-2A6A8B677628> /usr/lib/system/libsystem_coretls.dylib
0x100f30000 - 0x100f38ffb libsystem_dnssd.dylib (625.40.20) <86A05653-DCA0-3345-B29F-F320029AA05E> /usr/lib/system/libsystem_dnssd.dylib
0x100f3f000 - 0x100f68fff libsystem_info.dylib (477.40.5) <6B01C09E-A3E5-3C71-B370-D0CABD11A436> /usr/lib/system/libsystem_info.dylib
0x100f7e000 - 0x100f9cff7 libsystem_kernel.dylib (3248.40.184) <88C17B7F-1CD8-3979-A1A9-F7BDB4FCE789> /usr/lib/system/libsystem_kernel.dylib
0x100fb2000 - 0x100fe1ffb libsystem_m.dylib (3105) <08E1A4B2-6448-3DFE-A58C-ACC7335BE7E4> /usr/lib/system/libsystem_m.dylib
0x100fee000 - 0x10100aff7 libsystem_malloc.dylib (67.40.1) <5748E8B2-F81C-34C6-8B13-456213127678> /usr/lib/system/libsystem_malloc.dylib
0x101014000 - 0x10107aff7 libsystem_network.dylib (583.40.20) <269E5ADD-6922-31E2-8D55-7B777263AC0D> /usr/lib/system/libsystem_network.dylib
0x1010b2000 - 0x1010bafff libsystem_networkextension.dylib (385.40.36) <66095DC7-6539-38F2-95EE-458F15F6D014> /usr/lib/system/libsystem_networkextension.dylib
0x1010c6000 - 0x1010cfff3 libsystem_notify.dylib (150.40.1) <D48BDE34-0F7E-34CA-A0FF-C578E39987CC> /usr/lib/system/libsystem_notify.dylib
0x1010d8000 - 0x1010e0fef libsystem_platform.dylib (74.40.2) <29A905EF-6777-3C33-82B0-6C3A88C4BA15> /usr/lib/system/libsystem_platform.dylib
0x1010e9000 - 0x1010f2ff7 libsystem_pthread.dylib (138.10.4) <3DD1EF4C-1D1B-3ABF-8CC6-B3B1CEEE9559> /usr/lib/system/libsystem_pthread.dylib
0x101100000 - 0x101103fff libsystem_sandbox.dylib (460.40.33) <30671DCC-265F-325A-B33D-11CD336B3DA3> /usr/lib/system/libsystem_sandbox.dylib
0x10110a000 - 0x10110bfff libsystem_secinit.dylib (20) <32B1A8C6-DC84-3F4F-B8CE-9A52B47C3E6B> /usr/lib/system/libsystem_secinit.dylib
0x101111000 - 0x101122ff7 libsystem_trace.dylib (201.10.3) <25104542-5251-3E8D-B14A-9E37207218BC> /usr/lib/system/libsystem_trace.dylib
0x101131000 - 0x101131ff7 libunc.dylib (29) <DDB1E947-C775-33B8-B461-63E5EB698F0E> /usr/lib/system/libunc.dylib
0x101135000 - 0x10113aff3 libunwind.dylib (35.3) <F6EB48E5-4D12-359A-AB54-C937FBBE9043> /usr/lib/system/libunwind.dylib
0x101142000 - 0x10116bff7 libxpc.dylib (765.40.36) <2CC7CF36-66D4-301B-A6D8-EBAE7405B008> /usr/lib/system/libxpc.dylib
0x10118a000 - 0x1014ecf3f libobjc.A.dylib (680) <7489D2D6-1EFD-3414-B18D-2AECCCC90286> /usr/lib/libobjc.A.dylib
0x1015c9000 - 0x10160fff7 libauto.dylib (186) <999E610F-41FC-32A3-ADCA-5EC049B65DFB> /usr/lib/libauto.dylib
0x101625000 - 0x101678ff7 libc++.1.dylib (120.1) <8FC3D139-8055-3498-9AC5-6467CB7F4D14> /usr/lib/libc++.1.dylib
0x1016d5000 - 0x1016d6fff libDiagnosticMessagesClient.dylib (100) <4243B6B4-21E9-355B-9C5A-95A216233B96> /usr/lib/libDiagnosticMessagesClient.dylib
0x1016dc000 - 0x1016ecfff libbsm.0.dylib (34) <7E14504C-A8B0-3574-B6EB-5D5FABC72926> /usr/lib/libbsm.0.dylib
0x1016f6000 - 0x101903fff libicucore.A.dylib (551.51) <35315A29-E21C-3CC5-8BD6-E07A3AE8FC0D> /usr/lib/libicucore.A.dylib
0x1019ca000 - 0x1019dbff7 libz.1.dylib (61.20.1) <B3EBB42F-48E3-3287-9F0D-308E04D407AC> /usr/lib/libz.1.dylib
0x101a14000 - 0x101a16ff7 +pluginDesktopAlerts.so (???) <118CBE63-E775-3493-B2ED-B93C70791B79> /Library/Application Support/Dell/*/pluginDesktopAlerts.so
0x101a28000 - 0x101a34ff7 +pluginPatching.so (???) <6A243148-2AAC-3C1D-B7EC-6FDE150A8E78> /Library/Application Support/Dell/*/pluginPatching.so
0x101a51000 - 0x101a55fff +pluginRunProcess.so (???) <1CF96CF3-0BC0-3A9F-89FD-E34C611084C8> /Library/Application Support/Dell/*/pluginRunProcess.so
0x101a67000 - 0x101a6bfff +pluginWeb.so (???) <276F208D-5AD4-3007-817E-EEB5C3ECE1B7> /Library/Application Support/Dell/*/pluginWeb.so
0x7fff6b61e000 - 0x7fff6b6550d7 dyld (360.21) <D9B236BC-4AC1-325F-B3EF-3F06DBDA7119> /usr/lib/dyld
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 1
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 445
thread_create: 0
thread_set_state: 0
VM Region Summary:
ReadOnly portion of Libraries: Total=22.9M resident=0K(0%) swapped_out_or_unallocated=22.9M(100%)
Writable regions: Total=50.7M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=50.7M(100%)
VIRTUAL REGION
REGION TYPE SIZE COUNT (non-coalesced)
=========== ======= =======
Activity Tracing 2048K 2
Kernel Alloc Once 4K 2
MALLOC 36.2M 13
MALLOC guard page 16K 4
Process Corpse Info 2048K 2
STACK GUARD 56.0M 6
Stack 10.0M 6
VM_ALLOCATE 4K 2
__DATA 3436K 69
__LINKEDIT 4880K 53
__TEXT 18.2M 53
__UNICODE 552K 2
shared memory 12K 4
=========== ======= =======
TOTAL 133.1M 205
I've been researching the three error codes (EXC_CORPSE_NOTIFY, EXC_I386_GPFLT, and EXC_BAD_ACCESS (SIGSEGV), but haven't seen anything yet that makes sense to me.
6 Comments
[ + ] Show comments
Answers (1)
Please log in to answer
Posted by:
george.mattson@plansource.com
8 years ago
FYI to all, in the event anyone finds this at a later point; the issue turned out to be a script that was still enabled to run every 30 minutes on the Macs which reset amptools.
After disabling the script, the issue went away.
File this under Stupid User (admin) Mistakes.
I really gotta stop smoking crack... :D
I really gotta stop smoking crack... :D
Comments:
-
PEBKAC befalls us all. Thanks for the update! - JasonEgg 8 years ago
Also, we purchased another (brand new) Macbook Air some time after this all occurred. With a fresh install where kace.domain.com was (correctly) designated as the host, this issue is still occurring.
Kace tech support has been focused on the Macs, but I'm wondering if there is anything on the K1000 which might have been reconfigured incorrectly? - george.mattson@plansource.com 8 years ago
sudo /Library/Application\ Support/Dell/KACE/bin/amptools -resetconf host=kace.domain.com
sudo /Library/Application\ Support/Dell/KACE/bin/AMPTools stop
sudo /Library/Application\ Support/Dell/KACE/bin/AMPTools start - george.mattson@plansource.com 7 years ago
Some investigation revealed that the konea service was not running automatically at bootup like it is supposed to do even though 'konea.plist' was present in the /Library/LaunchDaemons directory.
I could manually start the service with:
sudo /Library/Application\ Support/Quest/KACE/bin/konea
After starting the service, the Mac would check-in as expected.
I discovered that the service had become disabled. To resolve the problem, I ran the following command:
sudo launchctl enable system/konea
After this, the agent worked properly. It appears that in the uninstallation process, somehow the konea service was disabled. And when reinstalling, the service remained disabled. - andy.gilmore 6 years ago
I've got some iMacs on Sierra that are doing the same thing. KACE agents seems to be running... and then it stops...
I tried the manual start command that Andy.gilmore left in his thread, but that didn't do anything for me. Anyone have any other suggestions or fixes for this? - melissa.arnold 6 years ago
Without knowing anything further about your environment, I'd check to see if you have any active scripts going on and/or if you have any other process pushing client updates out. You might also want to give Quest support a call. Back when I posted this, we called them the "Dell Helpless Desk," and they weren't much better when Quest initially took over, but they are significantly better now.
I would be curious, though, as to what versions you're on, on both the server and desktop side. - george.mattson@plansource.com 6 years ago
Here is my solution, hope it help you guys. (High Sierra)
Kace agent cannot start due to it cannot register to LaunchDaemon.
1. Check if konea.plist exist in /Library/LaunchDaemons/
1.1 If it doesn't exist, create kace.plist with command
sudo /Library/Application\ Support/Quest/KACE/bin/konea -install
2. If konea.plist exist run this command to register konea as a daemon
sudo launchctl load -w /Library/LaunchDaemons/konea.plist
with -w option it will permanently added to startup items.
3. Verify if konea start as a daemon with this command
sudo launchctl list | grep konea
If you've done it right this command will return something like this "608 0 konea"
4. Verify if konea service exist
ps aux | grep konea
That will return the result similar to :
root 608 0.0 0.1 558484588 11204 ?? Ss Tue03AM 0:07.15 /Library/Application Support/Quest/KACE/bin/konea
5. Stop/Start Kace agent with command :
sudo /Library/Application\ Support/Quest/KACE/bin/AMPTools stop
sudo /Library/Application\ Support/Quest/KACE/bin/AMPTools start
6. Run "runkbot" to verify connection to Kace server.
sudo /Library/Application\ Support/Quest/KACE/bin/runkbot 2 0
With runkbot command it somehow needs "konea" to be registered as a daemon first otherwise it will fail to find "KUID" - CLSMCSMII 6 years ago