This post is more than 5 years old
3 Posts
0
1219
July 14th, 2010 05:00
W2K8 host agent stops shortly after starting
Hi,
I've got about 8 host that are running W2K8 and all the host agents will go inactive shortly after starting. I have other W2K8 servers that are working fine. ECC is 6.1
this is in the mnragent_FATAL.log
Thread 21816 (failed thread):
Logical addr Function
0001:0001F35F (kernel32.dll) RaiseException
0001:00018AED (MSVCR71.dll) _CxxThrowException
0001:003F1AC2 (mnragent.exe) WriteFromErrorTS
0001:00042094 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:00041F10 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:00026D5E (mnragent.exe) MNRAUXEmptyRecycleBin
0001:00026891 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:000384D7 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:000104F2 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:001B694B (mnragent.exe) MNRAUXEmptyRecycleBin
0001:001C60C1 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:001C593F (mnragent.exe) MNRAUXEmptyRecycleBin
0001:001DC8E5 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:001CDC85 (mnragent.exe) MNRAUXEmptyRecycleBin
0001:00008565 (MSVCR71.dll) _endthreadex
0001:0007E3F3 (kernel32.dll) BaseThreadInitThunk
0001:0006CFED (ntdll.dll) RtlCreateUserProcess
0001:0006D1FF (ntdll.dll) RtlCreateProcessParameters
Any ideas?
Thanks in advance
ConorLawlor
2 Intern
•
233 Posts
0
July 14th, 2010 07:00
Hi Imajica,
For a test can you disable the three Watermarks DCPs " Watermarks for Volume Groups, Watermarks for Logical Volumes and Watermarks for File Systems" for the Windows host agent.
To turn off Watermarks DCPs, do the following:
On the Tree panel side of ControlCenter Console, expand the Administration folder
Expand Data Collection Policies folder.
Expand Policy Definitions.
Expand Host Agent for Windows.
Right-click on the watermark policy and disable. For example: Watermarks for Volume Groups_00, scroll down and select Disable from the menu or remove the hosts W2K8 Enterprise SP1from the policy.
Restart the host agent on one of the W2K8 host and confirm if the host will stay active.
This will force the Host agent to rescan its Data Collection Policies and disregard the Watermark DCPs.
If the host agent becomes active in the console and if you required the above watermarks DCP to be enabled.
Start them one at a time and monitor the host after to see if the issue re-occurs.
This will help isolate each of the Watermark polices to see which watermark is causing the issue on the hosts.
Enable 1 of the Watermarks for example the "Filesystem Watermark" leave it to run for its schedules cycle (default 15 minutes).
If it runs for its cycle without causing the agent to go inactive. Enable "Volume Groups " Watermark leave it to run for its schedules cycle and then enable "Logical Volumes ".
If the issue remains after performing the above actions it would be best to open an SR with Support.
Attach the following to the new SR from 1 of the problem 2008 hosts
Master agent
%ECC_INSTALL_ROOT%\exec\MGA610\MGA.log
%ECC_INSTALL_ROOT%\exec\MGA610\MGA_Bx.log.gz
%ECC_INSTALL_ROOT%\exec\MGA610\MGA.ini and %ECC_INSTALL_ROOT%\exec\ctg.ini
%ECC_INSTALL_ROOT%\exec\MGA610\MGA_Err.log
%ECC_INSTALL_ROOT%\exec\MGA610\MGA_Bx_Err.log
Host agent
%ECC_INSTALL_ROOT%\exec\MNR610\MNR.log
%ECC_INSTALL_ROOT%\exec\MNR610\MNR_Bx.log.gz
%ECC_INSTALL_ROOT%\exec\MNR610\MNR.ini
%ECC_INSTALL_ROOT%\exec\MNR610\MNR_Err.log
%ECC_INSTALL_ROOT%\exec\MNR610\MNR_Bx_Err.log
%ECC_INSTALL_ROOT%\exec\MNR610\MNR_Discovery.log.gz
Policies Export
Please follow the below Procedures:
Instructions for Policies Export from the Console:
Collect an export of the Policies table view for Storage Systems, Hosts, Connectivity Devices & Database Instances by going to the ECC console > Select ECC Administration on the blue toolbar > Hit the dropdown arrow > select Policies > in the Tree Panel on the left side of the Console, put a check next to "Storage Systems", "Hosts", "Connectivity", & "Database Instances" folders. > In the Target Panel on the right you will see the Policies view. Click anywhere in the right-hand window to be sure that it is the currently selected window. > Go to the top left-hand corner of the console screen and select File > Export > Browse > save the file as "Policies.csv" anywhere, such as the C:\ drive or on your Desktop.
emcreport output
The latest EMCReport for windows can be found at the following FTP site: ftp://ftp.emc.com/pub/emcgrab/Windows/
Regards
Conor Lawlor
imajica1
3 Posts
0
July 14th, 2010 06:00
HI Conor,
Thanks for your swift response
W2K8 Enterprise SP1
Agent version 6.1.0.7.344
Last few lines of MNR.LOG
SDK 11:49:02 C P T 28696 (16.85 MB) swrx_thread::create_thread => Creating new thread. (16 active and
- 0 blocked)
SDK 11:49:02 C P T 28696 (16.90 MB 52.000000K) swrx_thread::create_thread => Created thread 31824
SDK 11:49:02 C P T 31824 (16.96 MB 60.000000K) ThreadRun => SANITY: Begin internal thread wrapper
SDK 11:49:02 C P T 31824 (16.97 MB 8.000000K) ThreadRun => Start Thread "DataPolicy Thread, Checking
- Watermarks for Volume Groups", Act: 16, Blkd: 0 (31824)
SDK 11:49:03 SDK 1 P E 31324 (16.97 MB) swrx_exceptionhandler::UnhandledExceptionFilter => Fatal
- exception occured on this thread. Process is now terminating. Refer to file
- 'C:\ecc\exec\MNR610\mnragent_FATAL.LOG'
MNR 11:49:03 C P T 31824 (48.43 MB 32216.000000K) [Process DP] => Calling agent to process data policy
- Watermarks for Volume Groups
SDK 11:49:04 C P F 28696 (54.60 MB) [MGK_DataPolicy] => MGK_DataPolicy: creating thread for policy
- 0x01D35318
SDK 11:49:04 C P T 28696 (54.60 MB) swrx_thread::create_thread => Creating new thread. (17 active and
- 0 blocked)
SDK 11:49:04 C P T 28696 (54.65 MB 52.000000K) swrx_thread::create_thread => Created thread 31924
SDK 11:49:04 C P T 31924 (54.71 MB 60.000000K) ThreadRun => SANITY: Begin internal thread wrapper
SDK 11:49:04 C P T 31924 (54.71 MB 8.000000K) ThreadRun => Start Thread "DataPolicy Thread, Checking
- Watermarks for File Systems", Act: 17, Blkd: 0 (31924)
ConorLawlor
2 Intern
•
233 Posts
0
July 14th, 2010 06:00
Hi,
Are the hosts Windows 2008 SP2 or Windows 2008 R2?
Provide the exact version of the host agent installed (at start of MNR log).
Can you post the last 10 lines of the MNR.log %ECC_INSTALL_ROOT%\exec\MNR610\
Regards
Conor Lawlor
imajica1
3 Posts
0
July 14th, 2010 07:00
Hi Conor,
Thanks for the response. None of the servers are listed under the policy for the watermarks so I'll get a call logged.
Regards
Stuart
Lars4_bf562a
46 Posts
0
July 30th, 2010 02:00
FYI: We had similar/same problem and it was the built in Firewall in 2008, try to disable it for a short wile and restart the ECC agents ( we had to delete the lock files... first)
/Lars