Avago EFI SAS Driver Fail Issue during server boot

this is the ss of the issue i am facing 

Hi,
“Avago EFI SAS Driver” error is commonly associated with the RAID controller or attached drives

## What to try first
- Power the server off fully, disconnect all external storage and non-essential PCIe cards, then boot again. Dell community reports that removing PCIe cards can help isolate the device causing the firmware hang.
- If the system boots farther after removing devices, add hardware back one piece at a time to find the offender. A failing drive, RAID controller, or attached storage path is the most likely trigger here.
- Check the RAID controller and all physical disks from iDRAC/Lifecycle or during POST, because Dell notes that Avago EFI SAS driver health errors usually indicate a controller or drive issue.

is there any way to fix this issue remotely?
Thanks for the response though

Not really

but you could do this first if possible
Enter BIOS and check storage-related settings, especially integrated RAID/controller and slot disablement 
Inspect RAID/drive health in iDRAC or Lifecycle Controller once you can reach it .
If possible, update BIOS and iDRAC together after the system is stable enough to boot; Dell community guidance specifically recommends walking BIOS and iDRAC forward together when they are far behind 

not able to enter into bios as its getting stuck before reaching there.

also tried scheduling some jobs related to bios but lifecycle controller is also stucked

Hi,

May we know what happen before the YSOD occured, any additional hardware installed or any firmware upgrades that led to this error? The YSOD is pointing to -->RIP 562A4BD7 Metronome.efi +001BD7. 

Since you are unable to access BIOS to check if you are able to change mode to BIOS instead of UEFI, perhaps it is best to be in front of the server to troubleshoot. You did mentioned, you are able to schedule a job, that means iDRAC is accessible. 

Can you check in iDRAC logs, to see the lifecycle log is able to capture any issues, in such like PCIe error. Do you have a Mellanox card installed? 

Hi,

these are refurbished machine so probably there are some hardware changes will be there. 
Boot order i have tried changing from iDRAC virtual console from :normal boot , pxe boot, BIOS, via virtual CD/DVD. But all getting stuck at same point 
on iDRAC i see some errors like :

"A bus fatal error was detected on a component at slot 2"
"A fatal error was detected on a component at bus 93 device 2 function 0" (is this PCI mellanox slot 2 ? )
"System BIOS has halted"

yes mellanox card is present.

Hi, 

Try checking the PCIe slot assignments, the device ID might present' System > Overview > PCIe Slots. Then expand the PCIe slots to check the assigned Key column. 

If there is a Mellanox card installed, I would suggest removing it to check the issue if it still persist. 

Hi, 
So the conclusion is I checked all the parts of these servers and they are working fine. So I tried removing the HDDs and found out there are some disks because of them its getting stuck. Now i can't understand if the issue was in the HDDs then why on iDRAC its showing these errors:
"A bus fatal error was detected on a component at slot 2"
"A fatal error was detected on a component at bus 93 device 2 function 0" 
is there any reason behind this ?

Hi,

We may not know the extensive affected component that is the root cause of the error, you did great by identifying the cause is from the drives. Have you already checked what is bus 93 device 2 function 0 in iDRAC as I mentioned? 

Have you installed non-Dell drive in the server? - Do you have the Dell part number, maybe it could be incompatibility.

Hi,
I checked in racadm -> hwinventory and on bus 94 mellanox pci card on slot 2 is present. Maybe bus number showing on iDRAC is slightly different than actual. I'm not sure about this assumption can be a case.
Though a faulty HDD should not give NMI as there are more than one HDDs. This can be the case if OS is installed on that particular drive (assumption) but no OS was installed on these servers?
what could be the root cause according to you?

If any other having the same issue... try removing your HDDs and try to boot by using one disk at a time as in my case this worked. I tried all remote stuff via iDRAC and ssh but none worked.

:peace

Thx for letting us know, as I mentioned only troubleshooting via remote sometimes not helps but a short reseat or remove fix many problems.