1 Rookie
•
5 Posts
0
6756
March 6th, 2021 02:00
r720 Fan 4 still running after shutdown
I have two r720s BOTH have developed an issue where Fan 4 keeps running even when the system is off. One system has iDrac FW 2.65.65.65 and the fan stays on very low speed, the other FW 2.63.60.62 and the fan is always at full blast, system on or off.
The second system was purchased in order to fix the issue of the first with a MB swap. But it too developed this problem - on the same fan - after updating the FW to current.
I have read 1 other post here with the same problem, their solution being to revert to a much older (v1) FW. The fact I have 2 systems displaying an identical issue on the same fan seems to confirm it is the FW.
I realise these are older systems but can anyone at Dell offer some insight regarding which FW is recommended to fix this issue? 2.65.65.65 is the last for the r720 I believe - perhaps there is some other driver or combination of bios & FW that will stop these errant fan 4s?
System 1 - FW 2.63.60.62, bios 2.9.0 (Fan 4 full blast at all times)
System 2 - FW 2.65.65.65, bios 2.4.3 (Fan 4 on low speed after system is off)
jms99
1 Rookie
•
5 Posts
0
March 11th, 2021 00:00
OK - I have fixed the problem. It does seem to be a Firmware issue**. Specifically some firmware after 2.40.40.40.
To recap:
System 1 - Fan 4 always on full speed. - 2.63.60.62, bios 2.9
System 2 - Fan 4 always on, very low speed. - 2.65.65.65, bios 2.4
System 2 was originally upgraded from a v1 FW, walking through a few versions including 2.40.40.40 then directly to 2.65.65.65, I wasn't sure but I didn't notice the fan being on low all the time until I updated to 2.65.65.65. That's when I started noticing it.
So I rolled it back to 2.40.40.40 - the low speed fan 4 has stopped!
Next, as recommended, I updated system 1 (with the full speed fan 4) from 2.63.60.62 to latest 2.65.65.65. No change.
So I downgraded system 1 to 2.40.40.40 - this FIXED the fan 4 full speed issue.
Both systems are now on 2.40.40.40 and both are operating normally, system off = fans off. IDK exactly which FW causing the issue but it's between 2.40.40.40 and 2.63.60.62.
**(The complication is that System 1 was once running normally with 2.65.65.65 - the high speed fan 4 was an issue that developed after that system had been off for some time)
DELL-Joey C
Moderator
•
3.9K Posts
0
March 8th, 2021 00:00
Hi,
I tried checking if there is any notes on such issue on R720 iDRAC version, but there is no known issue stated. There are a few things to I would like to check. Does Fan 4 spins after a few minutes of wait? What happen if the power cables are pulled and installed back, does the fan spin? When the system is off, are there any LED lid? If you were to swap Fan4 and Fan3, does the issue follow the slot? Do you have iDRAC access? Have you tried resetting the iDRAC?
jms99
1 Rookie
•
5 Posts
0
March 8th, 2021 06:00
Hello joey, thanks for the reply.
The problem doesn't follow the fan, its always slot 4.
Fan spins up immediately power cable is plugged in. Every time.
The inner iDrac LED is lit when the system is off but power still connected.
Yes iDrac has been reset (and NVRAM wiped etc) I've got physical access to the machines.
I've read that fan 4 stays spinning on low speed in order to cool the iDrac. So for the 2nd machine it might be normal behaviour? Is that correct?
DELL-Stefan R
Moderator
•
790 Posts
0
March 8th, 2021 07:00
Hi jms99.
The iDRAC is running all the time, true, so it needs to be cooled.
I haven't heard about it yet tbh, but the more I think about it, the mere it makes sense.
I recommend that you update the iDRAC on the 2nd system also to the latest version to lower the speed of the fan.
If you come across this article again, where you read it, feel free to share it
BR
Stefan
jms99
1 Rookie
•
5 Posts
0
March 8th, 2021 09:00
Hi Stefan. Thanks for the input. I suspect it could be some sort of FW corruption - hopefully updating it will fix it.
The info about fans running for the iDrac was on a reddit thread I think, can't find the link.
Found this thread though that suggested down grading FW fixed it
https://www.dell.com/community/Rack-Servers/Dell-R720-FAN-4-still-runs-after-shutdown/m-p/6194496#M4108
DELL-Joey C
Moderator
•
3.9K Posts
0
March 8th, 2021 17:00
Hi,
Can I know if the fan do stop in period of time, have you monitored that? like it spins for about 5 to 10 minutes? Has the other firmware in the server have been updated? eg: BIOS.
Reset of iDRAC, what kind of reset would it be? Have you tried RACADM hard reset?
Try this, remove all components except the below and the fans. Then try connect the power cable and check if fan 4 still spins:
•One Power Supply
•Control Panel (for power button functionality)
•Working cable connection between control panel and system board
•One Processor (CPU) in socket CPU1 (minimum for troubleshooting)
•One Memory Module (DIMM) installed in socket A1
•System Board
•Backplane (for R720xd control panel connection)
DELL-Shine K
4 Operator
•
3K Posts
0
March 8th, 2021 18:00
@jms99 , As @DELL-Stefan R mentioned some components (E.g. iDRAC, NDC) on the server will be running even if server is powered off. Some of the FANS will be running at low RPM for cooling of these components. This is dependent on the server configuration as well. I would recommend to update iDRAC FW to latest on system 1 also and check whether FAN running at low RPM after the upgrade.
jms99
1 Rookie
•
5 Posts
0
March 10th, 2021 01:00
The fan on both machines stay on constantly, it doesn't stop, 1 day later still on.
I think I must have only done a soft reset (via F2 menu, or Lifecycle Controller, can't recall which), I haven't used RACADM. I will try it, if that fails I'll try the troubleshooting steps you suggest.
DELL-Joey C
Moderator
•
3.9K Posts
0
March 11th, 2021 01:00
Hi,
Thanks for updating us on the issue that you're having. A good share. I'll put a note on the case notes for engineering to take note.
Orion
1 Message
0
September 23rd, 2021 08:00
Hello all,
I noticed the same issue today after updating my R720. With firmware 2.40.40.40 (after shutdown) the number 4 fan isnt powered up and spinning. After updating to the FW 2.65.65.65 it does spin. I also tried FW 2.63.60.62 same thing. All that running the latest BIOS (2.9.0)
Withing a minute of shutdown the number 4 fan starts to spin at low rpms. I tried changing No4 with No3 fan and various other combinations with the same result. I read the comment above that it may be for cooling of the idrac but this means that with FW 2.40.40.40 and before it wasnt needed?
It seems like a firmware issue and ill revert back to 2.40.40.40 for now. If you can have a look please and maybe fix it? Thanks alot!
Bdub01
6 Posts
0
October 2nd, 2021 13:00
I can confirm all of the same things on my own R720. Fan 4 always runs at low RPM when the power is off. It does not matter what fan is swapped into that slot, it is slot 4 that is powered all the time. Also, previously there was an ability to manually adjust the fan % power that has been lost with later firmware. Because my boxes live in a room that is colder than necessary most of the time even the low 20% range is too high most of the time. They used to be at 10% with automatic increase on demand, but now I can not manually adjust the fan tolerances with the latest update. Apparently a lot of the fan control is now removed with the latest revisions.
elfriscia
4 Posts
0
November 8th, 2021 20:00
Having the same problem since I've updated.
elfriscia
4 Posts
0
November 8th, 2021 20:00
In the iDrac Settings, thermal options, change the fan settings to fan speed offset.
elfriscia
4 Posts
1
November 8th, 2021 22:00
100% confirmed to be a firmware problem, went through everyone starting with 2.40.40 and finishing with 2.65.65: Last one working good is 2.41.40, there's something implemented in 2.50.50 breaking the 4th fan or either its sensor. Its not a fan problem, swapped with another one and it does the same.
DELL-Joey C
Moderator
•
3.9K Posts
0
November 9th, 2021 00:00
Hi @elfriscia,
Thanks for the troubleshooting info that you have found out.
Can I know how many R720s do you have? Only one?
Can you let me know if the fan still spins after you have shutdown the server and waited for more than 15minutes. Have you tried resetting the iDRAC? Is BIOS up to date?