6 Posts
0
4956
January 5th, 2021 19:00
PE C6100: ipmitool = LAN Parameter Data does not match (Latest BIOS & BMC)
Hi there,
We've got a Dell PowerEdge C6100 system where the nodes are reporting the following error when we attempt to use ipmitool against them via the installed operating system (Debian) when using any commands other than "lan 1 active on/off" or setting a VLAN tag (but not setting the VLAN tag to OFF which also leads to the same error).
LAN Parameter Data does not match! Write may have failed.
There are other threads around which suggest root causes from outdated BIOS/BMC through to write-protection via LAN Failover (LFO) (which was recommended to be fixed via syscfg which we don't believe to be compatible with the C6100 node boards?). In this instance though, the BIOS & BMC on each node is up-to-date (1.81 & 1.33).
Our problem is that the VLAN ID shouldn't have been set, so we're now trying to remove the tag from IPMI/BMC so that the interfaces will come up. This isn't possible, nor is specifying a static IP, subnet or default gateway. All fail with the above error after a short while attempting to write the config. User attempting this is root, so it's highly unlikely to be related to the OS or privileges of any kind, but seems to be a peculiar and occasional bug.
If we try to change the IP addressing via BIOS > IPMI Configuration the newly-specified addressing doesn't save to the "Current BMC Config" (or similar wording) line after pressing enter on the "Enter new address" (or similar wording) line. Logically, that aligns with a silent failure. BIOS > IPMI has nothing beyond basic configuration, so we're unable to attempt removing the VLAN tag/ID from that route, which would've been the logical fix.
Should we be disabling the LFO (LAN Failover) using OpenManage on these nodes? Would that permit the updates that we need to do here? Or is there another solution that we're missing? It's frustrating that it happily accepted the VLAN ID initially and is now refusing to have it removed, and also won't take IP addressing. We can work around the addressing using DHCP, however now that it's tagged, it won't route through our untagged ports.
Relevant threads:
https://serverfault.com/questions/499377/ipmi-lan-parameter-data-does-not-match
The most interesting one is:
However we're unsure how to SSH into a card that we don't know the IP of (it's reporting none as being set) which also wants a tagged port and won't let us untag it. What's the best course of action here to reset the cards, or otherwise overwrite the erroneous configuration? It's an issue we're not seeing much of with the C6100 model.
There are many threads online about this, but most are resolved with a BIOS and BMC/IPMI update, which isn't applicable in this instance. We performed the BIOS and BMC updates successfully in late 2020 using FreeDOS.
Just to be clear, we can use DHCP (it accepts the command to move the BMC/IPMI LAN/NIC to use DHCP rather than static) so the only problem that we believe needs fixing is removing the VLAN ID. Or is there a larger issue?
Tagging @DELL-Chris H as he replied in one of the linked threads, however any and all help is appreciated!
Many thanks,
LST-AU
LST-AU
6 Posts
0
January 6th, 2021 00:00
G'day,
There ended up being no sane fix for this problem. Despite what other people had reported about BIOS & BMC being out-of-date, that wasn't the case here as I mentioned.
As we had to leave the data centre and travel inter-city, we were forced to work out a fix. That ended up being to re-flash the BMC via FreeDOS which led to several changes:
We then power cycled and configured the IP addressing via BIOS > IPMI, which accepted the changes unlike before the re-flash. The NIC was re-configured back to the dedicated card, though it's worth mentioning that the reported MAC address doesn't change in BIOS despite changing the setting UNTIL YOU REBOOT AGAIN.
From there, as the MAC had changed after a confirmation check in BIOS, we then had to update our MAC-to-IP binding configuration to match the dedicated NIC's MAC address. We disconnected the NIC, left it for 1-2 minutes, then reconnected and were able to access the IPMI interface through a web browser. Splendid.
It's a shame that there's no known fix for this, however we're grateful that the BMC flashing process wipes the configuration. That's not something that I've ever said in my ICT history, or at least not that I can recall!
Hopefully this helps someone in the future, as we were stuck on these bugged-out nodes out of a plethora of hardware that was reconfigured without incident, and were starting to question our sanity.
Cheers,
LST-AU
DELL-Joey C
Moderator
•
3.9K Posts
0
January 6th, 2021 01:00
Hi,
Thanks for sharing your resolution to the issue that you are having, apologies I couldn't reply sooner to try help out or suggest some methods to check if it does help you.
Just curious, did you run ipmitool lan set 1 vlan id off, and got the error "LAN parameter Data does not match! Write may have failed"?
Definitely there are fixes on iDRAC and BIOS to the error that you face, but I'm unsure if they are fixed on the server that you had on. In case if in future you face same issue, could you try running and outsource IPMITool ver. 1.8.18 to check if the issue persist. Alternative, you can se the raw command to disable vlanid: "ipmitool raw 0x0c 0x1 1 20 0x2 0x0".
LST-AU
6 Posts
0
January 6th, 2021 02:00
Many thanks for your reply, Joey.
You're right - that's the error we got when we ran that command as you've written it; same occurred when we tried to specify the IP address, subnet mask and default gateway. However, changing the mode from DHCP to/from static worked OK. Also fine was enabling or disabling channel access. Others have reported such behaviour.
Attempting a warm reset was unsuccessful, and while cold resets claimed to have sent the command to the BMC, there was no evidence of that completing properly. Reminded me of UDP traffic - "send and hope it's fine".
I've just confirmed that the version of ipmitool in-use on all servers was 1.8.18.
As for the raw command, while it's good to have on-hand for the future, could you please provide a reference for all Dell ipmitool raw commands? It seems to be a mixed bag online, though perhaps we've missed a manual?
Could you please check with others within Dell to see if the PEC6100 has been patched?
Thank you,
LST-AU
DELL-Erman O
Moderator
•
2.8K Posts
0
January 6th, 2021 05:00
Hello,
Unfortunately I'm not too familiar with the old C6000 model. I share some documents that I could find after researching and that I think might be helpful;
Dell PowerEdge C System Management https://dell.to/395ISlV;
Using IPMItool raw commands for remote management of dell PowerEdge Servers https://dell.to/3rULGLz
Using the Baseboard Management Controller pg 18 https://dell.to/2KXmG5J
https://dell.to/39lOCbF;
from github https://dell.to/3pP0mcY;
LST-AU
6 Posts
0
January 7th, 2021 17:00
Thank you Erman,
Appreciate the links, we'll keep those to-hand for the future.
With the PEC6100, could you please explain how we'd go about updating other components aside from the BIOS and BMC? For example, the RAID card (which doesn't seem to be listed on the website), fan module, etc.
This system is pre-DSU so we can't use our typical Dell update process. It could well be that there are no updates required for the chassis aside from what's been done, though it'd be great to have the resources on-hand.
Many thanks,
LST-AU
DELL-Joey C
Moderator
•
3.9K Posts
0
January 7th, 2021 18:00
Hi,
Well, C6100 is a relatively an old server, the new update utilities like SUU or DRM is out of support for C6100. So there is not ways to update the components, other than what you have done, using BMC and FreeDOS (which you have successfully done and I'm not knowledgeable in that) or to do the update in OS. Last I checked on the C6100 updates on RAID and Fan module are dated 2012/2013.