Start a Conversation

Unsolved

S

7 Posts

1009

December 26th, 2022 02:00

M1000e with M6348 switch, link down

Hello,

I have 8 M630 blade servers with
- Intel(R) 10GbE 2P X710-k bNDC (Fabric A
- BRCM 10GbE 2P 57810S-k Mezz (Fabric C)

A1: M6348 switch
C1: M6348 switch

For C1 (4 lanes), everything just fine.
But for A1 (2 lanes), not all links usable.
ONLY 1,4, 5, 8, 11-14 are usable like this










saknarak_0-1672049045806.png

So I cannot use slot 1-8 for fabric A

I try to swap, M6348 switch between A1 and C1. got same result.

Please suggest me, what I missing



 




Moderator

 • 

2.8K Posts

December 26th, 2022 06:00

Hello, I was trying to find something and found these slide pieces. Please check them,

refer link is https://slideplayer.com/slide/5703980/

 

 

 

Hope that helps!

7 Posts

December 26th, 2022 07:00

Thank you for your information.

I have 2 m1000e

The first one is work perfectly, I install M6348 switch into Fabric A1 slot.
All 16 ports online without any issue.

saknarak_0-1672066909103.png

slot 1-8 has M610, slot 9-16 has M620

But for second system, slot 1-8 has M630
M6348 installed on Fabric A1 slot, not all ports were online.
When I move M6348 into Fabric B1, all ports are online.

M630 installed with
A: Intel(R) 10GbE 2P X710-k bNDC
C: BRCM 10GbE 2P 57810S-k Mezz










Why not the same with the first system? so strange.

saknarak_1-1672067470326.png

I

Moderator

 • 

2.8K Posts

December 26th, 2022 07:00

thanks, that's really odd. Could the FW versions be different, please check control management FW in here https://dell.to/3VpUEhj

7 Posts

December 26th, 2022 18:00

May be not a firmware issue

System1:  firmware version 6.10 work fine

System2: firmware version 6.21 has this issue

But I will try to downgrade version first.

7 Posts

December 26th, 2022 19:00

I just rollback to 6.10, but it didn't help

7 Posts

December 26th, 2022 23:00

My system M100e
A1: M6348
C1: M6348

slot1-8: M630 with same config
A: Intel(R) 10GbE 2P X710-k bNDC
B: Nothing
C: BRCM 10GbE 2P 57810S-k Mezz

Result:
A1: Only slot 1,4,5,8,11-14 can be online
C1: All 16 slots can be online
------------------------------
I also have another system without problem
M100e
A1: M6348
B1: M6348
slot1-8: M610
A: Embedded LOM  (1Gbps)
slot9-16:M620
A: BRCM 10GbE 2P 57810s bNDC

Result: any slot can be online without any issue
--------------

4 Operator

 • 

3K Posts

December 26th, 2022 23:00

M6348 is a 1 Gig switch and looks like you have a 10Gig Mezz cards. Can you share below details

  • Mezz card installed on first chassis where all ports are coming up properly
  • Mezz card in Fabric A and C on second chassis is same across all blades?
  • What Mezz card installed on Fab B on second chassis where all ports are coming up properly

4 Operator

 • 

3K Posts

December 27th, 2022 02:00

Can you swap the M630 from a working slot to non working slot and see whether it is working. This way we can check whether it is an issue with the slot or with the server

4 Operator

 • 

3K Posts

December 27th, 2022 03:00

Can you swap one of the M630 SLED instead of switch and try.

7 Posts

December 27th, 2022 03:00

Yes, sure, I already try it. these results are

From
A1: M3648 (1) => some online
C1: M3648 (2) => all online

To
A1: M3648 (2) => some online
C1: M3648 (1) => all online

Also
A1: Pass-Through module => all online
B1: Any module => all online
A2: M3648 => some online

4 Operator

 • 

3K Posts

December 27th, 2022 03:00

When we install any switch or server to non working slot the failure continues and the same switch is working on other slots. As the issue seen with specific switch Slots and with specific server slots, I believe this could be a hardware issues on M1000e chassis midplane. 

7 Posts

December 27th, 2022 03:00

Yes sure, I had moved M630 into all slots, one-by-one
only slot 1, 4,5, 8, 11-14 work when M3648 in A1

Any M630 that put in slot 1, 4,5, 8, 11-14 will work.

No Events found!

Top