Skip to main content
Start a Conversation

Unsolved

WilliamDLB

1 Rookie

 • 

11 Posts

138

April 16th, 2025 14:25

InsightIQ 'simple' upgrade from 5.1 to 5.2 fails pre-check "There are 1 unhealthy pods."

When running the upgrade script ./upgrade-iiq.sh on an InsightIQ 5.1 'simple' (VMware OVA) it fails with:

[2025-04-15 12:12:19] [INFO] All nodes are in a healthy state.[2025-04-15 12:12:19] [INFO] There are 1 unhealthy pods.[2025-04-15 12:12:19] [INFO] Some pods are not in a healthy state. Exiting..[2025-04-15 12:12:19] [ERROR] [PRECHECK] Failed to run precheck, please run showupg --log to see more detailed logs.[2025-04-15 12:12:19] [INFO] [PRECHECK] Pods were not scaled down. No scaling up required.

We know nothing about Kubernetes, and have little desire to.

The command :

kubectl get pods -n atlantic

 which is suggested in the documentation shows a list of objects that are all 'running' except for one :

clustermanagement-nfs-d75d6f57b-k74gc                 0/2     Init:0/1   0              24h

Running the script again (as suggested) does no better.   The credentials are good, diskspace is good.   We just upgraded our one PowerScale to 9.7.1.7 and we knew that 5.1 was not compatible (and indeed monitoring is broken).  So we would rather like to be able to upgrade to 5.2.

We would raise an SR but it seems that InsightIQ only has telephone support.

Has anyone any suggestions where to look for 'unhealthy pods'?

Thanks

DELL-Josh Cr

Moderator

 • 

9.5K Posts

April 16th, 2025 20:33

Hi,

 

Thanks for your question.

What does showupg –log show?

 

You can try recreating that pod.

1.Login/SSH to InsightIQ primary-node

2. Edit the clustermanagement deployment using the below command:

kubectl edit deployment clustermanagement-nfs -n atlantic

3. deployment.yaml file will open in default editor

4. Remove the mount "cms-nfs-vol" from containers and volume "cms-nfs-vol" from volumes

5. After removing sections mentioned in step 4, Save the deployment file ':wq!'

6. clustermanagement-nfs will restart automatically, It will delete the existing POD and create a new one. 

7. Check the pod status. New clustermanagement-nfs pod will be spawned without any older NFS mount info:

kubectl get pods -n atlantic | grep clustermanagement-nfs

8. At this point the customer should be able to remove the NFS export from their cluster and the clustermanagement-nfs pod should not go into an init 0/1 state anymore.

 

 

Let us know if you have any additional questions.

WilliamDLB

1 Rookie

 • 

11 Posts

April 17th, 2025 10:13

@DELL-Josh Cr​ 

That is encouraging.  I followed the above procedure and afterwards for a while it showed this:

administrator@prdubriiq01:~/InsightIQ/upgrade> kubectl get pods -n atlantic | grep clustermanagement-nfs
clustermanagement-nfs-86d7d77b7d-g6c5d                2/2     Running       0              96s
clustermanagement-nfs-d75d6f57b-k74gc                 0/2     Terminating   0              42h

but then the 'Terminating' one went away.  All pods were now in a 'running' state, and re-running the upgrade has completed successfully:

**********************************************************************************
*                        Completed 100% of the Upgrade...                        *
**********************************************************************************

Very many thanks as I could never have worked that out myself.

DELL-Josh Cr

Moderator

 • 

9.5K Posts

April 17th, 2025 13:23

Great to hear. 

Was this post helpful?

View All

Top