Troubleshooting Kubernetes 01 : Delete Orphan Objects (CRDs)

 

When it comes to application maintenance, version upgrade is a kind of thing we have to attend to once a year or two.

But it would make your life easy if the applications are deployed in Kubernetes.

I thought to share recent activity we engaged with to upgrade KEDA in AKS (Azure Kubernetes Service).

Story Begins Here…

First of all we had to uninstall the existing version of KEDA.

First of all we had to uninstall the existing version of KEDA.

You can follow below blog posts to uninstall and install the KEDA.

https://keda.sh/docs/2.12/deploy/#uninstall

https://keda.sh/docs/2.12/deploy/#install

Now after uninstalling the KEDA, we should be able to delete the namespace keda , but unfortunately we can’t delete it.

When you execute “kubectl delete ns keda”, command, it was stucked in the “Terminating” status.

But when you check resources in that namespace you can't find any. So what can be the culprit?

Now the main reason we are using KEDA in our current project is to use ScaledJobs to process application data in an event driven way.

Now the main reason for this kind of situation is, still dependent orphan objects (CRDs) still living there.

Lets dig into the hole…

We can see all the KEDA specific api-resources from below command

1. Now we can get the “triggerauthentications” api-resources type objects and delete them as below.

2. Then we can get the KEDA specific CRDs and delete them as below.

But when you try to delete other CRDs , it will fail and those will still hanging aroung here.

3. What we can do to get rid of those is to differentiate finalizers using the kubectl patch command.

Now we can delete the remaining KEDA specific CRDs.

4. Now we need to check KEDA specific apiservices and delete them.

5. Finally you can see all the dependent orphan objects are deleted. As a last step we can delete the keda namespace now.