Failed to renew Certificate in Issuer in Kubernetes…

About

Until now, Let’s Encrypt has sent email notifications before TLS certificates expired. However, it seems that this notification service will be discontinued. As a result, there may be more cases where service providers fail to notice the expiration and their services become inaccessible.
This makes automated certificate management increasingly important.

However, in this unstable situation, my cert-manager failed to automatically renew the certificate. Since it took some time to resolve the issue, I’ve documented the troubleshooting process here for future reference.

Look for where the errors are…

When I checked certificate in k8s with this command,

kubectl get certificate -n ns

Below output was returned. As you can see, there is 404 eror when challenge was conducted.

  Type:      HTTP-01
  URL:       https://acme-v02.api.letsencrypt.org/acme/chall/----------
  Wildcard:  false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: wrong status code '404', expected '200'
  State:       pending
Events:
  Type    Reason     Age   From                     Message
  ----    ------     ----  ----                     -------
  Normal  Started    67s   cert-manager-challenges  Challenge scheduled for processing
  Normal  Presented  67s   cert-manager-challenges  Presented challenge using HTTP-01 challenge mechanism

I checked ingress next, but it is no doubt working.

kubectl get pods -A | grep ingress
ingress        nginx-ingress-microk8s-controller-ftbz5    1/1     Running   4 (31d ago)        91d

Lastly, I checked ingress class, It seems that I found something.

kubectl get ingressclass

NAME     CONTROLLER             PARAMETERS   AGE
nginx    k8s.io/ingress-nginx   <none>       2y85d
public   k8s.io/ingress-nginx   <none>       2y85d

Which ingressclass is one that is used as default?

If you execute the command, you can check the status of each ingressclass.

kubectl get ingressclass nginx -o yaml
kubectl get ingressclass public -o yaml

If you fine below settings from ingressclass, that ingressclass is used as default.

metadata:
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"

In my case, “public” is the default ingressclass.

Update Manifest of Cluster-Issuer

However, the ClusterIssuer manifest associated with my service didn’t include a setting for the default ingress class.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: your-email@your-domain
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: public  # ← nginx to public

So I changed the setting, as like showed above, nginx to public.

And then I connected the service with https correctly.

I hope cert-manage is going to update and manages certificates automatically.

Manifest of Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: public
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
  generation: 6
  name: yourservice-ingress
  namespace: yournamespace
spec:
  ingressClassName: public # setting this is important
  rules:
  - host: sub.yourdomain.com
    http:
      paths:
      - backend:
          service:
            name: yourservice_name
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - sub.yourdomain.com
    secretName: your-ingress-tls
status:
  loadBalancer:
    ingress:
    - ip: xxx.xxx.xxx.xxx

Check Certificate Request

microk8s kubectl describe certificaterequest -n yournamespace

  Conditions:
    Last Transition Time:  2025-05-02T14:28:45Z
    Message:               Certificate request has been approved by cert-manager.io
    Reason:                cert-manager.io
    Status:                True
    Type:                  Approved
    Last Transition Time:  2025-05-02T14:29:10Z
    Message:               Certificate fetched from issuer successfully
    Reason:                Issued
    Status:                True
    Type:                  Ready
Events:                    <none>