"unable to ensure pod container exists" and "failed to create containerd" errors

[woehrl01]

Hi,

I'm experiencing the following errors if there are multiple volumes used by this driver.

I think this is the main reasons for #151 #150

I disabled/enabled now multiple times this driver, and it is 100% related. I already tried multiple things, you can see in the related PR #152 to fix this, my latest try to work around was using a different containerd namespace for images and snapshots, but this causes the same issues.

Do you have any suggestions how to troubleshoot this behaviour further?

[mugdha-adhav]

Could this be caused due to container restarts? I have noticed pod restarts probably due to some improper error handling after containerd fails to pull the image.

I have seen the container restart after throwing below error.

F0129 15:23:21.872324       1 containerd.go:75] unable to retrieve local image "public.ecr.aws/docker/library/amazonlinux:2022.0.20220308.1": image "public.ecr.aws/docker/library/amazonlinux:2022.0.20220308.1": not found

We already have #136 issue created for this.

[woehrl01]

Ok, this can be closed and could be tracked down to containerd misbehaving because of a timeout. Systemd queue is stuck because of too many mountpoint operations. The solution was to reduce the overall mountpoints of our pods, which fixed this issue. Looks like that using this service was just the "last straw that breaks the camel's back"