Allow setting DigitalOcean rate limit

[tja]

What would you like to be added:

Requests to the DigitalOcean API should be spaced out to avoid the current rate limit of 250 requests per minute.

Why is this needed:

I host many domains (and subdomains) at DigitalOcean, in the order of a few hundred. The external-dns DigitalOcean provider is hammering the DO API unthrottled, thus hitting their rate limits of 250 requests per minute, resulting in HTTP status code 429 and a crash of external-dns (and a crash loop as it tries over and over again).

While others have complained about this before, see #1429, the solution so far has been to increase the page count. Unfortunately this does not solve the problem for me.

[Routhinator]

Additionally, this is not just effecting the longevity of the pod for me. This is happening in my home lab, and the result is the pod no only hammers the DO API, but it also triggers a ban from github API as it triggers a Cloudflare API ban and it takes affect across sites, and getting me banned from being able to pull code... It's been fun. I've resorted to shutting the pod down when I don't need a record to update, but that literally defeats the point of the service.

[Routhinator]

I think this needs to be renamed or a ticket needs to aggregate all the rate limit problems that external-dns is causing across multiple providers

#4532 for example is the same issue, different provider.

[tja]

I think this needs to be renamed or a ticket needs to aggregate all the rate limit problems that external-dns is causing across multiple providers

#4532 for example is the same issue, different provider.

@Routhinator: Without a major refactoring, rate limiting will have to be done by each provider individually. But I would prefer to have at least one generic option/flag instead of littering the options with provider-prefixed flags.

[Routhinator]

But I would prefer to have at least one generic option/flag instead of littering the options with provider-prefixed flags.

This is the key for me, there are so many provider specific flags for generic config options common to multiple providers, and many are not documented outside their respective prs and that one issue where they've documented a list of undocumented flags.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[Routhinator]

This is still very relevant.

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[Routhinator]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[Routhinator]

/remove-lifecycle rotten

[Routhinator]

/remove-lifecycle stale