Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL Aurora failover ends up on reader when using custom domain #1254

Open
szhnet opened this issue Jan 22, 2025 · 11 comments
Open

MySQL Aurora failover ends up on reader when using custom domain #1254

szhnet opened this issue Jan 22, 2025 · 11 comments
Assignees
Labels
bug Something isn't working

Comments

@szhnet
Copy link

szhnet commented Jan 22, 2025

Describe the bug

#386 The problem still exists.

A jdbc url like this: jdbc:aws-wrapper:mysql://[custom-domain]:3306/db?characterEncoding=utf-8&clusterInstanceHostPattern=?.[clusterid].us-east-1.rds.amazonaws.com&wrapperLoggerLevel=ALL

In the log there is something like this
The driver is unable to track this opened connection because the instance endpoint is unknown: '[custom-domain]'

java.sql.SQLException: Running in read-only mode

Expected Behavior

The failover should end up on the writer node.

What plugins are used? What other connection properties were set?

Default plugins are used.

Current Behavior

The connection ends up on the reader instead of the writer.

Reproduction Steps

See description.

Possible Solution

No response

Additional Information/Context

No response

The AWS Advanced JDBC Driver version used

2.5.4

JDK version used

11

Operating System and version

Linux

@szhnet szhnet added the bug Something isn't working label Jan 22, 2025
@szhnet szhnet changed the title (short issue description)Describe the bug MySQL Aurora failover ends up on reader when using custom domain Jan 22, 2025
@aaron-congo
Copy link
Contributor

Hi @szhnet,

Thanks for reaching out and raising this issue. We'll take a look at this and keep you updated as we investigate. Thank you for your patience!

@karenc-bq karenc-bq self-assigned this Feb 3, 2025
@karenc-bq
Copy link
Contributor

Hi @szhnet, I am unable to reproduce the error you are seeing on v2.5.4. Would you be able to share the full driver logs with us?
We have recently merged some fixes related to custom domains. These fixes are available in this snapshot build: aws-advanced-jdbc-wrapper-2.5.5-20250206.155700-54.jar. While we are still investigating the root cause of this issue, would you be interested in giving this snapshot build a try to see if this resolves your issue?

@szhnet
Copy link
Author

szhnet commented Feb 7, 2025

full log
app1.log

@karenc-bq
Copy link
Contributor

Hi, @szhnet, thank you for the driver logs. It looks like driver failover wasn't triggered because no network errors were detected. Would you be able to provide more information about your custom endpoint and your test setup?

  1. Does your custom endpoint point to a cluster endpoint or an instance endpoint?
  2. How are you triggering cluster failover?

@szhnet
Copy link
Author

szhnet commented Feb 8, 2025

  1. In Route 53 add cname [custom-domain] -> xxx.cluster-[clusterid].us-east-1.rds.amazonaws.com

  2. Use manual failover in the AWS Management Console.
    Image

  3. My test program executes an update sql every 5 seconds. If the execution is successful, you can see something like "FooService work count=0" in the log. After count=3, when I use manual failover, you can see that "Running in read-only mode" starts to appear.

  4. When I use "xxx.cluster-[clusterid].us-east-1.rds.amazonaws.com" in jdbc url, the test program is OK.

@longzhihun
Copy link

same here, when used Route53 domain cname to aurora endpoint, it doesn't work, do we have any workaround.

@longzhihun
Copy link

Image

@karenc-bq
Copy link
Contributor

Hi both, could you please try the snapshot build aws-advanced-jdbc-wrapper-2.5.5-20250206.155700-54.jar and set the verifyOpenedConnectionType=writer connection parameter to see if this resolves your issue?

@longzhihun
Copy link

@karenc-bq Hello, the link response 404

@szhnet
Copy link
Author

szhnet commented Mar 7, 2025

Hello
I tried the aws-advanced-jdbc-wrapper-2.5.5-20250303.181434-75.jar and set verifyOpenedConnectionType=writer, the problem still exists.

app2.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants