I have two K8s clusters in the same VPC that I want to connect to an MySQL
Aurora RDS cluster. One of the clusters can reach the RDS cluster just fine.
The other, however, cannot. I'll call these eks-cluster-working and
eks-cluster-broken.
I have a security group allowing traffic to the cluster:
| Type | Protocol | Port range | Source | Description |
|---|---|---|---|---|
| MYSQL/Aurora | TCP | 3306 |
sg-1 (eks-cluster-sg-working) |
This rule works |
| MYSQL/Aurora | TCP | 3306 |
sg-2 (eks-cluster-sg-broken) |
This one does not |
Both eks-cluster-working and eks-cluster-broken have the same default
security groups that AWS creates when making an EKS cluster and they are both
on the same K8s version (1.18). The one exception is that eks-cluster-working
has an extra security group for the load balancer service since it's hosting a
web service and the other one (eks-cluster-broken) is not. Both clusters
have outbound traffic rules that can reach 0.0.0.0/0.
I have zero firewalls, firewall policies and network firewall rule groups set up in the region the clusters are hosted in.
This is the Terraform configuration I have set up for peering from my VPCs to the default VPC.
resource "aws_vpc_peering_connection" "default_to_environment" {
count = local.num_environment_vpcs
peer_vpc_id = data.aws_vpc.environment[count.index].id
auto_accept = true
vpc_id = var.vpc_default_id
tags = {
Name = "Peer from default to ${data.aws_vpc.environment[count.index].tags["Name"]}"
tf_module = "vpc"
}
}
resource "aws_route_table" "r" {
vpc_id = var.vpc_default_id
route = local.route_table_routes
route {
cidr_block = "0.0.0.0/0"
gateway_id = "igw-<number>"
}
dynamic route {
for_each = [for r in local.route_table_routes : {
cidr_block = r.cidr_block
vpc_peering_connection_id = r.vpc_peering_connection_id
}]
content {
cidr_block = route.value.cidr_block
vpc_peering_connection_id = route.value.vpc_peering_connection_id
}
}
tags = {
Name = "default to environment VPCs"
}
}
Expected result
I expect eks-cluster-broken to be able to connect to the RDS cluster just
like eks-cluster-working can.
Question
What can cause this? Any suggestions as to how to go about debugging this?
