How to Route AWS ALB Traffic to a Cross-Account NLB

Sharing services across different AWS accounts often requires placing an Application Load Balancer (ALB) in one account and pointing it to a Network Load Balancer (NLB) in another.

Since you cannot use an ARN to target resources across accounts, the ALB must target the private IP addresses of the internal NLB.

Establishing the network path

Before the ALB can reach the NLB, both AWS accounts must be able to communicate at the network layer. You can achieve this using a VPC Peering connection for a direct link or AWS Transit Gateway for a hub-and-spoke architecture.

Once the connection exists, you must configure the routing:

1. The subnets hosting the ALB in Account A need route table entries pointing to Account B’s CIDR block via the Peering Connection or Transit Gateway.
2. The subnets hosting the NLB in Account B need return routes pointing back to Account A’s CIDR block.

Why target IP addresses?

An ALB target group usually points to EC2 instance IDs or Lambda functions. For a cross-account target, you must configure the target type as ip.

When AWS creates an NLB, it places one Elastic Network Interface (ENI) in each configured Availability Zone. These ENIs receive static private IP addresses from their respective subnets. To route traffic from the ALB to the NLB, you need to find these specific ENI IP addresses.

Finding the NLB IPs with Terraform

You do not want to hardcode the NLB IP addresses in your Terraform state. AWS provides data sources to dynamically look up the ENIs attached to a specific load balancer.

You can use the aws_network_interfaces data source to find the ENI IDs, and then loop through them to get the private IPs.

# 1. Get the target NLB details in Account B
data "aws_lb" "internal_nlb" {
  provider = aws.account_b
  name     = "internal-service-nlb"
}

# 2. Find the ENIs associated with this NLB
data "aws_network_interfaces" "nlb_enis" {
  provider = aws.account_b
  filter {
    name   = "description"
    values = ["ELB ${data.aws_lb.internal_nlb.arn_suffix}"]
  }
}

# 3. Fetch the actual IP addresses for those ENIs
data "aws_network_interface" "nlb_ips" {
  provider = aws.account_b
  for_each = toset(data.aws_network_interfaces.nlb_enis.ids)
  id       = each.key
}

Once you have the IPs, you can attach them to the ALB target group in Account A.

resource "aws_lb_target_group_attachment" "alb_to_nlb" {
  provider         = aws.account_a
  for_each         = data.aws_network_interface.nlb_ips
  
  target_group_arn = aws_lb_target_group.alb_tg.arn
  target_id        = each.value.private_ip
  port             = 443
}

What to check if health checks fail

If your ALB target group shows unhealthy targets, you should verify the standard network boundaries first.

1. Route Tables: Double-check that both accounts have the correct outbound and return routes via the Transit Gateway or VPC Peering connection.
2. Security Groups: The NLB Security Group in Account B must allow inbound traffic from the ALB’s subnet CIDR blocks in Account A. The target pods must also allow traffic from the NLB.
3. Network ACLs: Ensure the subnets in both accounts allow inbound and outbound traffic on the required ports.

The single AZ timeout problem

If the network paths are clear but you still see failures, you might be facing a configuration issue. During a recent deployment, I ran into a scenario where two of the IPs registered as healthy. The third IP, located in ap-southeast-5b, failed the health check with a Request timed out error.

My first thought was a missing route table entry or a blocking firewall rule. Checking the network paths showed identical routing across all three Availability Zones. A firewall blocking exactly one IP in a subnet did not make sense.

Next, I checked the VPC flow logs. The logs only showed packet drops at the destination. Dropping the ENI in that specific AZ and letting AWS recreate it did not solve the issue.

The problem lived at the load balancer configuration layer.

The default NLB behavior

AWS Network Load Balancers have a specific setting called Load balancer targets selection policy. By default, cross-zone load balancing is disabled.

This means an NLB node in AZ 1 will only send traffic to backend targets that are also located in AZ 1.

If the backend service runs on Kubernetes, the cluster scheduler decides where the pods live. After checking the pod IP addresses for the target service, I found that all the replicas were scheduled in ap-southeast-5a and ap-southeast-5c. There were zero pods running in ap-southeast-5b.

When the ALB sent a health check to the NLB IP in AZ 2, the NLB had no local pods to forward the traffic to. It dropped the packet, causing the ALB health check to time out.

Two ways to fix the routing

You can solve this at the application level or the infrastructure level.

1. Enforce pod distribution (Recommended)

You can update your Kubernetes deployment to use topologySpreadConstraints. This guarantees the scheduler places at least one pod in every Availability Zone.

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: internal-service

This is the better architectural choice. It ensures true high availability and keeps network traffic within the same zone.

2. Enable cross-zone load balancing

If you cannot change the pod scheduling rules, you can modify the NLB to allow cross-zone traffic.

resource "aws_lb" "internal_nlb" {
  # ... other settings
  enable_cross_zone_load_balancing = true
}

The NLB node in AZ 2 will now forward traffic to pods in AZ 1 or AZ 3. Be aware that AWS charges for inter-AZ data transfer. Enabling this setting on a high-traffic internal service will increase your monthly AWS bill.