Troubleshooting Slow AWS Direct Connect Performance and Latency Issues
AWS Direct Connect provides a private network path between an on-premises environment and AWS. It can deliver more predictable latency and higher throughput than internet-based VPN connectivity, but a provisioned circuit does not guarantee that every application will perform well. Congestion, routing asymmetry, packet loss, MTU mismatches, and overloaded network devices can all appear as “slow Direct Connect.”
Effective troubleshooting starts by separating a bandwidth problem from a latency problem. A circuit can have low utilization while applications experience delays caused by packet retransmissions, inefficient routing, DNS lookups, or a distant AWS Region. Likewise, a high-throughput file transfer may be limited by TCP window size or host CPU rather than the Direct Connect link itself.
Use measurements from both sides of the connection. AWS Direct Connect metrics, router interface counters, BGP logs, packet captures, and application timings provide a much clearer picture than a single speed test.
Establish A Baseline Before Changing Configuration
Record the expected path, including the customer router, Direct Connect location, virtual interface, Direct Connect gateway or transit gateway, and target VPC subnet. Identify whether traffic uses a private VIF, transit VIF, or public VIF. The path determines which routing tables and AWS resources must be inspected.
Collect latency and packet-loss measurements during normal and peak periods. Test from the same source and destination using repeated probes rather than relying on one ping. ICMP may be deprioritized or blocked, so compare it with TCP-based tests to the actual application endpoint. A useful baseline includes round-trip time, jitter, retransmissions, throughput, and connection setup time.
Check whether the symptom affects every workload or only a particular protocol. Slow SMB, database replication, and interactive SSH sessions often expose latency more clearly than large sequential transfers. If only one application is affected, inspect its connection pooling, timeouts, encryption overhead, and server-side performance before treating the circuit as the root cause.
Inspect Direct Connect And Router Health
In the AWS console, review the Direct Connect connection and virtual interface state. A connection that repeatedly changes state can cause BGP reconvergence and brief outages that users describe as slowness. CloudWatch metrics such as connection bytes, virtual interface traffic, connection errors, and BGP status help establish whether the issue is capacity, stability, or routing.
On the customer edge router, inspect interface errors, discards, CRC faults, overruns, duplex settings, queue drops, and CPU utilization. Optical handoff problems can produce errors even when the interface remains operational. For fiber connections, review receive and transmit light levels where supported, and compare them with the carrier’s acceptable range.
For LAG deployments, verify that member links are active and that traffic is distributed as expected. A single large flow usually remains on one hashed member, so aggregate LAG capacity does not automatically increase the speed of one TCP session. Check LACP state, hashing behavior, and per-member utilization instead of looking only at the bundle total.
Validate Routing And The End-To-End Path
Review BGP advertisements and learned routes on both the customer router and AWS side. Confirm that the preferred prefixes are using Direct Connect rather than an internet VPN, backup circuit, or unexpected transit path. AS-path prepending, local preference, MED, and static routes can influence return traffic and create asymmetric routing.
Asymmetric paths may still function, but they complicate packet inspection and can cause a firewall or stateful appliance to drop traffic. Compare the forward and return routes, especially when using a transit gateway, multiple Direct Connect locations, or redundant customer routers. AWS VPC route tables, transit gateway route tables, and network ACLs should all be checked for the intended destination and return path.
| Symptom | Likely Area | Useful Checks |
|---|---|---|
| High latency for all traffic | Routing, provider path, congestion | BGP path, traceroute, provider circuit data, regional distance |
| Low throughput with packet loss | Physical or Layer 2 issue | CRC errors, discards, optics, interface counters |
| One flow is slow while others are normal | TCP or application behavior | Window size, retransmissions, host CPU, parallel streams |
| Periodic pauses or disconnects | BGP or link instability | Connection state, BGP logs, LACP events, CloudWatch alarms |
| Large packets fail or stall | MTU or fragmentation | End-to-end MTU tests, firewall policies, encapsulation overhead |
| AWS access is unexpectedly indirect | Route selection | VIF type, transit gateway routes, VPN and static-route preference |
Test MTU, Packet Loss, And Throughput
An MTU mismatch can cause small requests to work while larger transfers stall. This is common when traffic crosses VPN overlays, firewalls, virtual appliances, or other encapsulated links. Test progressively larger packets with the “do not fragment” option from both directions, accounting for the operating system and device behavior.
Do not use a single threaded file copy as a definitive bandwidth test. TCP throughput depends on latency, receive buffers, congestion control, packet loss, and the number of parallel flows. A controlled test with a tool such as iperf3 can compare one stream with multiple streams, but it should run between approved test hosts and during a defined maintenance window.
Packet loss below one percent can severely reduce TCP performance on a high-latency path. Review interface drops, firewall logs, cloud network telemetry, and packet captures for retransmissions or duplicate acknowledgments. If packet loss begins outside the customer network, provide timestamps, source and destination prefixes, and circuit identifiers to the colocation or network provider.
Examine AWS Architecture And Application Behavior
A Direct Connect connection reaches an AWS location, not necessarily the workload’s physical Availability Zone. Accessing a distant Region or crossing several AWS network services can add latency. Confirm that workloads are deployed in the intended Region and that the selected VIF and gateway design matches the traffic pattern.
For VPC connectivity, inspect route tables, security groups, network ACLs, transit gateway attachments, and any inspection appliance. Stateful firewalls may become bottlenecks when connection counts, packet rates, or asymmetric flows increase. VPC Flow Logs can show accepted and rejected traffic, while Reachability Analyzer can validate whether a network path is possible; neither tool replaces packet-level performance testing.
Application timing often reveals the real bottleneck. Measure DNS resolution, TCP handshake, TLS negotiation, server processing, and payload transfer separately. Database replication may be constrained by commit latency, while file services may be limited by chatty protocols. Compression, caching, batching, and connection pooling can improve user experience without changing the Direct Connect circuit.
Coordinate Escalation With Evidence
Escalate to AWS or the Direct Connect provider when physical errors, repeated link flaps, unexplained packet loss, or abnormal optical levels remain after local checks. Include the connection ID, virtual interface ID, affected prefixes, timestamps in UTC, interface counters, BGP logs, and test results. Clear evidence shortens the time needed to isolate a carrier, AWS location, or customer-edge issue.
For persistent capacity constraints, evaluate a larger port, an additional connection, or a LAG. Redundancy should use separate devices and, where practical, separate Direct Connect locations or providers. A backup VPN can preserve connectivity during an outage, but its different MTU and latency characteristics should be tested before production failover.
Practical Checks To Prioritize
Use the following order to reduce unnecessary configuration changes:
- Compare AWS CloudWatch metrics with customer-router counters during the reported slowdown.
- Verify BGP advertisements, return routes, and whether traffic is bypassing Direct Connect.
- Test packet loss and MTU in both directions with timestamps and repeatable packet sizes.
- Compare single-stream and multi-stream throughput between controlled hosts.
- Review application timings and escalate with circuit, VIF, and provider evidence when the fault is outside the local network.
A disciplined workflow turns vague reports of “slow AWS” into measurable findings about latency, loss, path selection, capacity, or application behavior. Apply these checks to the affected Direct Connect topology, document the baseline, and use the resulting evidence to correct the responsible layer rather than simply increasing bandwidth.