Categories
Uncategorized

Adventures with Linux Outline Client and aws-iam-authenticator

Hi all,

below is a small engineering puzzle that I had to solve recently. The essential components:

  • a Linux Laptop (in my case, running the excellent ClearLinux distribution)
  • aws-iam-authenticator
  • Outline client (A shadowSocks client)

The setup was the following. A Kubernetes cluster, a bastion host using Outline as the means to connect and access the cluster. In the ~/.kube/config you can see the following stanza:

name: k8s.dev.SNIPPED

user:

exec: apiVersion: client.authentication.k8s.io/v1alpha1 args: [“token”, “–cache”, “-i”, “k8s.dev.SNIPPED”] command: aws-iam-authenticator

[SNIPPED]

Issuing commands such as kubectl get pods would fail, with a DNS resolution error Outline Client was enabled. The root cause for this was that our setup, UDP traffic was disabled over Outline. However, Outline would take over /etc/resolv.conf and add a options use-vc line, indicating that ALL DNS resolutions should happen over TCP.

aws-iam-authenticator communicates under the hood with https://sts.amazonaws.com and attempts to resolve this hostname using UDP. This does not play well with the existing Outline Client setup and eventually will fail with an i/o timeout along the lines of 10.0.85.1:highport -> 1.1.1.1:53:udp.

The easiest way I have found to fix this was the following: modify the routing table AFTER Outline client takes over. For my home network this can look along the lines of:

sudo route add -host 1.1.1.1 gw fritz.box wlp2s0

and Presto! DNS resolution works again for aws-iam-authenticator and kubectl workflow can proceed as normal. I tried experimenting with

export GODEBUG=netdns=cgo
export GODEBUG=netdns=go

but with both flavors of the resolver, it did not honor the options-vc.

Hope this is helpful to other people! Until next time!

Leave a Reply

Your email address will not be published. Required fields are marked *