Tunneling traffic to Istio from localhost

4 min read

I was curious if I could talk to Istio services from my local computer without doing any port forwarding or DNS entries for the services. And I still wanted to leverage the security features Istio provides and define auth policies for "humans".

Prerequisites

  • Working Istio (simple, the default installation is sufficient).
  • Access to Istio certificates. You can use the default ones provisioned by Istio or plug in your own ones as described in this guide.
  • Configured trustdomain. In this post I will be using the default cluster.local

Generating workload certificate

The whole purpose of creating workload certificates is to provide identity for my local machine. I don't need to reinvent the wheel here, so let's simply stick with the way Istio does it—a certificate containing a SPIFFE ID in the SAN value.

To get started, fetch the following certificates from Istio. I am the default certificates that Istio creates during installation.

  • ca-cert.pem: the generated intermediate certificates
  • ca-key.pem: the generated intermediate key
  • cert-chain.pem: the generated certificate chain which is used by istiod
  • root-cert.pem: the root certificate

Now generate your workload cert:

openssl genrsa -out workload-key.pem" 2048

cat << EOF > workload.cfg
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
countryName = EU
[v3_req]
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
basicConstraints = critical, CA:FALSE
subjectAltName = critical, @alt_names
[alt_names]
URI = spiffe://cluster.local/kirecek
EOF

openssl req -new -key workload-key.pem" -subj "/" -out workload.csr -config workload.cfg

openssl x509 -req -in workload.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial \
-out workload-cert.pem" -days 3650 -extensions v3_req -extfile workload.cfg

cat cert-chain.pem >> workload-cert.pem

That's it. Now we have a certificate that we can use for communication with Istio services.

Reaching service from outside of the mesh

To enter the mesh, I need some Gateway with AUTO_PASSTHROUGH mode. And for a very simple reason—we want to terminate TLS in the destination service sidecar, so we just "pass" a request through the gateway to the destination pod.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: autopass-gw
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*.cluster.local'
    port:
      name: tls
      number: 15443
      protocol: TLS
    tls:
      mode: AUTO_PASSTHROUGH

The gateway needs to be accessible from my local machine.

Building simple proxy

Well, of course I could just provide certificates to clients, but I don't want to touch application code.

This can be done in multiple ways, maybe even by local envoy; however, I ended up writing a very simple proxy service.

// simple helper for readying certificates
func loadCertFile(pool *x509.CertPool, path string) error {
	data, err := ioutil.ReadFile(path)
	if err != nil {
		return err
	}
	pool.AppendCertsFromPEM(data)
	return nil
}

func proxy(w http.ResponseWriter, req *http.Request) {
	u, _ := url.Parse(req.RequestURI)

	port := "80"
	if u.Port() != "" {
		port = u.Port()
	}

    // Format SNI to match routes on the gateway. Without this we will be getting 404.
    // To view the cluster configuration in the cluster you can use this example command:
    //     istioctl proxy-config cluster deploy/istio-ingress -n istio-system  -o=yaml | grep sni
	sni := fmt.Sprintf("outbound_.%s_._.%s", port, u.Hostname())

	caPool := x509.NewCertPool()
	if err := loadCertFile(caPool, "root-cert.pem"); err != nil {
		log.Fatalf("could not load certificate: %v", err)
	}
	
	certs, err := tls.LoadX509KeyPair("workload-cert.pem", "workload-key.pem")
	if err != nil {
		log.Fatalf("could not load certificate: %v", err)
	}

    // Create client with TLS config required for mTLS communication.
	client := &http.Client{
		Transport: &http2.Transport{
			TLSClientConfig: &tls.Config{
                RootCAs:      caPool,
                Certificates: []tls.Certificate{certs},
                ServerName:   sni,
                NextProtos: []string{
                    "istio-peer-exchange",
                    "istio",
                    "h2",
                },
                InsecureSkipVerify: true,
            },
			AllowHTTP:       true,
		},
	}

	req.URL = &url.URL{
		Host: "127.0.0.1:8443", // I proxy requests to port-forwared gateway. You can replace it with DNS pointing to your GW.
		Scheme:   "https",
		Path:     u.Path,
		RawQuery: u.RawQuery,
	}
	req.RequestURI = ""

    // proxy & tunnel the request via mTLS 
	resp, err := client.Do(req)
	if err != nil {
		http.Error(w, "Server Error", http.StatusInternalServerError)
		log.Fatal("ServeHTTP:", err)
	}
	defer resp.Body.Close()

    // Return response from proxied request.
	io.Copy(w, resp.Body)
}



func main() {
	http.HandleFunc("/", proxy)

	err := http.ListenAndServe(":9000", nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

Now, I should be able to call services running in Istio using the proxy I just wrote. In my cluster, I have httpbin running in the default namespace and it's exposed on port 8000, which means that other workloads in the cluster can access it via http://httpbin.default.svc.cluster.local:8000.

Since I am also part of the mesh, I can also do the same thing via the proxy!

curl -x localhost:8081 httpbin.default.svc.cluster.local:8000/headers

The request is a valid mTLS request in the mesh where all Istio policies are applied, and it will be nicely shown in Istio observability (metrics, traces, logs).

Summary

Workload cert generation can be nicely automated and the proxy could be something smarter; however, this was just for fun and I don't recommend following this approach.