Tunneling traffic to Istio from localhost

I was curios if I am able to talk to Istio services from my local computer without doing any port forwarding or DNS entries for the services. And I still wanted leverage security features Istio provides and define auth policies for “humans”.

Prerequisites

• Working Istio (simple, the default installation is sufficient).
• Access to Istio certificates. You can use the default ones provisioned by Istio or plug in your own ones as described in this guide.
• Configured trustdomain. In this post I will be using the default cluster.local

Generating workload certificate

The whole purpose of creating workload certificates is to provide identity for my local machine. I don’t need to reinvent the wheel here so let’s simply stick with the way Istio does it - certificate containing SPIFFE ID in SAN value.

To get started, fetch the following certificates from Istio. I am the default certificates that Istio creates during installation.

• ca-cert.pem: the generated intermediate certificates
• ca-key.pem: the generated intermediate key
• cert-chain.pem: the generated certificate chain which is used by istiod
• root-cert.pem: the root certificate

Now generate my workload cert:

openssl genrsa -out workload-key.pem" 2048

cat << EOF > workload.cfg
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
countryName = EU
[v3_req]
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
basicConstraints = critical, CA:FALSE
subjectAltName = critical, @alt_names
[alt_names]
URI = spiffe://cluster.local/kirecek
EOF

openssl req -new -key workload-key.pem" -subj "/" -out workload.csr -config workload.cfg

openssl x509 -req -in workload.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial \
-out workload-cert.pem" -days 3650 -extensions v3_req -extfile workload.cfg

cat cert-chain.pem >> workload-cert.pem

That’s it. Now we have a certificate that we can use for communication with Istio services.

Reaching service from outside of the mesh

To enter the mesh, I need some Gateway with AUTO_PASSTHROUGH mode. And for a very simple reason - we want to terminate TLS in destination service sidecar so we just “pass” a request through the gateway to destination pod.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: autopass-gw
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*.cluster.local'
    port:
      name: tls
      number: 15443
      protocol: TLS
    tls:
      mode: AUTO_PASSTHROUGH

The gateway needs to accessible from my local mashine.

Building simple proxy

Well, of course I could just provide certificates to clients but I don’t want to touch application code.

This can be done in multiple ways, even maybe by local envoy, however I ended up writing very simple proxy service.

// simple helper for readying certificates
func loadCertFile(pool *x509.CertPool, path string) error {
	data, err := ioutil.ReadFile(path)
	if err != nil {
		return err
	}
	pool.AppendCertsFromPEM(data)
	return nil
}

func proxy(w http.ResponseWriter, req *http.Request) {
	u, _ := url.Parse(req.RequestURI)

	port := "80"
	if u.Port() != "" {
		port = u.Port()
	}

    // Format SNI to match routes on the gateway. Without this we will be getting 404.
    // To view the cluster configuration in the cluster you can use this example command:
    //     istioctl proxy-config cluster deploy/istio-ingress -n istio-system  -o=yaml | grep sni
	sni := fmt.Sprintf("outbound_.%s_._.%s", port, u.Hostname())

	caPool := x509.NewCertPool()
	if err := loadCertFile(caPool, "root-cert.pem"); err != nil {
		log.Fatalf("could not load certificate: %v", err)
	}
	
	certs, err := tls.LoadX509KeyPair("workload-cert.pem", "workload-key.pem")
	if err != nil {
		log.Fatalf("could not load certificate: %v", err)
	}

    // Create client with TLS config required for mTLS communication.
	client := &http.Client{
		Transport: &http2.Transport{
			TLSClientConfig: &tls.Config{
                RootCAs:      caPool,
                Certificates: []tls.Certificate{certs},
                ServerName:   sni,
                NextProtos: []string{
                    "istio-peer-exchange",
                    "istio",
                    "h2",
                },
                InsecureSkipVerify: true,
            },
			AllowHTTP:       true,
		},
	}

	req.URL = &url.URL{
		Host: "127.0.0.1:8443", // I proxy requests to port-forwared gateway. You can replace it with DNS pointing to your GW.
		Scheme:   "https",
		Path:     u.Path,
		RawQuery: u.RawQuery,
	}
	req.RequestURI = ""

    // proxy & tunnel the request via mTLS 
	resp, err := client.Do(req)
	if err != nil {
		http.Error(w, "Server Error", http.StatusInternalServerError)
		log.Fatal("ServeHTTP:", err)
	}
	defer resp.Body.Close()

    // Return response from proxied request.
	io.Copy(w, resp.Body)
}



func main() {
	http.HandleFunc("/", proxy)

	err := http.ListenAndServe(":9000", nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

Now, I should be able to call services running in Istio using the proxy I just wrote. In my cluster, I have httpbin running in the default namespace and it’s exposed on a port 8000 which means that other workloads in cluaster can access it via http://httpbin.default.svc.cluster.local:8000.

Since I am also part of the mesh, I can also do the same think via proxy!

curl -x localhost:8081 httpbin.default.svc.cluster.local:8000/headers

The request is valid mTLS request in the mesh where all Istio policies are applied and it will be nicely shown in Istio obervability (metrics, traces, logs).

Summary

Workload cert generation can be nicely automated and proxy could be something smarter, however, this was just for fun and I don’t recommend to follow this approach.