Scenario
An organisation wanted to deploy each application into a separate Kubernetes namespace. Each application will be available at a subdomain of example.com, via a wildcard DNS entry of *.example.com pointing to the ingress controller’s service IP address. A single wildcard TLS certificate ( *.example.com ) will be used to protect all applications using the ingress controller. It was desired that only a single TLS secret should exist on the cluster to facilitate certificate renewal.
Challenges
-
Kubernetes secrets are only accessible from the namespace in which they are created.
We discussed having a single namespace with all ingress resources and the secret, however still keep each application’s services and deployments in separate namespaces.
-
nginx Ingress resources can only route to backend services within their own namespace.
Looked at http://voyager.appscode.com/ which can route to other namespaces, however this creates a new IP address for each application, which is not compatible with using a wildcard DNS name.
Another potential solution was was to use an service of type ExternalName to route traffic to the other namespace. Here’s how this worked:
Deployment
Install ingress controller into an ingress namespace
helm install --namespace ingress --name ingress stable/nginx-ingress --set rbac.create=false --set rbac.createRole=false --set rbac.createClusterRole=false
Create TLS secret in ingress Namespace
As this was not a live deployment we created a self signed wildcard certificate. In production a certificate would be acquired from a trusted certification authority:
openssl req -new -x509 -keyout wildcard.example.com.key -out wildcard.example.com.pem -days 365 -nodes -key wildcard.example.com.key
Then create a secret using that certificate:
kubectl create secret tls wildcard-example-com --key wildcard.example.com.key --cert wildcard.example.com.pem -n ingress
Create Helm Chart
Run the following command to create a new Helm chart in a directory named sample-app:
helm create sample-app
Add External Service
Added a service-ingress.yaml
file to the Helm chart templates directory to create a service of type ExternalName that deploys into the ingress namespace:
{{- $fullName := include "sampleapp.fullname" . -}}
apiVersion: v1
kind: Service
metadata:
name: {{ $fullName }}
namespace: ingress
spec:
ports:
- name: http
port: 80
type: ExternalName
externalName: {{ $fullName }}.{{ .Release.Namespace }}
Configure Ingress resource to deploy into the Ingress namespace
Specified a namespace to deploy the Ingress resource to by modifying the ingress.yaml
file
metadata:
name: {{ $fullName }}
namespace: ingress
Update the helm chart values.yaml
Specify image:
image:
repository: marrobi/linuxwebsite
tag: latest
pullPolicy: Always
Specify TLS settings:
tls:
- secretName: wildcard-example-com
hosts:
- sample-app.example.com
Deploy sample application
Deploy application into a new namespace called sample:
helm upgrade --install sample ./sample-app/ --namespace sample
Create DNS name
Retrieve the ingress controller’s IP address:
kubectl get service -n ingress
Record the EXTERNAL-IP of the ingress controller:
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-ingress-controller 10.0.57.175 51.140.163.143 80:32164/TCP,443:32638/TCP 11m
ingress-nginx-ingress-default-backend 10.0.254.194 <none> 80/TCP 11m
sample-sampleapp sample-sampleapp.sample 80/TCP 32s
For testing purposes I added a entry to the hosts file on my local machine, in the real world this would be an A record with your public DNS provider:
51.140.163.143 sample-app.example.com
Test the solution
Navigate to https://sample-app.example.com in a browser:
This solution works, however there may be other, "better" solutions. Do let me know if you a have an alternative solution.
Hi,
I'm wondering If you could give access to all the files involved in your sample. Sometimes, it's not clear between the "service.yml" or the "app.yml".
Thank you
Hi,
I spent a lot of time trying to make all this work and I succeeded!
The important thing that was missing in the description of the externalName is: .svc.cluster.local
So in the the conf for the service-ingress.yaml is :
{{- $fullName := include "sampleapp.fullname" . -}}
.svc.cluster.localapiVersion: v1
kind: Service
metadata:
name: {{ $fullName }}
namespace: ingress
spec:
ports:
- name: http
port: 80
type: ExternalName
externalName: {{ $fullName }}.{{ .Release.Namespace }}
Hope it will help