Introduction

Istio is an open source service mesh that (in my opinion) can improve the developer experience of building kubernetes native enterprise microservices.

It provides a lot of features out of the box such as service discovery, load balancing, security, etc. but the one I have used the most is Auth¹.

In this post I will go over how you can implement payload based authorization in Istio. I am assuming that you have a basic understanding of how the AuthorizationPolicy CRD works in Istio. If you don’t, I would recommend reading the official documentation.

Problem Statement

Let’s say there is a service responsible for making configuration changes on networking devices. Members of different teams across the organization can use this to request changes via a shared UI. All the changes requested through this are applied in a maintenance window (MW).

An example request to configure an interface on a switch looks like the following:

POST /api/v1/change/interface HTTP/1.1

Authorization: "Bearer <User JWT>"
Content-Type: "application/json"

This endpoint accepts the following JSON body which specifies some fields that are used to configure the interface. This endpoint also optionally accepts a special field push which immediately pushes the config to the device rather than waiting for the MW.

{
    "shut": false,
    "vlan": 42,
    // "push": true
}

Since, specifying the push field can affect the production network, only some members in the organisation should be able to successfully push changes oustside of the MW.

Other (possible) solutions

Since payload based authorization in Istio is a bit complex, there are some other solutions that need to be ruled out first before using this approach is justified.

• client side authorization - this works well until anything other than the shared UI is calling this service - for e.g. one-off scripts, other microservices, V2 of the UI, etc.

• Authz Policy friendly v2 REST API - the REST API can be more friendly to authorization policies and specify the parameter as a query param or an HTTP header. However, if this is a fairly popular legacy service, migrating to a v2 API may require more effort and resources than the approach outlined below

• defer payload based authz to the application - this maybe valid in some cases, but this does mean there is more code to be maintained and tested in our application codebase.

Payload based authorization in Istio

The AuthorizationPolicy CRD is the main way to authorize access to resources in Isito. Since the AuthorizationPolicy CRD can not access the payload, we extract the push field from the payload into the HTTP headers with the help of an EnvoyFilter.

Let’s refine the problem further, for all incoming requests into the Istio Sidecar Proxy we want to inspect the payload, and set a value in the header based on the payload contents. A caveat is that we want this inspection and extraction to occur before the authorization policies are evaluated. This can be achieved by running inline Lua code from our EnvoyFilter

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: extract-field
  namespace: namespace
spec:
  workloadSelecctor:
    labels:
      # your selectors
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND 
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connectionn_manager"
              subFilter:
                name: "envoy.filters.network.http_connectionn_manager"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.lua
          typed_config:
            "@type": "type.gooogleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
            inlineCode: |
                LUA CODE

In the above manifest, 2 configuration options are of note:

• by setting context: SIDECAR_INBOUND in the match criteria, we ensure the filter only processes incoming traffic to the sidecar proxy.
• using INSERT_BEFORE with the http_connection_manager subfilter positions this custom processing before Istio’s authorization filter chain, allowing us to modify headers before the authorization policies are evaluated

The following is a sample of Lua code that works well for our problem statement

function envoy_on_request(request_handle)
    -- Remove the header from the incoming request
    request_handle:headers():remove("X-Service-Push")

    local raw_body = request_handle:body()
    if raw_body == nil or raw_body:length() == 0 then
        -- request body was empty
        return
    end
 
    local body = raw_body:getBytes(0, raw_body:length())

    -- hacky string check for field value
    local push_val = string.match(body, '"push"%s:%s*([true|false])')

    if push_val == nil or push_val == '' then
        -- field value wasn't a boolean
        return
    end

    -- add header
    request_handle:headers():add("X-Service-Push")
end

P.S. you can use request_handle:logInfo("log message") to log messages in the sidecar logs. This can be helpful for quick debugging.

After this our AuthorizationPolicy can access the header as shown in the sample below

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: auth-policy
  # ...
spec:
  action: ALLOW
  rules:
    # Allow requests always when push wasn't requested
    - to:
      - operation:
          methods: ["POST"]
          paths: ["/api/v1/change/interface"]
      when:
        - key: request.headers[X-Service-Push]
          values: ["false"]
    # Allow certain credentials with specific JWT claims to request a push
    - to:
      - operation:
          methods: ["POST"]
          paths: ["/api/v1/change/interface"]
      when:
        - key: request.headers[X-Service-Push]
          values: ["true"]
        - key: request.auth.claims[role]
          values: ["pusher", "superadmin", "cowboy"]

A better EnvoyFilter

One of the issues with the above EnvoyFilter is that it relies on a hacky string regex check. This is susceptible to bugs and is not ideal. Ideally we want to parse the JSON payload using a lua libray and then apply the filter value. Let’s say we want to use json.lua, the lua in our envoy filter will look something like the following

function envoy_on_request(request_handle)
    -- this needs lua.json to be installed
    json = require "json"

    -- Remove the header from the incoming request
    request_handle:headers():remove("X-Service-Push")

    local raw_body = request_handle:body()
    if raw_body == nil or raw_body:length() == 0 then
        -- request body was empty
        return
    end
 
    local body = raw_body:getBytes(0, raw_body:length())

    local success, decoded = pcall(function() return json.decode(body) end)
    if not success or decoded.push == nil or type(decoded.push) != "boolean" then
        -- JSON parsing failed OR "push" field is missing or "push" type is not boolean
        return
    end
   
    request_handle:headers():add("X-Service-Push", decoded.push)
end

To require lua.json in our envoy filter, we need to ensure that the sidecar running envoyproxy has lua.json installed. This can be done by using a custom image for our envoyproxy and configuring Istio to use this custom image². A sample custom image is shown below:

FROM envoyproxy/envoy:v1.28.0

RUN apt-get update && apt-get install -y wget
RUN wget https://raw.githubusercontent.com/rxi/json.lua/master/json.lua -O /usr/local/share/lua/json.lua

There are multiple ways of specifying a custom sidecar image in istio, a few of them are listed below:

• using an annotation - https://istio.io/latest/docs/reference/config/annotations/#SidecarProxyImage
• using the ProxyImage and ProxyConfig CRDs - https://istio.io/latest/docs/reference/config/networking/proxy-config/#ProxyImage

Epilogue

What I like about Istio

I like Istio (for auth specifically) because it removes the onus of authz and authn from me as the developer and leaves me to only worry about business logic. My application server can assume that if a request is reaching it, it has a valid JWT and that the caller is authorized access to this resource.

Not only does this simplify the application code itself, but it greatly simplifies local development as I can use any dummy JWT for running my server locally. Overall the DX is greatly improved!

What I don’t like about Istio

It took a while to wrap my head around AuthorizationPolicies, EnvoyFilters and Istio in general. Istio is very opiniated and at times feel bloated/over-engineered

Also, I dont like the fact that I have to do write YAML, ew. This page summarizes my general feelings towards YAML³.

Notes