Request Response Transformation

Function Description

The transformer plugin can transform request/response headers, request query parameters, and request/response body parameters. Supported transformation operation types include deletion, renaming, updating, adding, appending, mapping, and deduplication.

Execution Attributes

Plugin execution phase: authentication phase
Plugin execution priority: 410

Configuration Fields

The configuration fields for each item in reqRules and respRules are as follows:

The configuration fields for each item in headers, querys, body are as follows:

Note:

• request transformer supports the following transformation objects: request headers, request query parameters, request body (application/json, application/x-www-form-urlencoded, multipart/form-data).
• response transformer supports the following transformation objects: response headers, response body (application/json).
• The plugin supports bidirectional conversion capability, meaning that a single plugin can perform transformations on both requests and responses.
• The execution order of transformation operation types is the order written in the configuration file, e.g., remove → rename → replace → add → append → map → dedupe or dedupe → map → append → add → replace → rename → remove.
• When the transformation object is headers, key is case-insensitive. When headers are operated and are rename or map, value is also case-insensitive (as this field has a key meaning). However, key and value fields in querys and body are case-sensitive.
• value_type is only effective for content type application/json for request/response bodies.
• host_pattern and path_pattern support RE2 syntax, valid only for replace, add, append operations. In a transformation rule, only one of the two can be optionally filled. If both are filled, then host_pattern takes effect while path_pattern becomes ineffective.

Transformation Operation Types

Configuration Example

Implement Routing Based on Body Parameters

Configuration example:

reqRules:
- operate: map
  headers:
  - fromKey: userId
    toKey: x-user-id
  mapSource: body

This rule extracts the userId from the request body and sets it in the request header x-user-id. This allows routing based on body parameters using Higress’s ability to match on request headers.

This configuration supports both application/json and application/x-www-form-urlencoded types of request bodies.

For example: For application/json type body

curl localhost -d '{"userId":12, "userName":"johnlanni"}' -H 'content-type:application/json'

The value of the userId field will be extracted from the JSON and set to x-user-id. The backend service will receive a request header with: x-user-id: 12.

After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header.

For application/x-www-form-urlencoded type body

curl localhost -d 'userId=12&userName=johnlanni'

The value of the userId field will be extracted from the form format k1=v1&k2=v2 and set to x-user-id. The backend service will receive a request header with: x-user-id: 12.

After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header.

JSON Path Support

You can extract fields from complex JSON according to GJSON Path syntax.

Common operations include, for the following JSON:

{
  "name": {"first": "Tom", "last": "Anderson"},
  "age": 37,
  "children": ["Sara","Alex","Jack"],
  "fav.movie": "Deer Hunter",
  "friends": [
    {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]},
    {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]},
    {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]}
  ]
}

You can achieve such extractions:

name.last              "Anderson"
name.first             "Tom"
age                    37
children               ["Sara","Alex","Jack"]
children.0             "Sara"
children.1             "Alex"
friends.1              {"first": "Roger", "last": "Craig", "age": 68}
friends.1.first        "Roger"

Now, if you want to extract the first field from the second item in friends from the above JSON formatted body and set it to the header x-first-name, while also extracting the last field to set it to the header x-last-name, you can use this plugin configuration:

reqRules:
- operate: map
  headers:
  - fromKey: friends.1.first
    toKey: x-first-name
  - fromKey: friends.1.last
    toKey: x-last-name
  mapSource: body

Request Transformer

Transforming Request Headers

reqRules:
- operate: remove
  headers:
  - key: X-remove
- operate: rename
  headers:
  - oldKey: X-not-renamed
    newKey: X-renamed
- operate: replace
  headers:
  - key: X-replace
    newValue: replaced
- operate: add
  headers:
  - key: X-add-append
    value: host-\$1
    host_pattern: ^(.*)\.com$
- operate: append
  headers:
  - key: X-add-append
    appendValue: path-\$1
    path_pattern: ^.*?\/(\w+)[\?]{0,1}.*$
- operate: map
  headers:
  - fromKey: X-add-append
    toKey: X-map
- operate: dedupe
  headers:
  - key: X-dedupe-first
    strategy: RETAIN_FIRST
  - key: X-dedupe-last
    strategy: RETAIN_LAST
  - key: X-dedupe-unique
    strategy: RETAIN_UNIQUE

Send Request

$ curl -v console.higress.io/get -H 'host: foo.bar.com' \
-H 'X-remove: exist' -H 'X-not-renamed:test' -H 'X-replace:not-replaced' \
-H 'X-dedupe-first:1' -H 'X-dedupe-first:2' -H 'X-dedupe-first:3' \
-H 'X-dedupe-last:a' -H 'X-dedupe-last:b' -H 'X-dedupe-last:c' \
-H 'X-dedupe-unique:1' -H 'X-dedupe-unique:2' -H 'X-dedupe-unique:3' \
-H 'X-dedupe-unique:3' -H 'X-dedupe-unique:2' -H 'X-dedupe-unique:1'
# httpbin response result
{
  "args": {},
  "headers": {
    ...
    "X-Add-Append": "host-foo.bar,path-get",
    ...
    "X-Dedupe-First": "1",
    "X-Dedupe-Last": "c",
    "X-Dedupe-Unique": "1,2,3",
    ...
    "X-Map": "host-foo.bar,path-get",
    "X-Renamed": "test",
    "X-Replace": "replaced"
  },
  ...
}

Transforming Request Query Parameters

reqRules:
- operate: remove
  querys:
  - key: k1
- operate: rename
  querys:
  - oldKey: k2
    newKey: k2-new
- operate: replace
  querys:
  - key: k2-new
    newValue: v2-new
- operate: add
  querys:
  - key: k3
    value: v31-\$1
    path_pattern: ^.*?\/(\w+)[\?]{0,1}.*$
- operate: append
  querys:
  - key: k3
    appendValue: v32
- operate: map
  querys:
  - fromKey: k3
    toKey: k4
- operate: dedupe
  querys:
  - key: k4
    strategy: RETAIN_FIRST

Send Request

$ curl -v "console.higress.io/get?k1=v11&k1=v12&k2=v2"
# httpbin response result
{
  "args": {
    "k2-new": "v2-new",
    "k3": [
      "v31-get",
      "v32"
    ],
    "k4": "v31-get"
  },
  ...
  "url": "http://foo.bar.com/get?k2-new=v2-new&k3=v31-get&k3=v32&k4=v31-get"
}

Transforming Request Body

reqRules:
- operate: remove
  body:
  - key: a1
- operate: rename
  body:
  - oldKey: a2
    newKey: a2-new
- operate: replace
  body:
  - key: a3
    newValue: t3-new
    value_type: string
- operate: add
  body:
  - key: a1-new
    value: t1-new
    value_type: string
- operate: append
  body:
  - key: a1-new
    appendValue: t1-\$1-append
    value_type: string
    host_pattern: ^(.*)\.com$
- operate: map
  body:
  - fromKey: a1-new
    toKey: a4
- operate: dedupe
  body:
  - key: a4
    strategy: RETAIN_FIRST

Send Requests: 1. Content-Type: application/json

$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \
-H 'Content-Type: application/json' -d '{"a1":"t1","a2":"t2","a3":"t3"}'
# httpbin response result
{
  ...
  "headers": {
    ...
    "Content-Type": "application/json",
    ...
  },
  "json": {
    "a1-new": [
      "t1-new",
      "t1-foo.bar-append"
    ],
    "a2-new": "t2",
    "a3": "t3-new",
    "a4": "t1-new"
  },
  ...
}

2. Content-Type: application/x-www-form-urlencoded

$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \
-d 'a1=t1&a2=t2&a3=t3'
# httpbin response result
{
  ...
  "form": {
    "a1-new": [
      "t1-new",
      "t1-foo.bar-append"
    ],
    "a2-new": "t2",
    "a3": "t3-new",
    "a4": "t1-new"
  },
  "headers": {
    ...
    "Content-Type": "application/x-www-form-urlencoded",
    ...
  },
  ...
}

3. Content-Type: multipart/form-data

$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \
-F a1=t1 -F a2=t2 -F a3=t3
# httpbin response result
{
  ...
  "form": {
    "a1-new": [
      "t1-new",
      "t1-foo.bar-append"
    ],
    "a2-new": "t2",
    "a3": "t3-new",
    "a4": "t1-new"
  },
  "headers": {
    ...
    "Content-Type": "multipart/form-data; boundary=------------------------1118b3fab5afbc4e",
    ...
  },
  ...
}

Response Transformer

Similar to Request Transformer, this only describes the precautions for transforming JSON-formatted request/response bodies:

Key Nesting .

1. In general, a key containing . indicates a nested meaning, as follows:

respRules:
- operate: add
  body:
  - key: foo.bar
    value: value

$ curl -v console.higress.io/get
# httpbin response result
{
 ...
 "foo": {
  "bar": "value"
 },
 ...
}

2. When using \. to escape . in the key, it indicates a non-nested meaning, as follows:

When enclosing a string with double quotes, use \\. for escaping

respRules:
- operate: add
  body:
  - key: foo\.bar
    value: value

$ curl -v console.higress.io/get
# httpbin response result
{
 ...
 "foo.bar": "value",
 ...
}

Accessing Array Elements .index

You can access array elements by their index array.index, where the index starts from 0:

{
  "users": [
    {
      "123": { "name": "zhangsan", "age": 18 }
    },
    {
      "456": { "name": "lisi", "age": 19 }
    }
  ]
}

1. Remove the first element of user:

reqRules:
- operate: remove
  body:
  - key: users.0

$ curl -v -X POST console.higress.io/post \
-H 'Content-Type: application/json' \
-d '{"users":[{"123":{"name":"zhangsan"}},{"456":{"name":"lisi"}}]}'
# httpbin response result
{
  ...
  "json": {
    "users": [
      {
        "456": {
          "name": "lisi"
        }
      }
    ]
  },
  ...
}

2. Rename the key 123 of the first element of users to msg:

reqRules:
- operate: rename
  body:
  - oldKey: users.0.123
    newKey: users.0.first

$ curl -v -X POST console.higress.io/post \
-H 'Content-Type: application/json' \
-d '{"users":[{"123":{"name":"zhangsan"}},{"456":{"name":"lisi"}}]}'
# httpbin response result
{
  ...
  "json": {
    "users": [
      {
        "msg": {
          "name": "zhangsan"
        }
      },
      {
        "456": {
          "name": "lisi"
        }
      }
    ]
  },
  ...
}

Iterating Array Elements .#

You can use array.# to iterate over an array:

❗️This operation can only be used in replace, do not attempt this operation in other transformations to avoid unpredictable results

{
  "users": [
    {
      "name": "zhangsan",
      "age": 18
    },
    {
      "name": "lisi",
      "age": 19
    }
  ]
}

reqRules:
- operate: replace
  body:
  - key: users.#.age
    newValue: 20

$ curl -v -X POST console.higress.io/post \
-H 'Content-Type: application/json' \
-d '{"users":[{"name":"zhangsan","age":18},{"name":"lisi","age":19}]}'
# httpbin response result
{
  ...
  "json": {
    "users": [
      {
        "age": "20",
        "name": "zhangsan"
      },
      {
        "age": "20",
        "name": "lisi"
      }
    ]
  },
  ...
}