Version: 2.0.0

PDP Sharding

PDP Sharding is a technique for scaling Policy Decision Points (PDPs) horizontally, by splitting the data into multiple PDPs, each responsible for a subset of your tenants.

Scaling up data with PDP sharding

When to use PDP Sharding

PDP Sharding is useful when you need to scale your PDP horizontally, In some cases it is more cost effective to run multiple smaller PDPs, than a single large one because the policy engines might not handle large amount of data as quickly as you need them to.

How it works ?

Each PDP is run with a specific shard ID (a number between 0 and the number of shards - 1), each shard will have only the data of the tenants associated with it. On top of your PDPs an Envoy server will proxy each authorization query request to handle the consistent hashing algorithm that will be used to map each request by its tenant to the consistent shard responsible for it. This is done by utilizing Envoy's native support for “Hash ring” load balancing. That is, choosing an upstream host using a consistent hashing algorithm (“Ketama”).

Data changes tracked by Permit.io are propagated to the specific PDP shard that is responsible for associated with them. The same consistent hashing algorithm will be used to map the data to the correct shard.

Basic Architecture

The flow of an authorization request will be as follows:

To achieve sharding, multiple PDPs should be run. Currently, a predetermined number of PDPs is assumed, although dynamic scaling would be available soon as well (upcoming feature).

For each SDK query to reach the correct PDP instance (holding the shard that has the correct tenant id), we are using Envoy. The SDK is configured with Envoy’s address as its PDP, and Envoy forwards the requests based on the Tenant ID to the corresponding PDP. Permit.io's backend uses that very same sharding algorithm, Thus both the locally deployed Envoy proxy and the Permit.io backend are mapping the same tenant ids to the same shard ids.

Quickstart

An Envoy container with a custom configuration should be deployed next to the set of PDPs. See quick guide here. The following steps will guide you through the process of setting up 7 shards PDP cluster with Envoy:

Create the envoy configuration file named envoy_custom.yaml :

admin:
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
  listeners:
    - name: sdk_requests
      address:
        socket_address: { address: 0.0.0.0, port_value: 10000 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: AUTO
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route:
                            cluster: pdp_cluster
                            hash_policy:
                              - header:
                                  header_name: X-Tenant-Id
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
    - name: pdp_cluster
      connect_timeout: 1s
      type: STRICT_DNS
      lb_policy: RING_HASH
      ring_hash_lb_config:
        maximum_ring_size: 1024
      dns_lookup_family: V4_ONLY
      load_assignment:
        cluster_name: pdp_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-0
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "0"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-1
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "1"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-2
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "2"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-3
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "3"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-4
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "4"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-5
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "5"
              - endpoint:
                  address:
                    socket_address:
                      address: pdp-shard-6
                      port_value: 7000
                metadata:
                  filter_metadata:
                    envoy.lb:
                      hash_key: "6"

Perform HTTP request to notify a migration to sharded PDP cluster is being made ( this request must be done with env level api-key ), the response json will contain a client_secret that should be used in the next steps as the new cluster PDP_API_KEY:
```
curl https://api.permit.io/v2/pdps/{proj_id}/{env_id}/configs/migrate-shards \
-X POST \
-H "Authorization: Bearer <PDP_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
    "num_shards": 7
}'
```

Copy the following docker-compose file to your local machine and replace the '<PDP_API_KEY>' with your API key and '<PATH_TO_ENVOY_CONV>' with the path to the envoy configuration file you created in step 1:

version: "3.9"
networks:
  pdp-net:
    name: pdp-net
services:
  envoy-proxy:
    image: envoyproxy/envoy:v1.26-latest
    networks:
      - pdp-net
    ports:
      - "9901:9901"
      - "10000:10000"
    volumes:
      - "<PATH_TO_ENVOY_CONV>:/conf"
    command:
      - "-c /conf/envoy_custom.yaml"
  pdp-shard-0:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=0
    ports:
      - "18180:8181"
  pdp-shard-1:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=1
    ports:
      - "18181:8181"
  pdp-shard-2:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=2
    ports:
      - "18182:8181"
  pdp-shard-3:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=3
    ports:
      - "18183:8181"
  pdp-shard-4:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=4
    ports:
      - "18184:8181"
  pdp-shard-5:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=5
    ports:
      - "18185:8181"
  pdp-shard-6:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=6
    ports:
      - "18186:8181"

Execute the docker-compose file using the following command:
```
docker-compose up
```
Perform HTTP request to notify a migration to sharded PDP cluster is being made ( this request must be done with env level api-key ), the response json will contain a client_secret that should be used in the next steps as the new cluster PDP_API_KEY:
```
curl https://api.permit.io/v2/pdps/{proj_id}/{env_id}/configs/migrate-shards \
-X POST \
-H "Authorization: Bearer <PDP_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
    "num_shards": 7
}'
```

version: "3.9"
networks:
  pdp-net:
    name: pdp-net
services:
  envoy-proxy:
    image: envoyproxy/envoy:v1.26-latest
    networks:
      - pdp-net
    ports:
      - "9901:9901"
      - "10000:10000"
    volumes:
      - "<PATH_TO_ENVOY_CONV>:/conf"
    command:
      - "-c /conf/envoy_custom.yaml"
  pdp-shard-0:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=0
    ports:
      - "18180:8181"
  pdp-shard-1:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=1
    ports:
      - "18181:8181"
  pdp-shard-2:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=2
    ports:
      - "18182:8181"
  pdp-shard-3:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=3
    ports:
      - "18183:8181"
  pdp-shard-4:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=4
    ports:
      - "18184:8181"
  pdp-shard-5:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=5
    ports:
      - "18185:8181"
  pdp-shard-6:
    image: permitio/pdp-v2:0.2.19-rc.1
    networks:
      - pdp-net
    environment:
      - PDP_API_KEY=<PDP_API_KEY>
      - PDP_SHARD_ID=6
    ports:
      - "18186:8181"

Execute the docker-compose file using the following command:
```
docker-compose up
```

Finally you can send an authorization check request to the Envoy container:

cURL
Python
Node.js

curl http://localhost:10000/allowed \
    -X POST \
    -H "X-Tenant-Id: default" \
    -H "Authorization: Bearer <PDP_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{
            "user": {
                "key": "user@example.com"
            },
            "resource": {
                "tenant": "default",
                "type": "document",
            },
                "action": "read"
        }'

await permit.check(
    # user
    "user@example.com",
    # action
    "read",
    # resource
    {
        "type": "document",
        "tenant": "default",
    },
)

await permit.check(
    // user
    "user@example.com",
    // action
    "read",
    // resource
    {
        type: "document",
        tenant: "default",
    }
);

Note: Sharded PDPs are currently available only for early-access-program customers, make sure to contact us to get access to it.

When to use PDP Sharding​

How it works ?

Basic Architecture​

Quickstart​

Contents

When to use PDP Sharding

Basic Architecture

Quickstart