Contract: Observer

What is it & why do I care about it?

The Observer Contract is an essential piece of Event Processing Workflow. Specifically, that workflow needs to invoke Observers in order to observe Resources. This contract defines how Pyrae interacts with Observer functions.

Related Objects/Processes

1. Event Processing Workflow
2. Observer
3. Observances
4. Resources

Version: 1

Description

This is the current version of the Observer contract, and new Observers should implement this format.

Input Structure

export interface ObserverLambdaInput {
    observer: {
        observerUrn: string;
        observerLambdaArn: string;
    };
    // a list of resources that we need to scan
    resources: Array<{

        // the Resource URN of this particular request
        urn: string;

        // metadata about the account
        account: {
            accountUrn: string;
            accountMetadata: any;
        };

        // credentials with permission to access that account, if the resource is located in AWS
        awsCredentials?: {
            accessKeyId: string;
            secretAccessKey: string;
            sessionToken?: string;
            expiration?: Date;
        };
    }>;
}

Output Structure

export interface ObserverLambdaOutput {
    // a list of Observances
    resourceResponses: Array<{
        // the response is always version 1
        version: 1;

        // In case the URN was non-canonical, it can be corrected by setting this field
        actuallyObservedUrn?: string;

        // the actual observance. It must be an object at the top level but it can be any shape within.
        observance: {
            [k: string]: any;
        };

        // This is a reference to the index in the request's `$.resources`. It must be set so that we know which request was fulfilled
        forRequestI: number;
    }>;
}

Usage Instructions

The Observer's input includes a batch of Resources. The Observer is expected to observe each input resource, and return the Observance in the output.

Each Resource in the input includes AWS Credentials, which is an AWS Assume Role Session. That session is constrained by the IAMPolicy defined on the Observer. The Resources in a single request may be in multiple accounts, so it's important to use the correct credentials and not reuse AWS SDK clients across resources.

The Observer's output is a batch of Observances. If the URN was non-canonical, the Observer can correct it by specifying an optional actuallyObservedUrn (Eg, if a stack changeset URN was provided, but the scanned object was actually the stack). If you wish to use this field, you must set the field PyraePermitReturningOtherResourcesFromObserver to true within the Observer's $.MetadataJson. This is because there are error cases in Observer post-processing for, for example, "urn was present in the output but missing in the output".

Each Observance must be tied back to the original request with the forRequestI field. This is required to keep a consistent trace thru the system so that we can say that a certain Observance is a consequence of a developer hitting a button, or a resource being changed, etc.

If you want to return multiple Observances per request, that's supported. Simply place multiple outputs in the output array with the same forRequestI, and set the field PyraePermitReturningOtherResourcesFromObserver to true within the Observer's $.MetadataJson.

Error Handling

The semantically correct error response is to return an Observance anyway, and include an "error" field in the Observance with a non-falsy value. For example:

return {
    version: 1,
    observance: {
        error: "An Exception Occurred",
        // or
        error: { "any_fields": 123 }
        // or any non-falsy value
    },
    forRequestI: 1
}

Observances with errors are not passed on to the Policy Contract, so you don't need to worry about that in policy writing.

If your Observer fails to meet this contract, or any of the later sanity checks, Pyrae may create an Observance with an error on your Observer's behalf. Pyrae will create an errored Observance in these scenarios:

1. If your Lambda function throws an uncaught exception, we will set the exception message for every Resource in the batch.
2. If you attempt to use actuallyObservedUrn without setting PyraePermitReturningOtherResourcesFromObserver.
3. If you attempt to return multiple Observations for the same forReqeustI, or you omit an index that is present in the request,either without setting PyraePermitReturningOtherResourcesFromObserver.
4. Any parsing error in this contract.

Sample

Request

{
    "observer": {
        "observerUrn": "urn:pyrae:observer:us-west-2:sAutx4ZxiqTJUzJdvky2km:observer/s3ApiGetBucketEncryptionObserver",
        "observerLambdaArn": "arn:aws:lambda:us-west-2:123456789012:function:PyraePoliciesLambdas-prod-s3ApiGetBucketEncryptionObserver",
    },
    "resources": [
        {
            "urn": "arn:aws:s3:::stackset-pyrae-onboarding-e23958-pyraetrailbucket-hi7fpkj4oa16",
            "account": {
                "accountUrn": "urn:pyrae:account:us-west-2:yR1c1WqSqEqryUc1HfrJhQ:account/AccountServicesProd-aZwRQ1gWw7B3wa2b1TW1LX",
                "accountMetadata": "{}"
            },
            "awsCredentials": {
                "accessKeyId": "your-access-key-id",
                "secretAccessKey": "your-secret-access-key",
                "sessionToken": "your-session-token",
                "expiration": "2023-06-01T00:00:00.000Z"
            }
        },
        {
            "urn": "arn:aws:s3:::pyraepolicieslambdas-dev-serverlessdeploymentbuck-nkbr4vjqhkl4",
            "account": {
                "accountUrn": "urn:pyrae:account:us-west-2:yR1c1WqSqEqryUc1HfrJhQ:account/AccountServicesProd-aZwRQ1gWw7B3wa2b1TW1LX",
                "accountMetadata": "{}"
            },
            "awsCredentials": {
                "accessKeyId": "your-access-key-id",
                "secretAccessKey": "your-secret-access-key",
                "sessionToken": "your-session-token",
                "expiration": "2023-06-01T00:00:00.000Z"
            }
        }
    ]
}

Response

{
    "resourceResponses": [
        {
            "version": 1,
            "forRequestI": 0,
            "observance": {
                "ServerSideEncryptionRule": {
                    "ApplyServerSideEncryptionByDefault": {
                        "SSEAlgorithm": "AES256"
                    }
                    "BucketKeyEnabled": true
                }
            }
        },
        {
            "version": 1,
            "forRequestI": 1,
            "observance": {
                "ServerSideEncryptionRule": {
                    "ApplyServerSideEncryptionByDefault": null
                }
            }
        }
    ]
}