Request

Response

    Flow Graph Engine APIs Rate Limits and Quotas

    The Flow Graph Engine APIs observes rate limits and a set of quotas to ensure that all clients get optimal service experience and that runaway applications don’t consume excessive resources. APS Rate Limits and Quotas describes rate limits and quotas applied by the APS platform. This section will discuss rate limits and quotas specific to the Flow Graph Engine APIs. Quotas and limits are enforced individually for each APS application making requests when a 2-legged token is used to call the service. When a 3-legged token is used these will be applied for the authenticated user instead.

    Rate Limits

    Rate limits specify a maximum number of requests per minute.

    Scope

    The Flow Graph Engine APIs sets a separate rate limit for each application making requests (specified by client ID) per API endpoint.

    Violation Notification

    If an application exceeds an endpoint’s rate limit, the server returns an HTTP 429 error (described in detail in APS Rate Limits and Quotas).

    Endpoint Rate Limits

    Job API limits

    MethodEndpointLimit (requests/minute)
    GET /queues/{queueId}/jobs 60
    POST /queues/{queueId}/jobs 10
    POST /queues/{queueId}/jobs:cancel 10
    GET /queues/{queueId}/jobs:updates 60
    GET /queues/{queueId}/jobs/{jobId} 60
    GET /queues/{queueId}/jobs/{jobId}/executions 60
    GET /queues/{queueId}/jobs/{jobId}/executions:updates 60
    GET /queues/{queueId}/jobs/{jobId}/outputs 60
    GET /queues/{queueId}/jobs/{jobId}/logs 60
    GET /queues/{queueId}/jobs/{jobId}/tasks 60
    GET /queues/{queueId}/jobs/{jobId}/logs 60
    GET /queues/{queueId}/jobs/{jobId}/logs 60

    Resource API limits

    MethodEndpointLimit (requests/minute)
    GET /spaces/{spaceId} 60
    POST /spaces/{spaceId}/resources:batch-get-download-urls 60
    POST /spaces/{spaceId}/resources:batch-get-upload-urls 100
    GET /spaces/{spaceId}/resources/{resourceId} 60
    GET /spaces/{spaceId}/resources/{resourceId}/download-url 60
    GET /spaces/{spaceId}/resources/{resourceId}/upload-urls 100
    POST /spaces/{spaceId}/uploads 100
    POST /spaces/{spaceId}/uploads:batch-complete 100
    POST /spaces/{spaceId}/uploads:batch-create 100
    POST /spaces/{spaceId}/uploads:complete 100

    Storage provider limits

    Different storage provider will enforce different limits.

    Storage providerMaximum number of parts for multipart upload and download
    scratch 25
    outputs 25

    Consumption quotas

    Quotas are reset at midnight UTC. If a 3-legged token is used the quota will be applied to the authenticated user. If a 2-legged token is used, the quotas will be applied to the APS app.

    LimitQuota
    Number of tasks executed per day 25,000
    Number of concurrent tasks running per executor before being queued 20
    Number of hours of compute per day 500
    Total number of bytes uploaded to scratch storage provider in the last 30 days 500 GB
    Total number of bytes used in the output storage provider 5 TB
    Total number of items uploaded to the scratch storage provider in the last 30 days 50,000
    Total number of items in the output storage provider 1,000,000

    Maximum CPUs and Memory

    For a given task the maximum of vCPU that can be requested is 16 and the maximum memory is 122880 MBs.