18 Jan 2018
Early warning - Changes to 'List Objects' API
This is an early warning of changes we plan to make to the response to the GET /oss/v2/buckets/<bucket-name>/objects endpoint - to allow you to prepare in advance for these changes. This is the API to list the contents of a bucket stored in our Object Storage Service (OSS) The changes will likely go into production in early March. We will announce the exact date for these changes closer to the time, so 'watch this space'. We will also be directly contacting anyone who has used this endpoint using the email address associated with their Forge account. We apologize for any inconvenience that these changes may cause for you.
Read on for more details ...
The Forge team is working on changes to our backend systems to help us scale with the continually increasing traffic demands on our services. In doing so, we've attempted to maintain the API contracts where possible. However, the List Objects API will be changing in a future update in a way that will likely affect your application if you use this endpoint.
Changes:
- Results may no longer be in lexical order
- startAt query parameter values for getting the next page of results will be in a different format.
The implication of (1) is simply that your application should not assume that results will be returned in a particular order.
For (2), if your application or service is is simply reading the value for the "next" field in the response, then you can continue to do so without any changes - since that URL includes all the necessary query params to handle the request for the next page of results. However, if you are parsing that URL to extract parameters, then the change to the value of the startAt parameter may require a change to your code.
Let's take a look at a sample response and the expected changes to the Response results
Examples
As an example we take a bucket named "test-bucket" in which we uploaded 100 files named "file-0001" to "file-0100".
The original list api would have worked as follows.
GET /oss/v2/buckets/test-bucket/objects (ORIGINAL API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0001",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0001",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0001"
},
... 8 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0010",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0010",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0010"
}
],
"next" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects?startAt=file-0010"
}We could request the next page just by using the "next" field in the response.
GET /oss/v2/buckets/test-bucket/objects?startAt=file-0010 (ORIGINAL API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0011",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0011",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0011"
},
... 8 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0020",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0020",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0020"
}
],
"next" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects?startAt=file-0020"
}Until there's no "next" field.
GET /oss/v2/buckets/test-bucket/objects?startAt=file-0090 (ORIGINAL API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0091",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0091",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0091"
},
... 8 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0100",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0100",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 9,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0100"
}
]
}
Objects would be returned in alphabetical order, and the "startAt" query parameter in "next" field could easily be interpreted as a objectKey.
This is no longer the case. let's see what the new results would be.
GET /oss/v2/buckets/test-bucket/objects (UPDATED API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0061",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0061",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0061"
}, {
"bucketKey" : "test-bucket",
"objectKey" : "file-0098",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0098",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0098"
},
... 7 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0067",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0067",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0067"
}
],
"next" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects?startAt=7%7C"
}
Results are no longer in order, and the "next" field, while it can be used to fetch the next page, it is no longer a simple objectKey.
It should not matter what the startAt field is, as long as it is passed directly to get the next page.
GET /oss/v2/buckets/test-bucket/objects?startAt=7%7C (UPDATED API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0031",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0031",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0031"
}, {
"bucketKey" : "test-bucket",
"objectKey" : "file-0068",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0068",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0068"
},
... 7 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0001",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0001",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0001"
}
],
"next" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects?startAt=14%7Cfile-0001"
}
Copy the "next" field...
GET /oss/v2/buckets/test-bucket/objects?startAt=14%7Cfile-0001 (UPDATED API)
{
"items" : [
... 10 items ...
],
"next" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects?startAt=23%7C"
}
Until there's no "next" field as we arrive to the last page.
GET /oss/v2/buckets/test-bucket/objects?startAt=84%7C (UPDATED API)
{
"items" : [
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0029",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0029",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0029"
},
... 8 more items ...
{
"bucketKey" : "test-bucket",
"objectKey" : "file-0097",
"objectId" : "urn:adsk.objects:os.object:test-bucket/file-0097",
"sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size" : 0,
"location" : "https://developer.api.autodesk.com/oss/v2/buckets/test-bucket/objects/file-0097"
}
]
}