Direct-S3 upload and download with SDKs

Note: the SDKs we are referring to in this blog post are developed and maintained by our developer advocacy team, and are not considered official. There is an ongoing effort to provide official SDKs for Autodesk Platform Services, and we plan to release the official OSS SDKs early next year.

In March 2022 we announced a new, optimized approach to upload/download data to/from our services (https://forge.autodesk.com/blog/data-management-oss-object-storage-service-migrating-direct-s3-approach), together with a reference implementation of the new logic for .NET and Node.js. Now we're happy to announce that the new direct-S3 upload/download has been added to our SDKs for .NET and Node.js as well. Let's take a look at how you can enable the new, efficient data transfer in your own applications.

Uploading to OSS

To upload designs to OSS (Object Storage Service), applications would typically use the ObjectsApi#uploadObject method from the SDKs, for example, like so:

In .NET:

// ...

var api = new ObjectsApi();
api.Configuration.AccessToken = token;
var obj = await api.UploadObjectAsync(bucketKey, objectKey, contentLength, content);

// ...

In Node.js:

// ...

const api = new ObjectsApi();
const obj = await api.uploadObject(bucketKey, objectKey, contentLength, content, {}, null, token);

// ...

This method is still available but is now marked as deprecated. In order to switch to the direct-S3 upload, upgrade to the latest version of our SDK, and use the ObjectsApi#uploadResources method instead:

In .NET:

// Upgrade the `Autodesk.Forge` NuGet package to version 1.9.7 or newer
var api = new ObjectsApi();
api.Configuration.AccessToken = token;
var results = await api.uploadResources(bucketKey, new List<UploadItemDesc> {
    new UploadItemDesc(objectKey, content)
});

In Node.js:

// Upgrade the `forge-apis` npm package to version 0.9.4 or newer
const api = new ObjectsApi();
const results = await api.uploadResources(
    bucketKey,
    [{ objectKey: objectKey, data: content }],
    {},
    null,
    token
);

The uploadResources method accepts multiple objects to be uploaded at the same time, and it always returns a list of results corresponding to the individual objects being uploaded. Each result includes properties such as error (a boolean flag indicating whether the upload failed), completed (complete response for the upload, containing object details in case of success, or error details in case the upload failed).

In .NET:

var results = await api.uploadResources(bucketKey, new List<UploadItemDesc> {
    new UploadItemDesc(objectKey1, content1),
    new UploadItemDesc(objectKey2, content2),
    new UploadItemDesc(objectKey3, content3)
});
foreach (var result in results)
{
    if (result.Error)
    {
        throw new Exception(string.Format("Upload failed: {0}", result.completed.ToString()));
    }
    else
    {
        var json = result.completed.ToJson();
        Console.WriteLine(json.ToObject<ObjectDetails>());
    }
}

In Node.js:

const results = await api.uploadResources(
    bucketKey,
    [
        { objectKey: objectKey1, data: content1 },
        { objectKey: objectKey2, data: content2 },
        { objectKey: objectKey3, data: content3 }
    ],
    {},
    null,
    token
);
for (const result of results) {
    if (result.error) {
        throw `Upload failed: ${result.completed}`;
    } else {
        console.log(result.completed);
    }
}

You can also pass additional options to the uploadResources method to adjust its behavior, a callback function to monitor the upload progress, or a callback function to refresh the access token in case the upload takes too long. For more information, please refer to the SDK repository on Github:

You can also check out the Simple Viewer tutorial which already uses the new SDK methods.

Downloading from OSS

To download objects from OSS, applications would typically use the ObjectsApi#getObject method from the SDKs (.NET, Node.js). With the updated SDK you can now use the ObjectsApi#downloadResources method which has a similar signature to ObjectsApi#uploadResources.

In .NET:

var results = await ObjectsAPI.downloadResources(
    bucketKey,
    new List<DownloadItemDesc>() {
        new DownloadItemDesc(objectKey1, "arraybuffer"),
        new DownloadItemDesc(objectKey2, "arraybuffer"),
        new DownloadItemDesc(objectKey3, "arraybuffer")
	},
	new Dictionary<string, object>() {
        // { "minutesExpiration", 5 },
        // { "useCdn", true }
    },
    // onDownloadProgress,
    // onRefreshToken
);

In Node.js:

const results = await objectsApi.downloadResources(
    bucketKey,
    [
        { objectKey: objectKey1, responseType: 'arraybuffer' },
        { objectKey: objectKey2, responseType: 'arraybuffer' },
        { objectKey: objectKey3, responseType: 'arraybuffer' }
    ],
    {
        // minutesExpiration: 5,
        // useCdn: true,
        // onDownloadProgress: (data) => { ... },
        // onRefreshToken: async () => { ... },
    },
	oAuthTwoLeggedClient,
    oAuthTwoLeggedClient.getCredentials()
);

For more details on the different options, please refer to the SDK repository on GitHub:

Tags:

Code Samples

Back to Community Blog

Direct-S3 upload and download with SDKs

Uploading to OSS

Downloading from OSS

Related Article

Resources