3 Sep 2020

Speed up viewable generation when using Design Automation for Inventor

When using Design Automation, you often want to show the updated model to the user as soon as possible. That means, after updating the model you also need to generate the viewables (SVF bubble) in order to show the updated model in the Forge Viewer.

People tend to create the SVF for their models using the Model Derivative service, however, if you are using Design Automation already then that would mean one extra step. If you are only interested in the viewables of the updated model and do not need the updated Inventor documents, then this approach would mean two extra steps: pushing the updated model to OSS + generating viewables for it using Model Derivative:

Model Derivative for SVF

In case of using Design Automation for Inventor, you can also create the viewables directly on the server. Here is a function that could be used in your AppBundle's code:

private void ExportToSvf(Document doc, string outputFolderPath)
{
    LogTrace($"** Saving SVF");
    try
    {
        TranslatorAddIn oAddin = inventorApplication.ApplicationAddIns.ItemById["{C200B99B-B7DD-4114-A5E9-6557AB5ED8EC}"] as TranslatorAddIn;

        TranslationContext oContext = inventorApplication.TransientObjects.CreateTranslationContext();
        oContext.Type = IOMechanismEnum.kFileBrowseIOMechanism;

        NameValueMap oOptions = inventorApplication.TransientObjects.CreateNameValueMap();
        DataMedium oData = inventorApplication.TransientObjects.CreateDataMedium();

        oData.FileName = System.IO.Path.Combine(outputFolderPath, "result.collaboration");

        // Setup SVF options
        if (oAddin.get_HasSaveCopyAsOptions(doc, oContext, oOptions))
        {
            oOptions.set_Value("EnableExpressTranslation", false);
            oOptions.set_Value("SVFFileOutputDir", outputFolderPath);
            oOptions.set_Value("ExportFileProperties", true);
            oOptions.set_Value("ObfuscateLabels", false);
        }

        oAddin.SaveCopyAs(doc, oContext, oOptions, oData);
        Trace.TraceInformation("SVF can be exported.");
        LogTrace($"** Saved SVF as {oData.FileName}");
    }
    catch (Exception e)
    {
        LogError($"********Export to format SVF failed: {e.Message}");
    }
}

Here are all the options for the SVF translator:

"EnableExpressTranslation": 
  Boolean value, indicates if use express producer when the source assembly is opened in express mode.
"SVFFileOutputDir": 
  String value, specify the output directory where the intermediate SVF files will be copied to. If not 
specified, the SVF intermediate files will be deleted after .collaboration file generated. 
"ExportFileProperties": 
  Boolean value, indicates whether export the file properties to SVF or not. The default is True.
"ObfuscateLabels": 
  Boolean value, indicates whether obfuscate the labels in model browser. The default value is False. The 
rule of label obfuscation are: 
  - Obfuscate the document/component name as simple number, like '7'.
  - Name the instance of the document as [Document Name]:[Index], like '7:1'.
  - Don't obfuscate the top node which is just the document name.
  - Don't obfuscate the name for BRep object, names like 'Solid 1' will be kept.

You have to decide how you want to provide the SVF's content to the Viewer:
a) Store it on your server
You could get Design Automation to zip up the folder with the SVF content and send it to you server. Then you could unzip it there and expose an endpoint on your server through which the Viewer can access the files 

b) Store it on OSS (or some other online storage)
If you don't want to store the SVF content on your server at all, not even while the user wants to view the model in the Viewer, then that would mean the following: the SVF content would need to be zipped up on the Design Automation server, sent to your server, then unzipped and each file would need to be added to OSS. Then you could implement an endpoint on your server that would redirect the Viewer to the requested file in the OSS bucket.

Move SVF content to OSS

There is an undocumented functionality in Design Automation that can help with both a) and b) scenarios. In case of a) it just helps you avoid the zipping and unzipping part. In case of b) it also helps you avoid sending the SVF files across the internet: from Forge to your server, and then back to the OSS bucket on Forge.

This functionality is the onDemand PUT, which enables you to specify an endpoint on your server that will be called once your WorkItem finished and it's time to upload the resulting files. Before uploading the files from Design Automation, this endpoint will be called with the names of the files in a specific folder (specified by the localName property of the onDemand parameter for the WorkItem) and you can reply with the URLs where each of them should be uploaded - see image at the top of this blog post.

This is what the implementation of that endpoint could look like in e.g. Node.js:

// The incoming message will be something like:
// {"permissions":"write","files":["textfile1.txt","textfile2.txt"]} 
// We have to return something like:
// { 
//   "textfile1.txt": "https://developer.api.autodesk.com/oss/v2/signedresources/a5ab33f3-8308-4458-8393-7c633f492c9c?region=US",
//   "textfile2.txt": "https://developer.api.autodesk.com/oss/v2/signedresources/31ad4c55-1b41-40e0-a8ca-ddb02dde8545?region=US"
// }
// The files will have names already URL encoded because of Windows restrictions on file names, 
// so no need to do encode() on them
router.post('/svf/callback', jsonParser, async function(req, res) {
    try {
        console.log("/svf/callback");
        console.log(req.body);

        let bucketKey = req.headers["viewables-bucket-key"];
        let urls = {};
        for (let fileNameEncoded of req.body.files) {
            let options = {
                uri: `https://developer.api.autodesk.com/oss/v2/buckets/${bucketKey}/objects/${fileNameEncoded}/signed?access=write`,
                method: "post",
                headers: {
                    "Content-Type": "application/json;charset=UTF-8",
                    "Authorization": req.headers["authorization"]
                },
                body: {
                    "minutesExpiration" : 45,
                    "singleUse" : true
                },
                json: true
            };
            
            let response = await requestPromise(options);
            urls[fileNameEncoded] = response.signedUrl;
        }

        console.log("Returning URLS: ", urls);
        res.json(urls);
    } catch (error) {
        console.log(error);
        res.json({ status: "failed", message: error.message });
    }
});

And here is the relevant part of the WorkItem options for setting up the onDemand callback:

"svfOutput": {
    "ondemand": true,
    "zip": false,
    "verb": "put",
    "localName": "SvfOutput",
    "url": process.env.FORGE_VIEWABLES_CALLBACK, // <server URL>/svf/callback
    "headers": {
        "Authorization": "Bearer " + credentials.access_token,
        "Content-type": "application/octet-stream",
        "viewables-bucket-key": bucketKey // where the viewables need to be uploaded
    }
},

SVF content is organized into folders and subfolders. However, inside OSS buckets that's not possible, so instead, you can make the path (including folder names and path delimiters) part of the file name:

SVF files in OSS bucket

Also, it seems that the onDemand PUT functionality expects a single level folder as well specified by its localName. So I had to create a function in he AppBundle code to flatten the folder where the SVF files were placed:

// We have to move the folder separation into the file name
// e.g. "myfolder"\"myassembly.iam" => "myfolder\myassembly.iam"
// On windows we might have to URL encode this, i.e.
// "myfolder%2Fmyassembly.iam"
public void flattenFolder(string folder, string rootFolder, string path)
{
  const string separator = "%2F";
  if (folder != rootFolder)
  {
    string[] filePaths = Directory.GetFiles(folder);
    foreach (string filePath in filePaths)
    {
      string fileName = path + Path.GetFileName(filePath);
      string filePathNew = Path.Combine(rootFolder, fileName);
      File.Move(filePath, filePathNew);
    }
  }

  string[] directoryPaths = Directory.GetDirectories(folder);
  foreach (string directoryPath in directoryPaths)
  {
    string directoryName = Path.GetFileName(directoryPath);
    // move its contents first
    flattenFolder(directoryPath, rootFolder, path + directoryName + separator);

    // Then delete it
    try
    {
      Directory.Delete(directoryPath);
    }
    catch (Exception ex)
    {
      LogTrace(ex.Message + ": " + directoryPath);
    }
  }
}

 

When accessing files with a forward slash in their names you need to use the URL encoded version of them (%2F), e.g. (see sections highlighted in red

{
  "bucketKey": "rgm0mo9jvssd2ybedk9mrtxqtwsa61y0_f18d1d62-c342-4193-9d03-00a9a9a87021",
  "objectKey": "output/1/result.svf",
  "objectId": "urn:adsk.objects:os.object:rgm0mo9jvssd2ybedk9mrtxqtwsa61y0_f18d1d62-c342-4193-9d03-00a9a9a87021/output%2F1%2Fresult.svf",
  "sha1": "2b61216e977f2570870ab01a30869b2973312bc4",
  "size": 1496,
  "location": "https://developer.api.autodesk.com/oss/v2/buckets/rgm0mo9jvssd2ybedk9mrtxqtwsa61y0_f18d1d62-c342-4193-9d03-00a9a9a87021/objects/output%2F1%2Fresult.svf"
}

When passing a bubble.json file to the Viewer directly (instead of passing the urn of the model) then the additional files will be requested with the forward slash not getting URL encoded, and so the Viewer will not be able to load them from the OSS bucket. Because of this, we will have to create an endpoint on our server to redirect the Viewer to the correct location of the file it's looking for. The implementation could look like this for Node.js:

router.get('/viewer_proxy/*', async function(req, res) {
    var id = decodeURIComponent(req.path)
    var tokenSession = new token(req.session);
    
    try {
        let bucketKey = tokenSession.getFolderPath();

        // when accessing objects from bucket the object path cannot contain '/' it needs to be URL encoded to '%2F'
        // that's what encodeURIComponent() achieves 
        // use redirect instead of downloading to server and passing it to client
        res.redirect(`https://developer.api.autodesk.com/oss/v2/buckets/${bucketKey}/objects/${encodeURIComponent(id)}`);
    } catch (error) {
        console.log(error);
        res.status(404).end();
    }
}

Then you can load the model in the Viewer like so:

Autodesk.Viewing.Document.load(
  "/viewer_proxy/bubble.json",
  onLoad, 
  onError
)

This is how the Viewer then can fetch the files from the OSS bucket:

Loading SVF files from OSS bucket

You can find a sample doing all the above here: https://forge-model-updater.herokuapp.com/

The source code is here: https://github.com/adamenagy/bucket-model-updater 

Note: the above solution is just one way of doing things. Moving the zip file containing all the files of the SVF to your server, unzipping it and serving it from there is probably easier and just as fast.

Related Article