Sometimes, developers need to be able to access the metadata of their designs in Autodesk Forge without necessarily using the viewer. In this post we will take a look at some of the options the platform offers in this regard.
Option #1: HTTP endpoint
The Model Derivative service provides an endpoint for retrieving metadata of a specific design in form of a JSON: GET :urn/metadata/:guid/properties. The JSON responses are easy to parse, and the endpoint can also be configured to only return properties of a single object with specific ID, making the requests faster and more efficient.
However, before deciding to use this option in your application, consider the following characteristics of this endpoint:
- JSON responses may be too verbose for complex designs; if the response exceeds 20MB, an additional URL parameter
forcegetmust be used in order for the request to be completed - Compared to other approaches, the output is less granular; for example, measurement units are not stored separately but "baked" into property values
Option #2: Sqlite database
When processing your designs, the Model Derivative service stores all the property metadata in a sqlite file. You can find it in the derivative manifest (using the GET :urn/manifest endpoint) as an asset with mime type application/autodesk-db, and with role set to Autodesk.CloudPlatform.PropertyDatabase, for example:
{
"urn": "dXJuOmZvbw",
"derivatives": [
{
"children": [
// ...
{
"urn": "urn:adsk.viewing:fs.file:dXJuOmZvbw/output/Resource/model.sdb",
"role": "Autodesk.CloudPlatform.PropertyDatabase",
"mime": "application/autodesk-db",
"guid": "6fac95cb-af5d-3e4f-b943-8a7f55847ff1",
"type": "resource",
"status": "success"
}
// ...
]
}
]
}
To download this asset, simply use the GET :urn/manifest/:derivativeurn endpoint, passing in the URN of the property database (note: url-encoded, not base64-encoded). For the example above, the endpoint would look like this: https://developer.api.autodesk.com/modelderivative/v2/designdata/dXJuOmZvbw/manifest/urn%3Aadsk.viewing%3Afs.file%3AdXJuOmZvbw%2Foutput%2FResource%2Fmodel.sdb.
Now, when you peek inside the sqlite file, you'll notice a couple of tables with strange names like _objects_attr, _objects_eav, _objects_id, _objects_val, etc. These tables form a structure called entity-attribute-value that we will describe at the end of this blog post.
Option #3: Viewer-optimized database
Forge Viewer uses a different representation of the property database (mainly for performance reasons), stored in several *.json.gz files with similar naming as the tables in the sqlite database, for example, objects_attrs.json.gz, objects_avs.json.gz, objects_ids.json.gz, or objects_vals.json.gz. The relationships between the data in these compressed JSON files is explained in the next section.
Note that the .json.gz files are not part of the derivative manifest described earlier. Instead, they are part of the internal format used by the Forge Viewer called SVF. You can identify and download these assets with tools like forge-convert-utils (specifically, the SvfDownloader class). The same library also provides a simple PropDbReader class that can be used to parse the .json.gz files and make basic queries over their data.
Here is sample code, that downloads the 5 files into memory and queries dbID=1: https://github.com/petrbroz/forge-convert-utils/blob/develop/samples/remote-svf-props.js.
// Node.js example
// npm install svf-utils
const { getSvfDerivatives } = require('./shared.js');
const { SvfReader, TwoLeggedAuthenticationProvider } = require('..');
const { APS_CLIENT_ID, APS_CLIENT_SECRET } = process.env;
async function run(urn) {
const derivatives = await getSvfDerivatives(urn, APS_CLIENT_ID, APS_CLIENT_SECRET);
const authenticationProvider = new TwoLeggedAuthenticationProvider(APS_CLIENT_ID, APS_CLIENT_SECRET);
for (const derivative of derivatives) {
const reader = await SvfReader.FromDerivativeService(urn, derivative.guid, authenticationProvider);
const propdb = await reader.getPropertyDb();
const props = propdb.getProperties(1);
for (const name of Object.keys(props)) {
console.log(`${name}: ${props[name]}`);
}
console.log(`Children: ${propdb.getChildren(1).join(',')}`);
}
}
run(process.argv[2]);Entity-attribute-value structure
Whether you're looking at the sqlite database or at the .json.gz files, the structure is basically the same - a collection of entity-attribute-value triplets (EAVs) linking together objects and their properties in the following way:
- entity is an index into an "entity" table (
_objects_idorobjects_ids.json.gz) containing information such as:- object ID (also known as "dbId")
- external ID (ID persistent across versions of the same design; different for each type of design format)
- attribute is an index into an "attribute" table (
_objects_attrorobjects_attrs.json.gz) containing information such as:- property name
- property category
- property type and units,
- property flags
- value is an index to a table of property values (
_objects_valorobjects_vals.json.gz)
Here's a diagram of how the 5 json files (objects_ids.json.gz, objects_attrs.json.gz, objects_vals.json.gz, objects_avs.json.gz, objects_offs.json.gz), join together to form our properties meta-data:
And here is a diagram of the same data, but in SQLite tables:
With this information we can start slicing & dicing the sqlite database in any way we need. For example, to get the IDs of all objects containing the word "Concrete" in their "Material" property, we could use the following query:
SELECT _objects_id.id AS dbId, _objects_id.external_id AS externalId, _objects_attr.display_name AS propName, _objects_val.value AS propValue
FROM _objects_eav
INNER JOIN _objects_id ON _objects_eav.entity_id = _objects_id.id
INNER JOIN _objects_attr ON _objects_eav.attribute_id = _objects_attr.id
INNER JOIN _objects_val ON _objects_eav.value_id = _objects_val.id
WHERE propName = "Material" AND propValue LIKE "%Concrete%"