Create an ETL Process for TradeTapp
This tutorial demonstrates how to perform ongoing data retrieval for an Extract, Transform, and Load (ETL) process, so you can store all of your TradeTapp data in your own data warehouse. The steps include retrieving qualification data of sub-contractors your company works with and periodically checking for updates of this data.
You can learn more about the TradeTapp data model in the TradeTapp Field Guide.
Before You Begin
- Register an app (select either the Traditional Web App or the Desktop, Mobile, Single-Page App application type).
- Make sure to select the BuildingConnected API via the API Access dropdown after registering your app. The TradeTapp API is housed under the BuildingConnected API heading.
- Acquire a 3-legged OAuth token with
data:read
scope.
Important: TradeTapp API v2 will return data points for instances where an invited subcontractor has made at least one historical submission or the data was imported. Data will not be returned for the following circumstances:
- Invitations where the subcontractor has been invited but has not yet made a submission
- Project approval data (This function is currently not supported by APIs)
Pagination
Each endpoint returns paginated results. As long as there are additional pages of results available, the response will include a nextURL
field indicating the URL to call for the next page.
If the nextURL
field is present in the pagination
object that is returned (e.g. qualifications.pagination.nextUrl
), you can call the provided URL to page through the remaining items before continuing your extraction.
Keep in mind that some objects have sub-resource pagination.
This might be relevant if you have a lot of custom questions (e.g. qualification.customQuestions.pagination.nextUrl
) in your questionnaire or the vendor has a lot of office locations (>50 questions or locations).
Subsequent Extractions
Your qualification response payloads will always include a updatedAt
field. The API enables you to query each endpoint using this timestamp with the filter[updatedAt]
query parameter so that you get only data that has been changed or added since your previous extraction.
If an object is new but has not been updated yet, the updatedAt
field will be equal to the submittedAt
field (i.e. it will not be null
) so that you can be sure you’re capturing all changes since your previous extraction.
The following steps provide examples of how to retrieve each data object type, along with its sample response.
Step 1: Retrieve a list of Qualifications
A qualification represents a solicitation of an individual company to qualify. There is one qualification object for each company that has submitted the questionnaire after having been invited to qualify.
Qualifications for a sub-contractor can be identified by the qualification.id
. This is seen in the response as the id
field which identifies the unique company id in the Builders network.
To get your company’s qualifications, call GET qualifications.
Request
curl 'https://developer.api.autodesk.com/construction/tradetapp/v2/qualifications' \
-H 'Authorization: Bearer <TOKEN_VALUE>'
Response
{
"pagination": {
"cursorState": "cD0xNDcxOA==",
"limit": 20,
"totalResults": 50,
"nextUrl": "/construction/tradetapp/v2/qualifications?cursorState=cD0xNDcxOA%3D%3D",
"previousUrl": null
},
"results": [
{
"id": "6307acdf1ea3270060e88e00",
"uniqueId": "U1010",
"questionnaireVersion": "Primary Qualification",
"qualification": {
"status": "QUALIFIED",
"internalSingleLimit": 50000,
"internalTotalLimit": 125000,
"dateExpires": "2024-10-03T00:00:00.000Z",
"summary": "Great profile. Qualified.",
"denialReason": null,
"preExpirationStatus": null,
},
"applicationStatus": "SUBMITTED",
"applicationDetail": "NEW",
"companyName": "New York Interiors ",
"yearFounded": 2004,
"stateFounded": "California",
"taxIdType": "US_FEIN",
"federalTaxId": "411901185",
"submissionCurrency": "USD",
"numberOfHomeOfficeEmployees": 586,
"numberOfFieldSupervisoryEmployees": 302,
"hasProfessionalLicenses": true,
"hasUnionAffiliations": true,
"hasParentCompany": true,
"parentCompanyName": "New York Contractors",
"currentEstimatedBacklog": 10000,
"insuranceBrokerContactName": "Matthiew Serot",
"insuranceBrokerCompanyName": "Lemke Group",
"insuranceBrokerContactPhone": "650000000",
"insuranceBrokerContactEmail": "jallawy1e4@lekmegroup.com",
"suretyBrokerAgencyCompanyName": "Raynor, Weber and Bergstrom",
"suretyBrokerContactName": "Cindy Lehrer",
"suretyBrokerContactPhone": "415000000",
"suretyCompanyName": "Marvin-Hagenes",
"suretySingleProjectBondingCapacity": 100000,
"suretyAggregateBondingCapacity": 500000,
"bankName": "Steuber, Leffler and Jerde",
"bankLineOfCreditTotal": 963320,
"bankLineOfCreditOutstanding": 7640438,
"bankContactName": "Jeanelle Allawy",
"bankContactPhone": "2124724015",
"bankContactEmail": "jallawy1e4@slj.com",
"submittedAt": "2017-02-20T00:00:00Z",
"updatedAt": "2023-06-26T00:00:00Z",
"csiCodesForWorkPerformed": [
{
"primaryCode": "3",
"secondaryCode": "37",
"tertiaryCode": "16",
"codeDescription": "Pumped Concrete"
}
],
"regionsServed": [
{
"region": "Florida"
}
],
"marketsServed": [
{
"market": "Hospitality"
}
],
"completedReferences": [
{
"projectName": "A Project Name",
"location": "8924 Sachs Court, Mattawa, ON, V5M",
"yearCompleted": 2021,
"value": 5004615,
"scope": "Plexiglass",
"referenceContactCompany": "Innojam",
"referenceContactName": "Nike Longworth",
"referenceContactPhone": "3959641188",
"referenceContactEmail": "nlongworth1e4@innojam.biz",
"isLargestProject": false
}
],
"companyContacts": [
{
"positionTitle": "Director of Operations",
"contactName": "Nat Girke",
"contactPhone": "1656366322",
"contactEmail": "ngirke1e4@newyorkinteriors.com",
"contactFax": "1656366322",
"contactType": "BOTH"
}
],
"insuranceCoverages": [
{
"insuranceType": "WORKERS_COMPENSATION",
"carrier": "Leafy Pea Insurance Company",
"perOccurrenceLimit": 100000,
"aggregateLimit": 1000000,
"policyExpirationDate": "2024-05-09",
"isNotApplicable": false
}
],
"experienceModificationRatings": [
{
"year": 2022,
"experienceModificationRating": 0.83
}
],
"osha300Results": [
{
"year": 2022,
"totalNumberDeathsBoxG": 0,
"totalNumberCasesDaysAwayBoxH": 1,
"totalNumberCasesRestrictionTransferBoxI": 0,
"otherRecordableCasesBoxJ": 0,
"totalHoursWorked": 16216
}
],
"certificates": [
{
"id": "6476478d249a0533bab39072",
"type": {
"id": "63653068d6df3afebe7a46fc",
"name": "Disadvantaged Business Enterprise (DBE)"
},
"agency": {
"id": "63653068d6df3afebe7a46fa",
"name": "United States Department of Transportation",
"website": "https://www.transportation.gov/civil-rights/disadvantaged-business-enterprise"
},
"number": "123456",
"expiresAt": "2023-07-03T17:00:00.000Z",
"fileName": "TEST PDF (1) (1).pdf"
}
],
"unions": [
{
"unionName": "Safety Officer Union",
"unionNumber": "7075"
}
],
"annualVolumeRevenues": [
{
"year": 2020,
"estimatedVolumeRevenue": 51546617
}
],
"professionalLicenses": [
{
"licenseType": "Electrician",
"licenseNumber": "D8665258258930",
"licenseState": "New York",
"issuingAgency": "New York Registry of Contractors",
"additionalDescription": "LNO123"
}
],
"companyOfficeAddresses": {
"pagination": {
"cursorState": null,
"limit": 50,
"totalResults": 1,
"nextUrl": null,
"previousUrl": null
},
"results": [
{
"isMainOffice": true,
"address1": "63 Sheridan Terrace",
"address2": null,
"city": "Jamaica",
"state": "New York",
"zipCode": "11447"
}
]
},
"customQuestions": {
"pagination": {
"cursorState": null,
"limit": 50,
"totalResults": 3,
"nextUrl": null,
"previousUrl": null
},
"results": [
{
"question": "How do you feel about your fincancial health?",
"section": "Financial",
"textResponse": "Good",
"numberResponse": null,
"booleanResponse": null
},
{
"question": "How would you rate your fincancial health on a scale of 1 to 10, 10 being the best?",
"section": "Financial",
"textResponse": null,
"numberResponse": 7,
"booleanResponse": null
},
{
"question": "Have you submitted your most recent financials?",
"section": "Financial",
"textResponse": null,
"numberResponse": null,
"booleanResponse": true
}
]
}
},
...
]
}
Step 2: Periodically Check for Updates
On a periodic basis, perform the preceding steps using the query parameter filter filter[updatedAt]
to check for updates since the previous time of data collection.
For example, to check for updates on or after 26th Jan. 2023 use the filter parameter filter[updatedAt]=2024-01-26T00:00:00.000Z..
of GET qualifications.