26 Sep 2024

Specific Limit for Jobs of Data Connector API

One of the typical scenarios working with ACC integration is to fetch up-to-date data in real time or get the latest updated data instantly.  Some ACC modules have not exposed corresponding API or webhook. The workaround is to take advantage of Data Connector API if it supports extracting the data. By extracting all data in batch, the application can analyze the data and synchronize with other workflows. It will take time for Data Connector job completes the extraction to get close to real-time data, so some applications post request by scheduleInterval=ONE_TIME in a dense interval of one day. 

However, Data Connector has its mechanism to manage the queue processing. The system flow controls one company not to overtake the queue of one hour. Since July, 2024, all regions are deployed with the restriction: Accounts have a limit of 24 jobs a day and individual Users have a limit of 24 jobs a day.  Actually, The logic is not on "day" boundaries, but rather a rolling 24 hour window.. e.g. assume end users have had extraction jobs by ACC UI within the 24 hours,  the jobs created by the application for the same account will be also counted for the 24 times. Similarly, if one ACC user creates jobs by ACC UI or application, all jobs number are also counted. 

So, you will see 429 error when your application hit the limit.  You may be confused with rate limit page of Data Connector which indicates a higher limit (60/minute/app). That is a limit for HTTP request in general. In the limit of 24 times/day, it is about frequency of running jobs of Data Connector. In essence, this endpoint below will be affected because in most cases, the hitting comes from requesting jobs by scheduleInterval=ONE_TIME. The job starts at once. The system will detect the rate quickly and throw the error of 429.

https://aps.autodesk.com/en/docs/acc/v1/reference/http/data-connector-requests-POST/  

By that means, the ideal workflow is one request of ONE_TIME per one hour averagely if you want to have up-to-date data regularly in the 24 hours. It can happen with the jobs by other types of scheduleInterval (by day, by week etc), once the running jobs count hit 24 times: account -level or user-level.  

If you need more real-time data, especially latest updated data only, the corresponding module API/webhook will be the best choice. If such APIs have not been available, please tell us (aps.help@autodesk.com) to add your request in the wishlist.  

 

 

Related Article