Introduction
Design Check is a common task during project life cycle, and the APS APIs can be very helpful to unlock designs data to compare against predefined standards. Now, integrating AI in this workflow, this can be improved further.
That's what I'll cover in this blog post, exploring three approaches with their pros and cons:
- Using AI to help coding a fully deterministic approach
- Using a RAG pipeline to check against predefined standards
- Using Skill and RAG to check against predefined standards
Use the samples in this post as a starting point for building review assistance tools, not autonomous compliance gates. The intended workflow is: AI surfaces potential issues and relevant clauses → a qualified reviewer confirms or overrides each finding → the reviewer signs off. The AI simply accelerates the process.
Prerequisites
The sample shared in this blog focus on checking a road dwg design stored in Forma Data Management.
It uses the data extracted by the Model Derivative API to compare the road curves radii against the standards defined as JSON, PDF, or an agent skill.
You'll need at least a sample file and a PDF with the standards to be able to run this sample.
You can find some samples here.
Approach 1: Deterministic (JSON Standards)
In this first approach, we need to specify a JSON structure for our check and the entire processing happens in the client. No indexing, no natural language, but any change in the standards require an update in the code.
In this context, AI is used to help coding the validation logic.
You can leverage the skill from this blog.
Pros
- Fully reproducible: Same inputs always produce the same output
- Auditable: The result traces back to an exact rule in the standards file
- Version-controllable: Standards live in source control alongside the code
Cons
- Manual conversion required: Official documents must be translated into structured data by hand
- No semantic understanding: Cannot handle conditional or nuanced requirements
- Maintenance burden: Must be updated whenever standards change
- Limited scope: Only checks what is explicitly defined. implicit requirements are missed
When to use
When you have restrictions to probabilistic approaches and the standards are stable and well-defined.
Approach 2: Probabilistic RAG
In this approach we start by uploading a PDF containing the procedures to be indexed.
Once indexed, we can leverage its content for the subsequent checks.
Pros
- Works from official documents: No manual conversion needed
- Semantic retrieval: Finds relevant standards even with varied phrasing
- Scales to large libraries: Multiple documents can be searched in a single query
- Lower setup cost: Index once, reuse across many queries
Cons
- Non-deterministic: The same query may return different results across runs
- Requires human verification: Citations may be incomplete, misquoted, or incorrect
- Service dependency: Design data is sent to an external provider
- Indexing latency: Processing new documents takes time
- Chunking sensitivity: Retrieval quality depends heavily on how source documents are split — standards tables and cross-references are often corrupted by chunk boundaries, causing thresholds to be missed or misread.
- Black-box reasoning: Cannot fully trace why a specific threshold was or was not applied
When to use
When the standards library is difficult to automate or changes frequently, and you need a tool to help reviewers navigate large documents faster. Treat the output as a first-pass triage.
Approach 3: Skills + RAG Hybrid
Pros
- Structured output: Enforced schema makes results easier to parse by downstream tools
- Decoupled: Update the standards index or the evaluation rubric independently
- Consistent evaluation: Rubric is defined once and reused across checks
Cons
- Non-deterministic: The same query may return different results across runs
- Requires human verification: Same hallucination risk as Approach 2
- Higher latency and token cost: Two model calls instead of one
- Service dependency: Design data is sent to an external provider
- Rubric overhead: Evaluation logic must be maintained and versioned separately
- Black-box reasoning: Cannot fully trace why a specific threshold was or was not applied
When to use
When you need a more structured compliance report, or when the evaluation rubric needs to be maintained and versioned separately from the standards content. Also useful when the same rubric will be applied across multiple projects or standards libraries.
Conclusion
With this sample you can find three different approaches to accomplish the same goal, with different flavors of AI usage.
Use those as base samples to explore and find your ideal workflow.
This can also be applied for different industries and use cases, and if the data you require isn't available with the Model Derivative API, you can also leverage Automation and the Data Models.