When implementing Appian AI Skills for Document Extraction within an automated workflow, how should a system architect design the process model to handle probabilistic uncertainty (low-confidence predictions), and how does the platform's 'Human-in-the-Loop' (HITL) architecture facilitate continuous model retraining?
When implementing Appian AI Skills for Document Extraction within an automated workflow, how should a system architect design the process model to handle probabilistic uncertainty (low-confidence predictions), and how does the platform's 'Human-in-the-Loop' (HITL) architecture facilitate continuous model retraining?
Answer
AI Document Extraction uses machine learning to find and pull information out of documents. Its predictions are sometimes wrong, so Appian returns a confidence score for each extracted field. Putting unvalidated predictions into a business process can bring incorrect or incomplete data into your records. Confidence-based routing with a human review step prevents that.
Implementation in an Appian process model
1. Extract from Document smart service
Use the Extract from Document smart service to pass a document to a trained Document Extraction AI Skill. It returns three outputs:
- Extracted Data: a map of the extracted field values.
- Confidence Scores: a map of confidence scores for each field prediction.
- Doc Extraction Id: an identifier used for document reconciliation.
2. Set the confidence threshold
The smart service has a built-in Confidence Threshold input that takes an integer from 0 to 100. If you set it to 85, any text field scoring below 85 comes back as null in Extracted Data. The threshold only applies to text entities, so other data types need their own validation.
3. Evaluate results with an XOR gateway
After extraction, an XOR gateway decides whether the document can be processed automatically or needs a manual check. Its conditions should confirm that every mandatory field has a valid value and meets your confidence requirements.
4. High confidence: straight-through processing
If all required fields pass the confidence and business-validation checks, the process takes the automated path. It maps the extracted data to the appropriate record type and saves it with the Write Records smart service, with no human involved.
5. Low confidence: human-in-the-loop validation
If a critical field has a low score, comes back null, or fails business validation, the process sends the document to manual review. The Reconcile Doc Extraction smart service lets an assigned user check and correct the extracted values. After the reviewer submits, you validate the Reconciled Data output and use it to update the records. If the reviewer marks the document invalid, the process can send it down a separate exception-handling path.
Example: invoice processing
Take an invoice with the confidence threshold set at 85%.
Extracted field | Confidence | Result |
|---|---|---|
Invoice Number | 96% | Pass |
Vendor Name | 92% | Pass |
Invoice Date | 89% | Pass |
Total Amount | 72% | Review |
Three fields clear the threshold, but Total Amount does not. It is a critical field, so the document goes to manual verification instead of updating the invoice record automatically.
Process flow
- Extract from Document returns the extracted values and their confidence scores.
- An XOR gateway checks confidence, required fields, and business rules.
- No => If everything passes, Write Records saves the validated data automatically.
- Yes => If anything fails, Reconcile Doc Extraction lets a human review and correct the fields. The corrected data is then validated and saved with Write Records.
Official Appian References
- Extract from Document Smart Service
- Reconcile Doc Extraction Smart Service
- Appian Gateways
- Write Records Smart Service