AI-Assisted Internal Workflow Tool

Plenty of internal tools exist because somebody answered the same question several times a week by hand. The temptation is to automate that with a model. The better question is whether the judgement involved can be written down at all.

Here it partly could. The repeated work was triaging inbound enquiries: routing them to a team, spotting the ones that matched a known pattern, and drafting a first response. A rule set handles routing and pattern-matching reliably, so that part is ordinary code. Only the drafting step needed a language model, and it was kept to that. A person approves every outgoing message, and the approval is recorded.

The consequential design decision was what happens when the system is unsure. It does not guess. Below a confidence threshold the enquiry goes to a person with no suggestion attached, because a plausible wrong answer costs more to catch later than a blank field does now.

Verification was done against a set of real historical enquiries where the outcome was already known. The tool was measured on whether it reached the same routing decision, not on whether the drafts read well.

Handover included that test set, so the routing can be re-measured after any change rather than trusted.