- Dependable AI analysis requires clear sources, accepted guidelines, and traceable inputs.
- Lacking proof and unfinished work should stay seen.
- Corrections should change into procedures to enhance future analysis.
We constructed our AI-enabled analysis course of for a reasonably bizarre cause: consistency. Working it taught us that consistency was solely the start.
The tougher activity was preserving all of the small distinctions that make a bit of analysis reliable sufficient to examine.
Learn one firm in March and one other in September and it’s surprisingly simple to finish up with two impressions moderately than two comparisons. The questions drift. The emphasis modifications. What one firm taught you impacts what you discover within the subsequent.
So, we formalized the analysis course of. Proof from filings would stay separate from calculations. Calculations would comply with outlined guidelines. Funding judgement would stay identifiable as judgement. Each firm would cross via the identical questions in the identical order.
That was the design. Then we began operating it.
The expertise remains to be modest: one quarterly cycle throughout roughly twenty corporations. However what stunned me was that consistency was not the arduous half. Separating proof, calculation and judgement was obligatory, however not ample.
The method additionally needed to protect issues we had barely thought of: which supply ruled, whether or not an announcement was provisional, whether or not an motion had truly occurred, who owned a bit of metadata, and whether or not unfinished work nonetheless regarded unfinished.
A lot of the failures had been mundane. That made them extra helpful.
