Do you want to monitor what an Al agent is about to do before it acts?
In our new paper, Beyond the Black Box: Interpretability of Agentic Al Tool Use, we explore how mechanistic interpretability can help surface signals around tool-use decisions, missed calls, unnecessary calls, and higher-risk actions.
We want to do both. In finance, highly regulated industry, understanding how models work is critical. In addition, mech interp will allow us to understand which current or new architectures could work better for financial applications.
Thank you for reading. One of the main reasons we've written the paper is to help with model validation of LLM usage in our highly regulated industry. We are also engaging with regulators.
The industry at the moment is mostly using closed sourced vendor models that are very hard to validate or interpret. We are pushing to move onto models, with open source weights and where we can apply our interpretability methods.
Current validation approaches are still very behavioral in nature and we want move it into mechanistic interpretation world.
Thank you. Agreed, we are exploring different ways to apply these interpretability methods to a wide range of transformer based methods, not just decoder based generative applications.
Paper introduces AI explainability methods, mechanistic interpretation, and novel Finance-specific use cases. Using Sparse Autoencoders, we zoom into LLM internals and highlight Finance-related features. We provide examples of using interpretability methods to enhance sentiment scoring, detect model bias, and improve trading applications.
Our paper introduces AI explainability methods, mechanistic interpretation, and novel Finance-specific use cases. Using Sparse Autoencoders, we zoom into LLM internals and highlight Finance-related features. We provide examples of using interpretability methods to enhance sentiment scoring, detect model bias, and improve trading applications.
My younger daughter, who is pretty good at making games in Scratch is not that interested in jumping into text/code based programming. I do think Scratch makes things a lot easier and text based programming is not thar appealing to kids. I will try to start her with Pygame but even that might make it seem very arcane and not very visual.
Terrence Tao has a very healthy view of what AI can and cannot achieve shorter term. It is refreshing to hear more grounded views from a top mathematician who is clearly well versed in the topic.
In our new paper, Beyond the Black Box: Interpretability of Agentic Al Tool Use, we explore how mechanistic interpretability can help surface signals around tool-use decisions, missed calls, unnecessary calls, and higher-risk actions.