The Case for Causal AI(ssir.org)
ssir.org
The Case for Causal AI
https://ssir.org/articles/entry/the_case_for_causal_ai
20 comments
The theory of causal inference tells you how to do this -- what is the nature of the relationship between correlation and causation? When can we draw conclusions from natural experiments? There is actual rigorous theory that has very clear answers to this. The conceptual tools are powerful.
Pearl's "Book of Why" may be a good starting point as a popularization, although sometimes Pearl is not so easy to read. And Pearl presents his own perspective on causal inference only -- there are other schools and techniques, although there are generally equivalences between them.
It is definitely possible to understand the core ideas as a layman, if you have a mathematical grasp of basic probability. If you want to go a little further, understanding linear regression helps too.
The problem is that actually applying the theory to draw causal conclusions on real-world data seems to be a very subtle and difficult process and even experts who specialize their entire careers in causal inference frequently make mistakes and disagree with each other. So I think a lot of humility is warranted.
Pearl's "Book of Why" may be a good starting point as a popularization, although sometimes Pearl is not so easy to read. And Pearl presents his own perspective on causal inference only -- there are other schools and techniques, although there are generally equivalences between them.
It is definitely possible to understand the core ideas as a layman, if you have a mathematical grasp of basic probability. If you want to go a little further, understanding linear regression helps too.
The problem is that actually applying the theory to draw causal conclusions on real-world data seems to be a very subtle and difficult process and even experts who specialize their entire careers in causal inference frequently make mistakes and disagree with each other. So I think a lot of humility is warranted.
It’s interesting to study, but it doesn’t magically make hard problems tractable. It seems to make certain common mistakes less likely. When you do statistical analysis or machine learning to model p(x,y) or its friend p(y|x) it’s possible that you mistake correlations for causation (because there is no way to express directionality of dependency). If I have data about the measurement on a barometer and atmospheric pressure then I may learn a model that predicts higher pressure when the barometer reading is high. But if I manually force the barometer reading higher that doesn’t increase the atmospheric pressure. Causality provides a mathematical framework to express that idea.
The problem I have with causality is how to find accurate causal diagrams from unstructured observational data. We _could_ guess and check every possible causal relationship, but that’s at least exponentially hard—in which case causality is useless.
People seem to have reasonably good capabilities for generating candidate causal hypotheses from observational data (basically all of modern science), but most of the material I’ve found on causality focuses on the theoretical benefits it provides rather than on practical applications at scale. (I don’t care if we can automate finding the causal graph for barometer & air pressure; how do I find a causal graph for classifying fake/real news from plain text data?)
The problem I have with causality is how to find accurate causal diagrams from unstructured observational data. We _could_ guess and check every possible causal relationship, but that’s at least exponentially hard—in which case causality is useless.
People seem to have reasonably good capabilities for generating candidate causal hypotheses from observational data (basically all of modern science), but most of the material I’ve found on causality focuses on the theoretical benefits it provides rather than on practical applications at scale. (I don’t care if we can automate finding the causal graph for barometer & air pressure; how do I find a causal graph for classifying fake/real news from plain text data?)
I guess it's tricky because the real world is full of feedback loops. If you want a causal model for fake news then your model needs to include some representation of incentives for ad revenue and clickbait. How does the causal inference framework handle feedback loops?
I believe DAG-based causal inference isn’t able to handle feedback loops (acyclic) or nonlinearity (linear). Nonlinearities include stuff like deadbands and delays.
Control theory models handle these things just fine. But control models are hard to apply to sociological/epidemiological domains, where causal inference dominates.
From what I gather, causal inference is useful for designing studies. I’m not sure if they’re used for prediction — would appreciate if someone in the know could chime in.
Control theory models handle these things just fine. But control models are hard to apply to sociological/epidemiological domains, where causal inference dominates.
From what I gather, causal inference is useful for designing studies. I’m not sure if they’re used for prediction — would appreciate if someone in the know could chime in.
> DAG-based causal inference isn’t able to handle feedback loops (acyclic) or nonlinearity (linear)
I don't think this should be true, and if it is, then "causal inference" should be qualified to refer only to a specific modeling framework. As a counter example, it's possible to formulate a nonlinear differential equation model of Covid spread and infer the parameters to construct a plausible, causal, generative model.
I don't think this should be true, and if it is, then "causal inference" should be qualified to refer only to a specific modeling framework. As a counter example, it's possible to formulate a nonlinear differential equation model of Covid spread and infer the parameters to construct a plausible, causal, generative model.
Interesting. I wonder if someone has tried to combine the two. I guess modern deep reinforcement learning is one such combination because it combines feedback (reinforcement) and probabilistic descriptions but maybe there are other interesting combinations of probability, causality, and feedback.
The sole way would be to have global knowledge of every datapoint; omniscience. It’s impossible.
So, the solution is to get comfortable with the lack of rigor. What can be known in a system without rigor? That’s the question to make rigorous, I think.
So, the solution is to get comfortable with the lack of rigor. What can be known in a system without rigor? That’s the question to make rigorous, I think.
> What can be known in a system without rigor? That’s the question to make rigorous, I think.
Who is working on making that rigorous?
Who is working on making that rigorous?
These folks used a synthetic control to do the exact quasi experiment you described: https://www.iza.org/publications/dp/13319/face-masks-conside...
The article outlines two approaches to causal AI
> There are two approaches to causal AI that are based on long-known principles: the potential outcomes framework and causal graph models. Both approaches make it possible to test the effects of a potential intervention using real-world data. What makes them AI are the powerful underlying algorithms used to reveal the causal patterns in large data sets. But they differ in the number of potential causes that they can test for.
Does anyone have references and tutorials for either approach?
> There are two approaches to causal AI that are based on long-known principles: the potential outcomes framework and causal graph models. Both approaches make it possible to test the effects of a potential intervention using real-world data. What makes them AI are the powerful underlying algorithms used to reveal the causal patterns in large data sets. But they differ in the number of potential causes that they can test for.
Does anyone have references and tutorials for either approach?
I suggest a different approach: we use Plan Analysis (Caspar/Grawe). This is a neuropsychotherapy instrument that can be used to actually map behavioural causality. These models here assume that you know causes for behaviour, and you only have to map them. BS. The biggest problem with an AI designed as such is that it doesn't have CONTEXT. Behaviour is never 'if A then B'.
Imbens and Rubin’s book “Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction” is an excellent reference for the Potential Outcomes approach. There are also several summaries online done by Rubin which do a great job of explaining the core concepts and how they apply in concrete examples. Rubin’s class in grad school was a large factor in my decision to focus my PhD on Causal Inference, and that book is one I return to frequently.
Thanks.
Someone posted this flowchart in a previous HN thread on Causal Inference frameworks: https://www.bradyneal.com/which-causal-inference-book (I picked up Counterfactuals and Causal Inference and Elements of Causal Inference, and would recommend both).
Thanks.
A recent thing I was learning/thinking about using neural networks to do program synthesis (to try to match some kind of fine-grained data), then run the program and see the actual output. Something similar could probably work for other types of symbolic computation. Let the neural network give you an intuition in a format you can actually check. So its basically there to speed up the concrete search/analysis.
How does having a program / algorithm and checking it on various input values help with understanding causality?
Neural networks are largely black boxes whereas a computer program or other symbolic configuration is a compositional interpretable description.
The program is the explanation of the output, i.e. the program causes the output.
The program is the explanation of the output, i.e. the program causes the output.
But why is that a causal explanation? If I can write down a simulation of planetary motion then that doesn't necessarily explain the causal mechanism behind why the planets actually move. In fact, there are simulations for planetary motion and none of them are causal explanations because they don't actually move the planets.
For example, can I answer questions like what are the main causes that are moving corona stats?
With intervention I suppose you can conduct an experiment where you (randomly) pick cities and make half of them wear masks and half of them not. But this is of course unethical! And some stuff you want to know the effects of (e.g., how did gatherings at protests affect the infection rate) are one time events that can't be replicated again.
So all we have left are lots of natural experiments. Different countries/states/communities are handling the situation differently with a wide range of outcomes. No two communities are directly comparable since they differ along many other dimensions other than corona policy. But as a human I am still drawing plenty of conclusions on what caused what and to what degree. So it seems like a solvable problem. How do I make it rigorous?