Built a small experiment around conditional compute by reasoning type.
This repo (1) classifies prompts into 6 reasoning types (weighting/consensus/deduction/comparison/causal/lookup) via simple patterns, then (2) runs GPT-2 Small and compares neuron activation patterns by type (overlap + “type-specific” neurons). On a toy set (48 prompts) the classifier is ~92% accurate, and some type pairs show low overlap.
Has potential applications in query-aware routing / MoE-style gating, serving-cost reduction (skip irrelevant compute for “lookup-ish” prompts), prompt triage (send hard cases to stronger models/tools), and interpretability (what subnetworks light up for what reasoning demands).
This repo (1) classifies prompts into 6 reasoning types (weighting/consensus/deduction/comparison/causal/lookup) via simple patterns, then (2) runs GPT-2 Small and compares neuron activation patterns by type (overlap + “type-specific” neurons). On a toy set (48 prompts) the classifier is ~92% accurate, and some type pairs show low overlap.
Has potential applications in query-aware routing / MoE-style gating, serving-cost reduction (skip irrelevant compute for “lookup-ish” prompts), prompt triage (send hard cases to stronger models/tools), and interpretability (what subnetworks light up for what reasoning demands).