This could partially be blamed on education being limited to few people as a direct side effect of caste system -> But the data seems to indicate otherwise at least from surveys around 1800.
From Dharampal "Beautiful Tree"
> It is, however, the Madras Presidency and Bengal-Bihar
data which presents a kind of revelation. The data reveals the
background of the teachers and the taught. It presents a picture
which is in sharp contrast to the various scholarly pronounce-
ments of the past 100 years or more, in which it had been
assumed that education of any sort in India, till very recent
decades, was mostly limited to the twice-born amongst the
Hindoos, and amongst the Muslims to those from the ruling
elite. The actual situation which is revealed was different, if not
quite contrary, for at least amongst the Hindoos, in the districts
of the Madras Presidency (and dramatically so in the Tamil-
speaking areas) as well as the two districts of Bihar. It was the
groups termed Soodras, and the castes considered below them
who predominated in the thousands of the then still-existing
schools in practically each of these areas.
> In most areas, the Brahmin scholars formed a very small
proportion of those studying in schools. Higher learning,
however, being more in the nature of professional specialisation,
seems in the main to have been limited to the Brahmins. This
was especially true regarding the disciplines of Theology,
Metaphysics, Ethics, and to a large extent of the study of Law.
But the disciplines of Astronomy and Medical Science seem to
have been studied by scholars from a variety of backgrounds
and castes. This is very evident from the Malabar data: out of
808 studying Astronomy, only 78 were Brahmins; and of the 194
studying Medicine, only 31 were Brahmins. Incidentally, in
Rajahmundry, five of the scholars in the institution of higher
learning were Soodras. According to other Madras Presidency
surveys, of those practising Medicine and Surgery, it was found
that such persons belonged to a variety of castes. Amongst them,
the barbers, according to British medical men, were the best in
Surgery.
From Dharampal "Beautiful Tree"
> It is, however, the Madras Presidency and Bengal-Bihar data which presents a kind of revelation. The data reveals the background of the teachers and the taught. It presents a picture which is in sharp contrast to the various scholarly pronounce- ments of the past 100 years or more, in which it had been assumed that education of any sort in India, till very recent decades, was mostly limited to the twice-born amongst the Hindoos, and amongst the Muslims to those from the ruling elite. The actual situation which is revealed was different, if not quite contrary, for at least amongst the Hindoos, in the districts of the Madras Presidency (and dramatically so in the Tamil- speaking areas) as well as the two districts of Bihar. It was the groups termed Soodras, and the castes considered below them who predominated in the thousands of the then still-existing schools in practically each of these areas.
> In most areas, the Brahmin scholars formed a very small proportion of those studying in schools. Higher learning, however, being more in the nature of professional specialisation, seems in the main to have been limited to the Brahmins. This was especially true regarding the disciplines of Theology, Metaphysics, Ethics, and to a large extent of the study of Law. But the disciplines of Astronomy and Medical Science seem to have been studied by scholars from a variety of backgrounds and castes. This is very evident from the Malabar data: out of 808 studying Astronomy, only 78 were Brahmins; and of the 194 studying Medicine, only 31 were Brahmins. Incidentally, in Rajahmundry, five of the scholars in the institution of higher learning were Soodras. According to other Madras Presidency surveys, of those practising Medicine and Surgery, it was found that such persons belonged to a variety of castes. Amongst them, the barbers, according to British medical men, were the best in Surgery.