It may not be as much of a factor at bicycle speeds, but I've read that heavy helmets can dramatically increase whiplash in things like motorsport. The increased mass on your head means higher forces trying to rip it off. Obviously the other benefits of real helmets outweigh that issue, but I'd imagine a concrete helmet with very poor shock absorption and very high mass would not fair so well.