The difference is that most/all sugars are not inherently harmful, and we believe that they can be consumed safely at reasonable doses. A bit of glucose or fructose won't kill you, and is a (small) natural component of your diet. So banning sugar would be rather hard, compared to complex toxic chemicals which can't be processed by our bodies, decay slowly, and where we really have fairly limited reason to put in products.