To play devil's advocate here, why don't you think we do in fact know better? They're homeless as a result of their life choices and we're in a position where we could help them as a result of choices we made in our lives. It may not be their fault, but do you think in their current state they can make the most rational, logical choices? Would it be better for us to enable these people's bad habits/mental illness and let them destroy themselves further with alcohol and/or other drugs? If we are going to make the case that we shouldn't blame them for their state of being, and that they're victims, then we need to accept that a lot of them need help from themselves. If on the other hand you want to respect their free will and self-determination or whatever, then hey, they made their bed. I'm certainly not going to fund their lifestyles if they have agency and self-determination and this is what we get.
Real people can misidentify anyone as well, "colored" or not(using their terminology). Does it stop becoming racist once it becomes good enough? Or should we stop arresting any POC suspects because we might make a mistake?
I think we can make a lot better arguments against the use of facial recognition by the government and particularly law enforcement than racism.
So...can I open a Whites Only restaurant?