You can say that in portuguese, it's completely correct but it seems to be much less used than "junto ao muro" as in "next to the wall/adjacent to the wall". Even being correct in this use, "contra" is a word very charged with adversarial context. Maybe that's the source of the strangeness the thread OP feels about the use of "against".