You can get by with just Swedish if you live in a city or town where a large percentage of the population are native Swedish speakers. In Ekenäs, for example, around 80% of the population have Swedish as their native language.
Along the coast of southern and western Finland there are many bilingual Swedish/Finnish municipalities where most of the native Swedish speaking population lives. And on the island of Åland the overwhelming majority of the population are native Swedish speakers.
On the other hand in Central and Eastern Finland you aren’t going to get by with just Swedish.