There's nothing wrong with this. Public/Private keys are for sharing secrets and proving identity.
Assuming you are using a sane cipher and mode, your risks boil down to any of a myriad of methods of stealing/bruteforcing your key or grabbing the unencrypted file.
Assuming you are using a sane cipher and mode, your risks boil down to any of a myriad of methods of stealing/bruteforcing your key or grabbing the unencrypted file.