I agree that CryptoAPI is nasty, but if your key doesn't have the CRYPT_EXPORTABLE flag set when it was created then it can not be exported by CryptoAPI, and can only be accessed maliciously by breaking the CryptoAPI (by patching it in memory, which requires full admin access to a running system) or accessing the OS disk offline.
CryptoAPI does not hand over the key and delegate signing and encryption to the application. Instead, it provides APIs for signing and encryption and does not disclose the private key.
I think most linux distro's protection of certificate private keys is pretty weak (just storing a .pem file somewhere) and I can't say anything about OSX but on Windows you can prevent the private key from being disclosed even if the user wants to export it and enforce entering a password or PIN each time the certificate is used. Getting access to the physical hardware is about the only way to bypass the CryptoAPI, which is pretty far from 'any' client compromise.
OpenId, OAuth, et al are similar to Kerberos: Using a central authentication server to issue tokens that are trusted for authentication to a third party. OAuth seems a little bit weaker compared to Kerberos since there is no mutual authentication, instead we tend to rely on HTTPS for this.
CryptoAPI does not hand over the key and delegate signing and encryption to the application. Instead, it provides APIs for signing and encryption and does not disclose the private key.