You are mistaken. TPMs these days are as tamper resistant as it gets, and a manufacturer would have severe difficulty selling one without an independent security certification proving that it actually is. Which means a state-of-the-art security lab with highly skilled experts and millions worth of specialized equipment attempts to break the chip for an extended amount of time, trying all sorts of known attacks, utilizing extensive access to information on chip internals that a real-life attacker normally would not have, _and in effect fails to break it nonetheless_.
Google for 'Tarnovsky attack' and spend an evening watching thrilling videos to see to which lengths a top-notch adversary needed to go to break into these chips even in 2010-13. As for the chances in 2018, draw your own conclusions.