Physical and environmental variations require the addition of safety margins to the clock frequency of digital systems, making it overly conservative. Aggressive, but reliable, dynamic clock frequency tuning mechanisms that achieve higher system performance, by adapting the clock rates beyond worst case limits, have been proposed earlier. Even though reliable over-clocking guarantees functional correctness, it leads to higher power consumption and overheating. As a consequence, reliable over-clocking without considering on-chip temperatures will bring down the lifetime reliability of the chip. In this paper, we analyze how reliable over-clocking impacts the on-chip temperature of microprocessors, and evaluate the effects of overheating, due to reliable dynamic over-clocking mechanisms, on the lifetime reliability of such systems. We, then, evaluate the effects of performing thermal throttling, a technique that clamps the on-chip temperature below a predefined value, on system performance and reliability. Our study shows that a reliably over-clocked system, along with dynamic thermal throttling, achieves around 25% performance improvement, while operating within 355 K.