A Trump adviser proposed letting homebuyers use 401(k) funds for down payments, though Trump himself has expressed skepticism about the idea. While tapping a 401(k) could help buyers break into a tough housing market, it may come at the expense of long-term retirement savings. Experts say early withdrawals can significantly reduce retirement wealth over time, as you miss out on decades of compounding growth.