Retirement without a Mortgage

You are retiring in a few months and have a $150,000 balance on your primary home. Do you use your retirement funds to pay off the balance of your home, or do you keep paying the mortgage payment during retirement? The advice that you get seems to depend on your advisor’s compensation and his or her understanding of personal money management.

I often talk about the legal standards of suitability and fiduciary within the financial services industry. Very often our new clients come to us from brokerage firms where they feel neglected by the advisor that is always seeking the next sale rather than maintaining relationships with his or her clients. Recently we have been competing for new business with retirees that are entering into their first client-advisor relationship. While the investment strategy of our firm is very different than the mainstream broker, what our prospects have noticed more is that our advice on what to do with their remaining mortgage balance is very different from what a mainstream broker is saying.

Example

Let’s say you have a mortgage balance of $150,000 at an interest rate of 3.25% and a monthly payment of $1,100 per month. Any financial advisor can pull out a calculator and show you that if you keep your $150,000 invested with him, you could make on average 7% per year, netting you 3.5% after accounting for mortgage interest before you calculate your mortgage deduction on your Federal tax return. On paper this sounds good; you make more money and the advisor collects more commission. However, there are some flaws in this thinking.

Cash Flow and Taxes

One of the biggest factors in making the decision to retire is how much cash flow do you need to live comfortably during retirement. In our example a $1,100 per month payment could be significant in a middle class retiring household. The $150,000 left in investments actually would have to produce a return of 8.8% for 15 years to maintain principal until the home was paid off. Instead, by paying off the home, the retiree’s cash flow need would decrease by $1,100.

In the paragraph above a rate of return of 8.8% over a 15 year period is very possible if invested in a higher risk tolerance for such a return. However, it is important to understand that that an 8.8% or any positive rate of return is never guaranteed when investing in index funds, mutual funds or individual securities. If funds are used to pay off the home, you are guaranteed a 3.25% rate of return in our example. How? By not having the mortgage you are saving having to pay out 3.25% in mortgage interest and possibly other fees such as PMI.

I often hear of advisors telling their clients not to pay off their home because of losing their deduction for their mortgage interest. Let’s look at the math behind this advice. If you have a home with a monthly payment of $1,100, and the interest portion is $400 per month, you have paid around $4,800 in interest that year, which creates a tax deduction. If your home was paid for, you would lose this deduction.

If you don’t have a $4,800 tax deduction and you’re in a 18% tax bracket, you will have to pay $864 in taxes on that $4,800. According to the math, we should send $4,800 in interest to the bank so we don’t have to send $864 in taxes to the IRS. I think the client should live mortgage free and not make an $864 trade for $4,800.

Debt for Investing

If your house was paid off, would you take out a loan to invest it in the stock market? Amazingly many people reading this would actually pause to think about it. The answer should be no. Why would you risk your house to make more money? Greed! By not paying off your home you are essentially putting your home at risk, or at the very least your retirement.

Client Behavior – Peace of Mind

When the world news cycle is rocking your portfolio with volatility and markets are hitting 52-week lows, many individual investors tend to get very conservative. A good financial advisor can help keep the eyes on the big picture, but in the end it is the client’s decision on what action takes place. If the client feels better paying off the mortgage at this point, then he or she has sold their portfolio at a rock bottom price to have the peace of mind that at least their house is paid for. The problem now is that there is less of a portfolio working to generate income and growth for retirement. Paying off the home at the beginning of retirement allows the client to have the peace of mind at the start of the retirement journey, better long-term investing behavior and hopefully a larger nest egg.

Making an IRA Withdrawal to Pay off the Mortgage

If all your retirement money is inside an tax deferred account, paying off the home in stages may be the best option. If you retire in November then you could pull half the payment out immediately then the other half in January of the next year, keeping your withdrawals taxed at a lower rate in each year. You can take the funds out over a three year period or more depending on your tax situation. Planning for tax efficient withdrawals is a little different for every family. You should discuss this type of strategy with your tax advisor. Your portfolio manager should keep the funds allocated for home payoff in a CD or a short-term financial instrument. You do not want to put these funds at risk.

Exceptions

It is impossible and not my intent to give financial advice via any blog as each family has unique situations. One of the most common exceptions is a situation where paying off the home would not allow enough funds to generate retirement income. This would be where the home payoff is a very big number and the retirement fund is lower than it needs to be. This is where hard decisions have to be made. The retiree can downsize the home, work longer, or possibly refinance the home to get a lower payment while understanding that the home will never be paid off. Generally in this situation, working longer is the better option but sometimes where retirement is forced, downsizing is the only option.

Another exception could be where a client’s income comes from. Perhaps you and your spouse have six digit annual pension payments. If so, this might change how we think about your financial security. Of course, this depends on where these pensions are coming from.

For the Younger Investor

For those in their 20’s and 30’s don’t become house poor. Forget about what your friends and family have and live within your means. A 15-year mortgage would be the ideal situation. For those within 10-15 years of retirement, start planning now to make those extra payments to have no mortgage at your retirement date. Your retirement lifestyle will be better for it.

Careful Where Your Advice Comes From

I think the bottom line in the differences in the opinion on this subject comes from how an advisor gets paid and if they have the capacity to think independently from their corporate literature. These large national brokers would stand to lose a lot of commission revenue if their thousands of brokers nationwide were told to give sound financial advice. Looking at my peers, it seems that the fiduciary fee only advisors say pay it off, where the national brokerage firms seem to want to gamble with their client’s best interest.

Remember that fiduciary + fee-only = your best interest.