You need to first compare the $163k to the current home value, is it 80% or less? If not then you may not be able to actually refinance because paying off the $39k still doesn't get you to 80% of todays LTV. If the house is actually worth more than the combined $202k you may not need to pay off the full $39k in order to do this, keeping some in the 401(k) and avoiding penalties.
If the appraisal works out to do this the tax and penalties are real costs today along with the closing costs on the new loan, though in the long run the taxes need to be paid and if you are in the same rate today as you are when you make the retirement withdrawal the tax effect is a wash. If you want to be conservative include the cost of tax today, otherwise only focus on the penalty and closing costs. I would do it for myself given your loans and rates along with the current rates out there and I would strongly encourage you to use all your savings as an increase in your retirement contributions. Also, depending how far into the loan you are consider moving to a 15 year loan instead.