You could use solar thermal panels to preheat your water on the way to the water heater. They pay for themselves pretty quickly and work any time (to greater and lesser degrees) the outdoor temperature is 60 or above.

Yes, things that last do cost more, initially. In Europe, costs tend to be evaluated over the lifespan of the building, that’s one reason their commercial buildings are 20 years ahead of ours in technology for energy efficiency.

I don’t have an easy answer on the priorities thing, wish I did. As a general observation, people do what’s most important to them… I’m not about to tell you that staying out of debt is more important than saving energy… it’s your choice, and I don’t think you’ve chosen unreasonably.