Provided you keep your guards clean inside, they should only sustain cosmetic surface scuffing on the sides, but it is important to remember that you still risk rust. But walking around in guards when off the ice is far safer than not wearing them. If you're going to be off the ice for a lengthy period of time, however, just take your boots off and give your feet a rest.