Bum Phillips, the coach of the Houston Oilers in the late '70s and early '80s (Luv ya Blue!) famously wore his silverbelly cowboy hat everywhere except while coaching the Oilers in the Astrodome. Why didn't he wear his hat in the Astrodome? Why, because his momma taught him that a gentleman didn't wear his hat indoors, of course!
With all due respect to Mrs. Phillips, that's not quite right. There are two reasons why a gentleman must take off his hat: if he's indoors in a private space or if respect obligates him to do so. It's perfectly appropriate for him to wear his hat indoors if he is in a public space. Thus, he can keep it on while in the lobby or corridors of an office building or walking around a mall. It comes off once he enters a private space, like his office or a house or a table at a restaurant. It also comes off when he wishes to pay respect to someone or something, meaning that going hatless at funerals or during the playing of the national anthem or when talking to a lady.
Of course, there are always exceptions. Rules of etiquette need not be followed when doing so would result in a manifest absurdity or violations of other rules of etiquette or decorum. Consider, for example, the rule that a gentleman takes off his hat when talking to a lady. What if it's raining or a gale is blowing? Etiquette does not require that he freeze to death, and so he may put his hat back on after taking it off to pay his respect. How about a diner lunch counter? If he takes off his hat, he has to put it somewhere; and given the layout of a lunch counter, that somewhere would be a place where someone else could have sat. What's more discourteous? Keeping the hat on, or taking up a space with a hat that another customer might want to use to sit and eat? And so he keeps his hat on. Normally, he would take his hat off in an elevator if a lady is present, but what if the elevator is crowded? Taking his hat off and holding it makes the elevator more crowded, and so he need not and should not do so. The point is that etiquette is supposed to be a set of rules based on common sense and common decency. It is not an excuse to inconvenience those that one comes in contact with.