Photography does not seem to be considered a harmless pursuit by many. While they may not technically see it as a crime, it is viewed with suspicion by building managers and security guards. I have been challenged on three occasions in the past fortnight while snapping the exterior of buildings from publicly accessible vantage points, including the street. I try not to get annoyed with such officious nonsense, but it's not easy. I'll generally stand my ground if in a public place, and not let it get in the way of getting the shot I'm after.
It's another matter inside buildings of course. I'll ask permission where appropriate and if there's someone to ask. One such occasion recently was inside Southern Cross station. The reality of the situation is that in this grand publicly owned space people, especially travellers and tourists, are taking pictures everywhere without seeking permission. However, on this occasion, I thought it best to ask as I wanted to use a tripod and take my time photographing the roof. I found the appropriate official, answered their questions (exactly how do you answer the "WHY" without sounding artistically pretentious???), and set about my task. As it turned out, it was worth the bother because, as I walked up the steps to the security office, I spotted a most curious arrangement of tables and stools at the cafe below. As for Mr Curly, he was part of a Spring Racing Carnival promotion. He was happy to strike a pose and I snapped him without bothering to seek the approval of the station poobahs. Please don't report me to the authorities.