Depends, if it's some building and you want to take a fancy lampshade. Do it, just don't get caught and don't brag.

If it's the palace at Knossos and you're sneaking past Greek customs by night-time packet boat so you can rebuild an entire temple inside your museum at Oxford, maybe don't