Mcgolrick Park at midnight in the snow. Any early morning with all the bums and drunks passed out on Nassau Avenue. NYC still looks best between 3am and 6am, just before the light comes in to shine on all its flaws.
Within an couple hours' drive of NYC, you can find lots of sleepy seaside towns. Not only can you get your soft ice cream and salt water taffy on, but you can still spot cool, hand-lettered signs and other relics of a past generation's amusements.