The permanent piece called "Meeting" at MoMA PS1 is an etherial experience of light and color and quiet. Sit and look through the cutout in the ceiling and watch the light change both outside and in. You can't tell what you're looking at until maybe a bird flies by.
Within an couple hours' drive of NYC, you can find lots of sleepy seaside towns. Not only can you get your soft ice cream and salt water taffy on, but you can still spot cool, hand-lettered signs and other relics of a past generation's amusements.