You can't say you've seen NYC until you've seen it from a boat. Hop on the ferry from anywhere to anywhere and you'll see what I mean. Not only will you see excellent old buildings and factories--the inner workings of the old industrial city--but you'll head out under working bridges and see the skyscrapers in all their towering glory. And once you've done it, do it again, at night.
Occasionally I'll look at the Manhattan skyline and wonder what it was like in 1931 when people around the world were saying, "Holy shit, did you hear what they just built in New York?" Just get up there and trip out about humanity. And be sure to keep an eye out for the unmarked locked door guarding a set of stairs leading to a hidden terrace that was originally designed as a fucking docking station for zeppelins.
The Bronx Botanical Garden is a place my father and I have been going to since I was a child. Here's a photograph I took of him here. It has hundreds of species of trees and flowers. Sitting in the middle of the Bronx, it's most recognizable as a green oasis from the grays and concrete of city life.