Take the B61 bus into Red Hook and get off at Van Brunt/Dikeman Street, right across the street from my favorite bakery: Baked. If they have them, get the black and white cookies and the funfetti sprinkle cake, or pretty much anything else.
Right outside of Baked, there are lots of lovely and unique (read: expensive) stores, including Erie Basin for vintage jewelry, Foxy and Winston for paper goods, and Kempton and Co. for bags and candles.
After browsing, walk over to Conover Street to check out Cacao Prieto chocolates and Widow Jane whiskey distillery, located together in a beautiful, open factory space. I liked the American Oak and the rum.