Make sure to walk over to the Ile-St. Louis which is connected to the island where Notre Dame is by a footbridge. Less tourists, better place to grab a bite or ice cream (see Berthillon) and a cool neighborhood.
It's essentially just a mini Target, but still, I could spend hours here. They usually have supermarkets as well as clothing, cosmetics, etc. It's a great place to pick up incidentals.