The French Quarter is wonderful. Decatur Street was by far my favorite area. It's full of shops and plenty of artists, especially on the weekends, and wonderful food. Definitely do a walk around the quarter to get in all the sights of the beautiful buildings. Walk Bourbon street at least once at night.
We just passed by Gudauri twice on our way to and from Stepentsminda. There is an amazing viewpoint where you can see mountain peaks covered in snow and basically you walk on snow too (it was in May). Amazing place!
Come a bit before sunset to save a good spot for yourself, a bit away from other groups. Nice view, quite peaceful. Locals smile and stare at you but in a curious way!
Clean beach but the waves are rather strong. Not much to do around but also means not many tourists and kids. There a few bars where you can hang out at the evenings but not much choice. Anyways, the beach is nice to chill for a day or two.