There are a few temples to visit, an island and what is the most interesting is a bamboo bridge. It's build completely of bamboo, no steel at all. Amazing construction!
There is a resort with a lake and water park where you can pay around $5 for full day of activities. You can pay for the night too if you want to stay.
I stayed in Kampot for 2 days. One day we visited Kep for a day, strolled National Park and chilled at the beach and second day we visited Bokor National Park close to Kampot, walked around the town and market. Everything is close by. Unfortunately there's no beach in Kampot, only the river but I wouldn't swim there. It's becoming more and more touristic so better to visit sooner than later.