Wanganui is a pretty little city with an amazing river. It has the only earthbound elevator in the Southern Hemisphere (inside Durie Hill!) Also a good place for the kids is Kowhai Park on the edge of the Wanganui River. There are some great river activities further upstream, but in the town itself the riverboat is about it.
Be warned, the people are crazy! Some in a great way and some in a crazy way. But they make great friends.