We stayed at a hotel for a couple days in the town of Sivas. It’s run by a German family who speaks English, German, and Greek. Excellent out of the way place with a killer little cafe in the middle of the town. This is definitely out of the way of the tourists but fantastic! It was our favorite part of Greece

http://www.shivas.com/

Also check out Falasarna/Kissamos on the western coast. There is a lot of history and a killer little restaurant (Spelios?) on the road that traverses down to the ruins. It’s worth having dinner while watching the sun set. Not sure about lodgings as we took an afternoon trip out there by car.

Can’t help much with the Crete issue, but you will likely have to travel through the mainland to get there. There should be a very nice & effective train from the international airport to downtown Athens. It is well worth staying a night there to visit the Acropolis if nothing else. If you overnight in Athens, I highly recommend The Student and Travellers Inn. Enjoy!

I’ve found the western side of Crete, from Rethymnon to Chania province and the southern coast, have less drunken tourists. You can find some out of the way places to stay, swim and places to eat. The worst tends to be the north east coast, the stretch from from Heraklion to Agios Nikolaos – here you can forget about a relaxing time!

We have stayed in some pre-booked places but this site might be worth looking at, not sure how much you want to spend, they might be a bit expensive http://www.i-escape.com/greece/boutique-hotels