i’ve not been to leith – but have spent about three weeks time in edinburgh over the last two years. all of the things to do suggestions above are great – so i thought i’d add a great place to stay.
i’ve always stayed at the brodies 2 hostel. http://www.brodieshostels.co.uk/
the “2” location is the newer, non-smoking one. it’s super, super clean, and the staff and people staying there have always been wonderful. and i know they even have private rooms for two people – if that’s something you two might want to go for on the honeymoon!
also, the location cannot be beat – right on the royal mile.
i love visiting scotland so much – it’s a special place. you will have a lovely time!

ps – with your limited time – i’d def. recommend the royal mile and a trek up arthur’s seat.