I know that this is meant to be a guide to Edinburgh, but if you have the time to spare there are plenty of other parts of Scotland in easy traveling distance which are well worth a visit.
Glasgow is the biggest city in Scotland and is less than an hour from Edinburgh by frequent train service. The express train service drops you off at Queen Street station which is right in the centre of Glasgow. The city had a bad reputation for a long time due in the most part to the slum conditions brought about when it was one of the most industrialised cities of the British Empire and a centre of the foremost shipbuilding area in the world. Nowadays things are totally different and the city centre in particular houses some of the finest Victorian architecture anywhere - a policy of renovating older buildings rather than replacing them has resulted in a delightful area with some excellent shopping and lively nightlife.
One thing in particular that should not be missed is the Burrell Collection. This was amassed by a wealthy Victorian citizen and left to the city on condition that it was to be kept together and exhibited only in Glasgow - resulting in the collection languishing in storage for many years until a suitable site could be arranged for its display. If you only visit one Museum or Gallery in Scotland then this should be it! (This is not to recommend only visiting one.)
St. Andrews
Stirling
Linlithgow
East Lothian