From Edinburgh, it's time to head even further north into the Scottish Highlands. There are a couple of filming locations scattered around, but it shouldn't take more than a day to visit them all.
Near the village of Glencoe you'll find a little lake surrounded by steep grassy banks, on which Hagrid's hut and pumpkin patch were built for Prisoner of Azkaban. You can park at the Clachaig Inn and walk to the site, which is conveniently marked on Google Maps as "Site Where Hagrid's Hut Was Filmed."
The slopes around Glencoe were the backdrop for a lot of the exterior shots for the third film, including the iconic scene where Hermione punches Malfoy in the face. There are also some great hikes around here if you've got the time.