I stayed four nights at a 108-year-old apartment (an Airbnb), a 5-minute walk from Hallgrímskirkjaand Laugavegur. And walking distance to the oldest public bath house in Iceland - Sundhöll Reykjavíkur(Sundhollin).
If this is your first trip I recommend joining City Walk Walking Tour: https://citywalk.is/ and The Golden Circle Tour.
Mentioned in this answer:
Hotel Odinsve. A 4 star hotel not actually "in" downtown, but it's only a 5-10 minute walk away. And being in an upscale, mostly residential area just beyond downtown means it's quiet at night, which isn't something you'll find in any of the downtown hotels.
Reykjavik is very small, very safe and very walkable so there really isn't a 'best' location in the city centre (called 'downtown' in the US). Iceland isn't a cheap country and accommodation costs are high, so your opinion of 'best' in budget terms may not chime with the opinion of others. There are plenty of hotels but even more apartments, guesthouses and airbnbs in Reykjavik.
The local authorities have set the pickup/drop-off stops for airport buses and all group tours. The heart of the city centre is largely pedestrianised so you'l usually have to walk a bit to get to a bus stop. You can see all the bus stops, with photos and list of nearby accommodation, on this link:
I've just stayed at the Centerhotel Midgardur, almost opposite bus stop 10 and a fairly new addition to the Reykjavik Centerhotel chain. Friends stayed at the Centerhotel Arnarhvoll, nearer the harbour. My hotel was clean, comfortable and warm (I could control my own heating, which matters!) with speedy lifts and friendly, helpful staff who all spoke excellent English. The bed was comfortable, the curtains kept out the light (that really matters in summer Iceland) , the hot water (geothermal, like most hot water in Iceland) was ample and the breakfast buffet (Continental + some hot dishes, with wonderful fresh bread) was really excellent. Midgardur also has its own spa with a small geothermal pool and its own restaurant, with good food at reasonable (for Iceland) price.
Reykjavik hotel rooms are of course European-sized (Iceland is neither the US or Canada) and the decor tends towards Scandanavian: clean & minimalistic. The only tiny criticism I had about the Centerhotel was the lack of a drawer to store my underwear!
Most first-time visitors take the Golden Circle and South Coast tours, both of which last almost all day and include stunning scenery: lavafields, glaciers, geysirs, black beaches, mountains, waterfalls etc etc. The Snaefellsnes tour is also superb...perhaps more so because you're not sharing it with quite so many other tour buses (especially if a cruise ship is berthed) and, of course, there are whale-watching and other tours available as well.
Do make sure you pack ample layers, good waterproofs and footwear plus hat/scarf/gloves even if you're visiting in summer. Icelandic weather changes very quickly indeed and the windchill can easily make it feel much colder than it actually is, especially once you're away from Reykjavik.
Alcohol is very expensive indeed so it's worth taking advantage of the happy hours offered by Reykjavik bars & hotels. You can download an app ('Appy Hour' from google and itunes) which lists all the happy hour times.
Iceland is stunning. Enjoy your trip!
Mentioned in this answer:
I stayed 4 nights at the Skuggi downtown and loved it. Smaller typical European size rooms but clean, modern, and a good breakfast. Centrally located middle of the hill. I will say the last day I spent time at the spa at the Hilton and that looked great too but not as centrally located.
Make sure you leave plenty of time for the blue lagoon and get there early in the am. Book the golden circle tour to see the falls and the geysers.
Try the hot dog place down by the harbor that bill Clinton made popular and the fries shop in town.