I don't have tons of time and I want to do it on foot from near the Atlantic Centre
Quidi Vidi is very pleasant and scenic. The brewery there also does tastings, and I thought it was interesting learning about the quirks of brewing beer on the island. Most importantly, there's a public bus that goes to Quidi Vidi. I recall there being some tour buses that go up to Signal Hill, but there is no public bus. So you'd have to walk it. I also recall a lot of people doing so, but it's a very long way up a tall hill.
So I would enjoy the town then, if you have time, take the bus to Quidi Vidi.
The answer bocmaxima has given is right.
I was at both and well worth a visit ;-))). Busy adding in my tips . . .