Is it worth the time or should give it a pass
I agree with Mary that it is a personal choice. The Changing of the Guard ceremony originally took place in the Palace of the Whitehall (the official residence of the King in London) until 1689. The ceremony moved to Buckingham Palace in 1837 when Queen Victoria made it her home in London.
Either go early and get the best spots at the Front Gate or stand on the steps of Queen Victoria's statue, located opposite the palace. I've seen the ceremony twice from both spots.
Only you can decide whether the Changing of the Guard is 'worth seeing'. It's a purely personal choice though, if you're only likely to visit London once, imo it's a piece of historical ceremony which does deserve a look.
There isn't really one 'best' location as such imo, though if you want to see the actual ceremony you'll need to be at Buckingham Palace Road itself. The 'best' views come from getting as close to the crowd barriers as possible.
If you're happy to just see the troops, horses, bands etc then anywhere along the route is fine though...again..you need to be as close to the barriers as possible.
This page details the route and timings:
and this page has suggestions for 'best' places to stand:
If you don't fancy that, you might consider the Changing of the Life Guard (also known as the Horse Guards...the ones on horses?) which takes place at Horse Guards Parade almost every day. It's a smaller ceremony and less crowded:
Mentioned in this answer: