Try a weekday during the summer. Take the basic tour if it's your first time. You can make reservations in advance for an exact departure time. If you go on Wednesday, you can drive across the street to WR Hearst State Beach and grab a sandwich at Sebastian's General Store. (They are also open on the weekend.) Skip the food in the Castle reception area. Also, continue up Hwy 1 for a few miles to check out the elephant seals. Not to be missed!
If going in summer, visit early before the tourist crowds. I would also suggest getting the ticket in advance. The castle is busy and crowded in summer. Also, it can often be in fog during summer mornings and you definitely want to see the view. If arriving in afternoon, buy your ticket and fill the time waiting by viewing the movie or driving across the highway to the pier, beach, and historic market Just five minutes away.