You can visit Kinkakuji then take a bus there. I took a bus to Gion then walk to Arashiyama & Kiyomizudera. Bus guide available at the visitor center at Kyoto station.
Whether to go depends on you, to me both Kinkakuji and Kiyomizudera is worth the trip, but Kiyomizudera and Arashiyama has more things to see