Yaw control with a gyroscope?

I've been designing a small SSTO that uses a gyroscope for stability in space and the upper parts of the atmosphere.

So far I'm mostly happy with the Roll and Pitch PID settings I've set, there is none for Yaw however. If I give any yaw input while the craft is in low atmosphere or vacuum it will continue in that direction without stopping. RCS has proven to be difficult to solve this problem with as again, I'm using the PID controls found in the flight info tab.

How can I manage this?