According to this article the public and private key are not that different from "normal" RSA keys. So you can use the same RSAPublicKey and RSAPrivateCrtKey as containers. The problem is that the KeyStore implementations are very limited (pkcs#12, jks, jceks) - they cannot be used to store a single private key. You would need to create a certificate chain for the public key. This could be a self signed certificate, but it is quite a hassle just to store a key.
You could also create your own KeyStore implementation but that seems to be a huge load of work. Implementing KeyStoreSpi in your own provider is slightly less complicated, but it requires your provider to be signed with a private key and a certificate signed by Oracle.
Basically I would go with the self signed certificate trick explained above or with the serialization scheme proposed by Duncan.
For non believers:
Exception in thread "main" java.lang.IllegalArgumentException: invalid zero-length input chain
    at java.security.KeyStore$PrivateKeyEntry.<init>(KeyStore.java:393)
and 
Exception in thread "main" java.lang.NullPointerException: invalid null input
    at java.security.KeyStore$PrivateKeyEntry.<init>(KeyStore.java:390)