You're executing mkdir with the following arguments:
"-p", "/home/stuff/Keyring", "&& touch", "/home/stuff/Keyring/keyring.gpg"
(the first argument to exec() is the process to execute) so the -p tells mkdir to build parent directories if required, and the remaining arguments are the directories to create. Hence your problem/issue (I suspect you'll have a dir somewhere called '&& touch')
It looks like you want to execute a shell script, so you need to encapsulate the above as such e.g. provide arguments such as:
/bin/sh -c "mkdir -p /home/stuff/Keyring && touch /home/stuff/Keyring/keyring.gpg"
i.e. you're executing /bin/sh, and providing the commands on the shell command line.
Or better still, use the java.io.File API or similar, and avoid forking processes altogether?