Just delete /System/Library/Frameworks/Ruby.framework, /Library/Ruby, and erb, gem, irb, rdoc, ri, ruby, and testrb from /usr/bin.
Warning: OS X expects and sometimes relies on frameworks and software shipped with the system. It would be a very bad idea to remove the system Ruby. It might even get replaced next time you update OS X -- or it might not, but OS X might expect it to be there, and then bad things will happen. Remove it at your own risk. I am providing this answer because you asked, but I would not recommend removing a system framework.
Why do you want to remove it? It's not hurting anything to sit there, unused.