There is a "$REYCLE.BIN" folder in my external hard drive; can I delete it?
What do I need to do?
Personally I would not delete it, I would just disable it for that drive. It won't hurt to have on empty hidden folder on your drive.
Right click on Recycle Bin>Properties and highlight the drive. Then change the "Settings for selected location" to "Don't move files to the Recycle Bin. Remove files immediately when deleted."

When you delete file from your external hard drive, it's been gone to some hidden place in your external hard drives. $REYCLE.BIN folder in this circumstances may be the folder contains those files or links to those files. If you can see some files in this folder, I suggest do not delete it. Delete it may be a harmless action, but we don't need to do that, right?