Since, as you state, your net connection does not exceed 50mbps in either direction, and your NIC card operates at 100mbps in either direction, you have bandwidth to spare at the NIC. It is not the source of your bottleneck, so there is no pressing need to upgrade your NIC.
If you did upgrade your NIC, unless you purchased a gigabit NIC like THIS you would still be purchasing a NIC operating at 100mbps in either direction... just like the one you have. If you did purchase a gigabit NIC, you would also have to purchase a gigabit router to connect it to, in order to have faster than 100mbps between you and the router. But... purchasing a gigabit NIC and Router would not increase your net speed, since you only have 50mbps upload. You could theoretically operate the server with a Wireless G Nic, at 54mbps and still provide just as fast of a connection.
If you are having 100 people connect, your biggest potential bottlenecks are the amount of Ram, the speed of your processor and the number of cores/threads.