Just get the position of the last character occurrence.
for(String serverId : serverIds) {
 sb.append(serverId);
 sb.append(",");
}
sb.deleteCharAt(sb.lastIndexOf(","));
Since lastIndexOf will perform a reverse search, and you know that it will find at the first try, performance won't be an issue here.
EDIT
Since I keep getting ups on my answer (thanks folks ), it is worth regarding that:
On Java 8 onward it would just be more legible and explicit to use StringJoiner.
It has one method for a simple separator, and an overload for prefix and suffix.
Examples taken from here: example
Example using simple separator:
    StringJoiner mystring = new StringJoiner("-");    
    // Joining multiple strings by using add() method  
    mystring.add("Logan");  
    mystring.add("Magneto");  
    mystring.add("Rogue");  
    mystring.add("Storm");  
    System.out.println(mystring);
Output:
Logan-Magneto-Rogue-Storm
Example with suffix and prefix:
    StringJoiner mystring = new StringJoiner(",", "(", ")");    
    // Joining multiple strings by using add() method  
    mystring.add("Negan");  
    mystring.add("Rick");  
    mystring.add("Maggie");  
    mystring.add("Daryl");  
    System.out.println(mystring);
Output
(Negan,Rick,Maggie,Daryl)