To solve this issue you must use SETLOCAL EnableDelayedExpansion and ! instead of % for the SET statement like so,
SET t=!test:~1,%number%!
If I understand this correctly, the ! makes the interpreter read test:~1, %number% instead of before where it was reading test:~1 and number separately due to the interpreter thinking the expression was closed because of using two % symbols. In another way, it thought you meant two expressions because of the paired % symbols. Thanks to Aacini for linking this answer which helped me find the solution!