Here is a pure batch-file solution which regards all of your requirements (see also rem comments):
@echo off
setlocal EnableExtensions
rem definition of temporary file:
set "TEMPFILE=%TMP%\~AlphaNum.tmp"
rem first loop structure for listing the files with `dir`
rem and creating the temporary file for later sorting:
> "%TEMPFILE%" (
    for /F "eol=| tokens=1,2,3,* delims=_" %%I in ('
        dir /B /A:-D "*_*_file*_*.ext"
    ') do (
        rem store different parts of the file name into variables:
        setlocal DisableDelayedExpansion
        set "LINI=%%~I" & rem this is the very first part (prefix)
        set "LINL=%%~J" & rem this is the part left to `file*`
        set "LINM=%%~K" & rem this is the part `file*` (`*` is numeric)
        set "LINR=%%~L" & rem this is the part right to `file*`
        setlocal EnableDelayedExpansion
        rem extract the numeric part of `file*` and pad with leading `0`s;
        rem so the index is now of fixed width (12 figures at most here):
        set "LINN=000000000000!LINM:*file=!" & set "LINN=!LINN:~-12!"
        rem write file name with replaced fixed-width index and without
        rem word `file`, followed by `|`, followed by original file name,
        rem to the temporary file (the `echo` is redirected `>`):
        echo !LINI!_!LINL!_!LINN!_!LINR!^|!LINI!_!LINL!_!LINM!_!LINR!
        endlocal
        endlocal
    )
)
rem second loop structure for reading the temporary file,
rem sorting its lines with `sort`, generating new indexes
rem (sorting the previously built fixed-width indexes as text
rem result in the same order as sorting them as numbers) and
rem building the new file names (original first part + new index):
set "PREV="
for /F "eol=| tokens=2 delims=|" %%I in ('
    sort "%TEMPFILE%"
') do (
    setlocal DisableDelayedExpansion
    rem get the full original file name, and its extension:
    set "FILE=%%~I"
    set "FEXT=%%~xI"
    rem this loop iterates once only and extracts the part before the first `_`:
    for /F "eol=| tokens=1 delims=_" %%X in ("%%~I") do (
        set "CURR=%%~X"
    )
    setlocal EnableDelayedExpansion
    rem if the current prefix equals the previous one, increment the index;
    rem otherwise, reset the index to `1`:
    if /I "!CURR!"=="!PREV!" (
        set /A SNUM+=1
    ) else (
        set /A SNUM=1
    )
    rem remove `ECHO` from the following line to actually rename files:
    ECHO ren "!FILE!" "!CURR!-!SNUM!!FEXT!"
    rem this loop iterates once only and transports the values of
    rem some variable values past the `setlocal`/`endlocal` barrier:
    for /F "tokens=1,* delims=|" %%X in ("!SNUM!|"!CURR!"") do (
        endlocal
        endlocal
        set "SNUM=%%X"
        set "PREV=%%~Y"
    )
)
rem remove `REM` from the following line to delete temporary file:
REM del "%TEMPFILE%"
endlocal
The toggling of EnableDelayedExpansion and DisableDelayedExpansion is required to make the script robust for any special characters that might occur in file names, like %, !, (, ), & and ^. Type set /? into the command prompt to find brief information about delayed variable expansion.
This approach relies on the following assumptions:
- the files match the pattern *_*_file*_*.ext;
- none of the file name parts *contain_characters on its own;
- the indexes after the word fileare decimal numbers with up to 12 digits;
- the part between first and second _takes precedence over the index after the wordfilewith respect to the sort order; so for instance, supposing there are two filesabc_AAA_file8_*.extandabc_BBB_file5_*.ext,abc_AAA_file8_*.extwill be renamed toabc-1.extandabc_BBB_file5_*.exttoabc-2.ext, becauseAAAcomes beforeBBB; if this is not the desired behaviour, exchange theechocommand line in the first loop structure by this one:
 echo !LINI!_!LINN!_!LINL!_!LINR!^|!LINI!_!LINL!_!LINM!_!LINR!;
- the newly generated indexes are not padded with leading zeros;
With the sample files from your question, the temporary file contains the following lines:
abc_foo_000000000001_morestuff.ext|abc_foo_file1_morestuff.ext
abc_foo_000000000002_morestuff.ext|abc_foo_file2_morestuff.ext
efg_goo_000000000001_morestuff.ext|efg_goo_file1_morestuff.ext
jkl_zoo_000000000000_morestuff.ext|jkl_zoo_file0_morestuff.ext
jkl_zoo_000000000001_morestuff.ext|jkl_zoo_file1_morestuff.ext
jkl_zoo_000000000004_morestuff.ext|jkl_zoo_file4_morestuff.ext
xyz_roo_000000000006_morestuff.ext|xyz_roo_file6_morestuff.ext