Problem 1 - Normal vs Delayed expansion of variables
Normal expansion using %var% occurs at parse time, and the entire IF statement is parsed all at once, including the contents within the parentheses. So something like echo (%usertimeouthours%) displays the value of USERTIMEOUTHOURS as it existed before the IF statement was executed (before SET /P set the value).
The fix is to enable delayed expansion at the top using setlocal enableDelayedExpansion and use !var! instead of %var%. Delayed expansion occurs at execution time instead of parse time.
Problem 2 - Unescaped special characters
There are a number of characters that have special meaning and must be either escaped with ^ or quoted if you want the character to be treated as a string literal.
One of the special characters is ). It will terminate any code block opened with ( unless it is escaped or quoted. You need to escape that character as ^) when you use it in your ECHO statements because those statements are within a parenthesized block of code.
Simplification of your SET /A statement
You do not need to expand variables when used in a SET /A computation. You can simply use the variable name without enclosing in percents or exclamations. This convenience only works with SET /A.
@echo off
setlocal enableDelayedExpansion
set /p usercommand= "Input:"
if "%usercommand%" equ "done" (
  set /p usertimeouthours= "Input Hours: "
  echo (!usertimeouthours!^)
  set /p usertimeoutminutes= "Input Minutes: "
  echo (!usertimeoutminutes!^)
  set /p usertimeoutseconds= "Input Seconds: "
  echo (!usertimeoutseconds!^)
  set /a answer= usertimeouthours*3600+usertimeoutminutes*60+usertimeoutseconds
  echo !answer!
  goto end
) else (
  echo finished
  goto user
)
:end