No you can't.
A stack frame extends from the point where the first parameter is pushed onto the stack up to the last modification of the stack due to any action or instruction. I.e. allocating space for a dynamic variable using alloca() or pushing a register, or simply storing a register in preallocated stack position. 
Actual compiler technology is so advanced that even the standard functions prologues and epilogues can be modified for the most aggressive optimization, making even more complex the frame unwinding.
Acting only on the base frame, that in the case of a 32bits X86 code corresponds to the value stored in the %ebp register, isn't enough.
Most recent evolution, as also @Jester said, demonstrated that even %ebp isn't necessary for exceptions stack unwind, and aggressive optimization switches in compilers, as -fomit-frame-pointer in GCC, doesn't save it on stack anymore (see libunwind below).
To access the previous frame, starting from the base frame address hold in %ebp, you need the stack width of the stack frame for the relative function, inclusive of all stack operations.
This information is normally stored in a specific section of the executable, generally using a list of records about stack operations. The sections are:
- .pdatafor Microsoft PE executables
- .eh_frameand- .eh_frame_hdrfor ELF format (Linux).
Depending on which OS you want to work on you can get more info starting from https://www-user.tu-chemnitz.de/~heha/viewchm.php/hs/Win32SEH.chm/Win32SEH.htm for Microsoft executables.
For Linux based system you can start to look here https://gnu.wildebeest.org/blog/mjw/2007/08/23/stack-unwinding/ and eventually have an in deep reading about the DWARF debugger http://wiki.dwarfstd.org/index.php?title=Special:SpecialPages, and the https://www.nongnu.org/libunwind/ for a technically detailed immersion.