I tried following the instructions in Backspace key not working in vim , but I still have the same problem: BACKSPACE inserts a ^?
That is to say: inserting set bs=indent,eol,start in my ~/.vimrc did not solve the problem.
I am sure that I am loading the correct source file (I did a test of another setting, and the change propagated).
I am using: Vi IMproved 7.2 on Ubuntu with Gnome. In the terminal application, backspace works properly.

