The reason is because your view will be reused in your adapter. 
public class ImageAdapter extends BaseAdapter {
    private Context context;
    private final String[] someprivatevariable;
    public ImageAdapter(Context context, String[] mobileValues) {
        // Your boring constructor
    }
    public View getView(int position, View convertView, ViewGroup parent) {
        LayoutInflater inflater = (LayoutInflater) context
            .getSystemService(Context.LAYOUT_INFLATER_SERVICE);
        View gridView;
            // REUSE VIEW IF NOT NULL CODE
        if (convertView == null) {
                  .....
                  .....
                  you generally create your view here
        } else {
            gridView = (View) convertView;
        }
        return gridView;
    }
}
If your view is forced not to reuse then you can scrolltoposition by saving this value
int index = gridview.getFirstVisiblePosition(); 
and restore the value by 
gridview.smoothScrollToPosition(int index)