Iam having a ViewHolder class like this
    static class ViewHolder {
    protected String fileName;
    protected Bitmap bitmap = null;
    protected CheckBox checkBox;
    protected int position;
    protected int resourceId = 0;
    protected ImageView imageView;
    protected TextView textView;
}
and within my getView()
@Override
public View getView(int position, View convertView, ViewGroup parent) {
    final ViewHolder viewHolder;
    if(convertView != null)
    {
        ViewHolder holder = (ViewHolder) convertView.getTag();
        if(!holder.fileName.equals(fileList.get(position)))
            convertView = null;
    }
    if(convertView == null)
    {
        convertView = inflater.inflate(R.layout.image_layout, null, false);
        viewHolder = new ViewHolder();
        viewHolder.imageView = (ImageView) convertView.findViewById(R.id.imageView);
        viewHolder.textView = (TextView) convertView.findViewById(R.id.text);
        // Set viewHolder attributes
        viewHolder.position = position;
        viewHolder.fileName = fileList.get(position);;
        // set the checkbox
        viewHolder.checkBox = (CheckBox) convertView.findViewById(R.id.checkBox);
        // Set the path of the file
        final String filePath = context.getBasePath(position);
        if(new File(filePath).isDirectory())
        {
            viewHolder.imageView.setImageResource(R.drawable.folder);
            viewHolder.resourceId = R.drawable.folder;
        }
        else
        {
            String mimeType = Utility.getMimeType(filePath);
            if(mimeType.contains("image"))
            {
                loadImage(viewHolder, viewHolder.imageView, filePath);
            }
            else
                viewHolder.resourceId = handleFile(viewHolder.imageView, filePath);
        }
        convertView.setTag(viewHolder);
    }
    else
    {
        viewHolder = (ViewHolder) convertView.getTag();
        if(viewHolder.bitmap == null)
            viewHolder.imageView.setImageResource(viewHolder.resourceId);
        else
            viewHolder.imageView.setImageBitmap(viewHolder.bitmap);
    }
    viewHolder.checkBox.setOnCheckedChangeListener(new OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton checkBox, boolean checked) {
            viewHolder.checkBox.setChecked(checked);
            listener.onFileStateChanged(viewHolder.position, checked);
        }
    });
    // set the fileName
    viewHolder.textView.setText(viewHolder.fileName);
    if(checkBoxVisibility)
        viewHolder.checkBox.setVisibility(View.VISIBLE);
    else
        viewHolder.checkBox.setVisibility(View.INVISIBLE);
    return convertView;
}
And at this time the when i refresh the Adapter using notifydatasetChanged() without changing data, the ViewHolder that i get by calling getTag() was returning wrong tags. It was working like
If the position is 1 the holder object was of the view at position 0. ie; the previous object was returned..
Atlast I found a post
which is not the same but with the same method and the solution works for me .. I don't know how ..
The Solution was
setting the width of ListView to MATCH_PARENT in xml, In the post it was height. 
Does any one know , What is the reason for such a behaviour ? There is no logic that i can find behind this issue
 
     
    