How to "apply" backspace characters within a text file (ideally in vim)

Question

I have a log file with backspace characters in it (^H). I'm looking through the file in Vim and it can be quite hard to see what's going on.

Ideally I'd like to be able to "apply" all the ^H on a given line/range so that I can see the final result.

I'd much rather do this within Vim on a line-by-line basis, but a solution which converts the whole file is better than nothing.

Answer 1

Turn on the 'paste' option (using :set paste), and then press dd i <CTRL-R> 1 <ESC> on each line that you want to apply the backspaces to. This also works if you delete multiple lines, or even the whole file.

The key here is that you are using <CTRL-R> 1 in insert mode to 'type out' the contents of register 1 (where your deleted lines just got put), and 'paste' option prevents Vim from using any mappings or abbreviations.

Answer 2

I googled this while trying to remember the command I had used before to `apply' backspaces, and then I remembered it: col -b - here is the manpage. (It does a little more and comes from BSD or more exactly AT&T UNIX as the manpage says, so if you are on Linux you may need to install an additional package, on debian its in bsdmainutils.)

Answer 3

Simplistic answer:

:%s/[^^H]^H//g

where ^^H is:

1. Literal ^ character
2. Ctrl-V Ctrl-H

and repeat it couple of times (until vim will tell you that no substitutions have been made

If you want without repetition, and you don't mind using %!perl:

%!perl -0pe 's{([^\x08]+)(\x08+)}{substr$1,0,-length$2}eg'

All characters are literal - i.e. you don't have to do ctrl-v ... anywhere in above line.

Should work in most cases.

Answer 4

All right, here is a bare-metal solution.

Copy this code into a file named crush.c:

#include <stdio.h>

// crush out x^H sequences
// there was a program that did this, once
// cja, 16 nov 09

main()
{
    int c, lc = 0;

    while ((c = getchar()) != EOF) {
        if (c == '\x08')
                lc = '\0';
        else {
                if (lc)
                        putchar(lc);
                lc = c;
        }
    }
    if (lc)
        putchar(lc);
}

Compile this code with your favorite compiler:

gcc crush.c -o crush

Then use it like this to crush out those bothersome sequences:

./crush <infilename >outfilename

Or use it in a pipeline ("say" is a speech-to-text app on the Mac)

 man date | ./crush | say

You can copy crush to your favorite executable directory (/usr/local/bin, or some such) and then reference it as follows

  man date | crush | say

Answer 5

Just delete all occurrences of .^H (where . is the regex interpretation of .):

:s/.^H//g

(insert ^H literally by entering Ctrl-V Ctrl-H)

That will apply to the current line. Use whatever range you want if you want to apply it to other lines.

Once you done one :s... command, you can repeat on another line by just typing :sg (you need to g on the end to re-apply to all occurrences on the current line).

Answer 6

How about the following function? I've used \%x08 instead of ^H as it's easier to copy and paste the resulting code. You could type it in and use Ctrl-V Ctrl-H if you prefer, but I thought \%x08 might be easier. This also attempts to handle backspaces at the start of the line (it just deletes them).

" Define a command to make it easier to use (default range is whole file)
command! -range=% ApplyBackspaces <line1>,<line2>call ApplyBackspaces()

" Function that does the work
function! ApplyBackspaces() range
    " For each line in the selected lines
    for index in range(a:firstline, a:lastline)

        " Get the line as a string
        let thisline = getline(index)

        " Remove backspaces at the start of the line
        let thisline = substitute(thisline, '^\%x08*', '', '')

        " Repeatedly apply backspaces until there are none left
        while thisline =~ '.\%x08'
            " Substitute any character followed by backspace with nothing
            let thisline = substitute(thisline, '.\%x08', '', 'g')
        endwhile

        " Remove any backspaces left at the start of the line
        let thisline = substitute(thisline, '^\%x08*', '', '')

        " Write the line back
        call setline(index, thisline)
    endfor
endfunction

Use with:

" Whole file:
:ApplyBackspaces
" Whole file (explicitly requested):
:%ApplyBackspaces
" Visual range:
:'<,'>ApplyBackspaces

For more information, see:

:help command
:help command-range
:help function
:help function-range-example
:help substitute()
:help =~
:help \%x

Edit

Note that if you want to work on a single line, you could do something like this:

" Define the command to default to the current line rather than the whole file
command! -range ApplyBackspaces <line1>,<line2>call ApplyBackspaces()
" Create a mapping so that pressing ,b in normal mode deals with the current line
nmap ,b :ApplyBackspaces<CR>

or you could just do:

nmap ,b :.ApplyBackspaces<CR>

Answer 7

Here's a Bash-based filter you can use to process the whole file:

#!/bin/bash
while read LINE; do
  while [[ "$LINE" =~ '^H' ]]; do
    LINE="${LINE/[^^H]^H/}"
  done  
  echo "$LINE"
done

Note that where ^H appears, it is entered into vim using CTRL-v CTRL-h, and the ^^H is entered as SHIFT-6 CTRL-v CTRL-h.

Answer 8

Here's a much faster Awk filter that does the same:

#!/usr/bin/awk -f
function crushify(data) {
  while (data ~ /[^^H]^H/) {
      gsub(/[^^H]^H/, "", data) 
  }                                                     
  print data
}

crushify($0)

Note that where ^^H appears, the first caret in ^^H is a caret (shift-6) and the second caret with H is entered (into vim) by typing CTRL-v CTRL-H