Delete last 162 lines in a series of files, rename files within forloop

Question

I want to delete last 162 lines from a series of files with bash scripting.

for i in RMSF_CA_proA_*; do | tac $i | sed '1,162d' | tac >> tmp && mv tmp $i

This gives me error

bash: tac: command not found.

I also want to rename the files sequentially within the for loop.

Carlo Corradini · Accepted Answer · 2023-01-22T16:37:43.487

This should work:

#!/usr/bin/env sh

# Fail on error
set -o errexit
# Enable wildcard character expansion
set +o noglob

# ================
# CONFIGURATION
# ================
# File pattern
FILE_PATTERN='RMSF_CA_proA_'
# Lines to delete
N_LINES=162

# ================
# MAIN
# ================
{
  # Check command 'sed' exists
  command -v sed > /dev/null 2>&1 || {
    printf "Command 'sed' not found\n" >&2
    exit 1
  }
  # Check command 'mv' exists
  command -v mv > /dev/null 2>&1 || {
    printf "Command 'mv' not found\n" >&2
    exit 1
  }

  index=0
  for file in "$FILE_PATTERN"*; do
    # Skip if not file
    [ -f "$file" ] || continue

    printf "Analyzing file '%s'\n" "$file"

    # https://stackoverflow.com/a/13383331/6676781
    # Delete last 'N_LINES' from 'file'
    sed \
      -i \
      -e :a \
      -e "\$d;N;2,${N_LINES}ba" \
      -e 'P;D' \
      "$file"

    # Rename 'file' to 'index'
    printf "Renaming file from '%s' to '%s'\n" "$file" "$index"
    mv "$file" "$index"

    # Increment index
    index=$((index = index + 1))
  done
}

Adjust the configuration parameters to your preference.
Using sed, remove the last $N_LINES lines from $file (see this). The change is made in place, so there is no need to save the output and then modify the file.

On Mac OS X sed command does not accept empty -i option. Therefore, add '' after -i (see this):

sed \
  -i '' \

I edited the code as follows: FILE_PATTERN='RMSF_CA_proA_' Sfx='_I.dat' and then in the end: printf "Renaming file from '%s' to '%s'\n" "$file" "$FILE_PATTERN$index$Sfx" mv "$file" "$FILE_PATTERN$index$Sfx" The renaming is not synchronized accordingly: lcbc@MacBook:~/data_analysis/E1-E1_100Aenm/charm36/Umb_5kcal_mol-1_3/analysis/proA~>sh rename_RMSF.sh Analyzing file 'RMSF_CA_proA_1.dat' Renaming file from 'RMSF_CA_proA_1.dat' to 'RMSF_CA_proA_1_I.dat' Analyzing file 'RMSF_CA_proA_10.dat' Renaming file from 'RMSF_CA_proA_10.dat' to 'RMSF_CA_proA_2_I.dat — Prathit, Jan 23 '23 at 13:51

score 1 · Answer 2 · answered Jan 22 '23 at 17:15

tac is neither standard nor necessary. When you want to edit a file, use a file editor like ed, rather than its derivative sed (which is used to edit streams).

for i in RMSF_CA_proA_*; do
    printf '%s\n' '$' '-161,$d' 'w' | ed -s "$i" > /dev/null
done

ed reads its commands from standard input, one command per line. The $ command makes the last line of the file the current line. The -161,$d deletes the desired lines, with the range extending from the 161st line before the current line up to and including the current line. w saves the changes before exiting.

With the caution to try this on a copy of the biggest file first. Older versions of `ed` could not deal with GB+ files. Good luck to all. — shellter, Jan 22 '23 at 18:07

score 0 · Answer 3 · answered Jan 23 '23 at 09:45

0

This might work for you (GNU sed):

sed -i ':a;$d;N;s/\n/&/162;Ta;P;D' RMSF_CA_proA_*

Create a window of 162 lines and at end of file delete it.

answered Jan 23 '23 at 09:45

potong

55,640
6
51
83

Delete last 162 lines in a series of files, rename files within forloop

3 Answers3