1

I am trying to work out how I can add the contents of a set of files1 defined by

ls /home/dir1/*-split.csv.*

And then add the contents of each file from files1 into another set of files e.g. files2 defined by

ls /home/dir2/*.py.*.py

Both sets of files1 and files2 will always have the same number of files in them.

I have managed to get this work for 1 file using 

files1=($home/dir1/'*'-split.csv.'*')
grep -h '^[^\s]*' $files1 | xargs -I '{}' sed -i  s'/userid/{}/' /home/dir2/*.py.*.py

But I don't know how to turn this into a loop so that the files from files1 are inserted into the files in files2.

As above, the sed statement looks to replace 'userid' in files2 which are located at /home/dir2/*.py.*.py with the contents of the file in files1 located at /home/dir1/*-split.csv.*.

All the files in files2 have 'userid' in them and this is the string to replace each time with the contents of a file from files1.

Does anyone have any ideas? Thanks!

Edit

Hi guys, thank you for your comments and pushing me for a better explanation!

Hopefully this sheds some more light on things... list of files like this...

ls /home/dir1/*-split.csv.*
just-users-split.csv.aa
just-users-split.csv.ab
just-users-split.csv.ac
just-users-split.csv.ad

Each file contains a single line of text e.g.

cat *.aa
"userPhD","userdegree","user123",

cat *.ab
"userxyz","userabc","user456"

In files2, I have a corresponding list of 4 files e.g.

ls /home/dir2/*.py.*.py
twud.py.1.py
twud.py.2.py
twud.py.3.py
twud.py.4.py

All of the *.py.*.py files are a copy of each other and contain a 91 line python script.

I then want to insert the contents of

just-users-split.csv.aa into twud.py.1.py

just-users-split.csv.ab into twud.py.2.py

just-users-split.csv.ac into twud.py.3.py

just-users-split.csv.ad into twud.py.4.py

The inserting can either be done at the top or bottom of the *.py file, or in an ideal world, the contents of just-users-split.csv.aa would be inserted on line 33 to replace the current string ids = "userid".

Finally, in this example there are 4 files to loop through but in other situations, it may be that there are 5, 10 or 20 pairs of files to loop through.

Does this make any more sense?

Shakiba Moshiri
  • 21,040
  • 2
  • 34
  • 44
Bruce
  • 29
  • 5
  • I find the description of your problem rather confusing. – Michael Vehrs Mar 17 '17 at 10:48
  • Hi Michael, I'm just trying to insert the contents of one set of files into another set but I need to do this for multiple files - does this help? – Bruce Mar 17 '17 at 10:52
  • 1
    For me it's still quite confusing. Better if you show some input files with a few lines each and what you expect the output to be. Just show the minimum possible, but enough so that an answer to the abbreviated problem could be extended by you to the general case. (For example, maybe it would be enough to show two files from `files1` and two from `files2` with just one line of content each.) – jas Mar 17 '17 at 12:37
  • Part of my confusion is, how do you know which files from files1 to put into which files within files2?? – grail Mar 17 '17 at 13:07
  • 1
    How are you matching files? do you want all content from first list of files to be inserted to all of the second list of files? Or, is it one-to-one? which requires a mapping. – karakfa Mar 17 '17 at 17:40
  • In `files1` if I run `ls /home/dir1/*-split.csv.*` I get just-users-split.csv.aa just-users-split.csv.ab etc. Each file contains a single line of text e.g. `cat *.aa` `"jennyarntzenPhD","chambda","SlickTrilly",` In files2, I have a corresponding list of 4 files e.g. `ls /home/dir2/*.py.*.py` `twud.py.1.py` `twud.py.2.py` etc. The *.py.*.py files are a 91 line copy of each other I then want to insert the contents of the 4 files just-users-split.csv.aa into twud.py.1.py just-users-split.csv.ab into twud.py.2.py etc. I might have 10 or 20 pairs of files to loop – Bruce Mar 20 '17 at 06:32
  • @jas , I've added copies of the files from `files1` and `files2` into the question @grail , hopefully the updated question answers how to identify the files from `files1` and `files2` @karakfa it's basically a one-to-one matching e.g `just-users-split.csv.aa` into `twud.py.1.py` and `just-users-split.csv.ab` into `twud.py.2.py` etc. – Bruce Mar 20 '17 at 06:39

1 Answers1

0

You can do the following:

#!/bin/bash

files1=($(ls /home/dir1/*-split.csv.*))
files2=($(ls /home/dir2/*.py.*.py))

for (( i = 0; i < ${#files1[@]}; i++ )); do
  content=($(sed -n 1p ${files1[$i]}))     
  sed -i "33s/.*/ids = $content/" ${files2[$i]} 
done

The variable i in the for loop is in the range [0,n), n being the size of files1/files2 arrays.

In every round of the for loop, the variable content gets assigned to the first line of the file in dir1. This variable is then used to replace line 33 of maching file's content within dir2.

For example:

$ cat -n /home/dir1/just-users-split.csv.ab | sed -n '1p'
     1  "userPhD","userdegree","user123"
$ cat -n /home/dir2/twud.py.1.py | sed -n '33p'
    33  ids = "userid"
$ ./script
$ cat -n /home/dir2/twud.py.1.py | sed -n '33p'
    33  ids = "userPhD","userdegree","user123"
assefamaru
  • 2,751
  • 2
  • 10
  • 14