22

I was just thinking of writing a shell script to implement the obliterate functionality in an easy to do way (externally, using the suggested way, but automated).

Here's what I had in mind:

On the client

  1. svn list -R > file-list.
  2. filter file-list in several ways like grep to create a file "files-to-delete", something like a set of grep XXX file-list>>files-to-delete.
  3. transfer files-to-delete to the server using scp.

On the server

  1. Dump the repository svnadmin dump /path/to/repos > repos-dumpfile, this can be kept as a backup too.
  2. Filter the dump file, for each word in "files-to-delete", do: cat repos-dumpfile | svndumpfilter exclude $file > new-dumpfile
  3. Create a new repository and load the new file to it svnadmin create new-name; svnadmin load new-name < new-dumpfile

Would this work? How can it fail? Any other ideas?

Joachim Sauer
  • 302,674
  • 57
  • 556
  • 614
Osama Al-Maadeed
  • 5,654
  • 5
  • 28
  • 48
  • 4
    As noted below, cat new-dumpfile | svndumpfilter exclude $file > new-dumpfile will not work, as it will clobber new-dumpfile. Never use the same file for reading and writing (should be obvious...). – sleske Jul 30 '09 at 07:58

4 Answers4

7

Yes, that script would work. But usually you don't obliterate that many files. Usually obliterate is only needed if you commit confidential information accidentally.

Are you sure you want to use obliterate for so many files?

Stefan
  • 43,293
  • 10
  • 75
  • 117
  • 1
    One example reason to obliterate a tree may be if one checked in a third party component only to decide later that the license is unacceptable for whatever reason... – i_am_jorf May 17 '12 at 22:12
6

I think cat new-dumpfile | svndumpfilter exclude $file > new-dumpfile is a dangerous example. new-dumpfile will not be completely processed and it's contents will be probably lost, no?

From the comments below: the new-dumpfile will surely be lost, because the shell will clobber (truncate to zero length) it even before starting up the command.

EFraim
  • 12,811
  • 4
  • 46
  • 62
mark
  • 6,308
  • 8
  • 46
  • 57
  • why do you say "will not be completely processed and it's contents will be probably lost"? – Osama Al-Maadeed Mar 20 '09 at 20:52
  • 2
    I think that, with a large file, reading and simultaneously writing to it will not work. I guess even for small files, no? "cat file | do_something > file" looks like it will overwrite itself which can happen before being completely processed. Methinks. – mark Mar 21 '09 at 08:25
  • Yes, definitely, this will clobber new-dumpfile. It doesn't matter how big it is: The shell will set up the output redirection to new-dumpfile before even starting svndumpfilter, and will truncate new-dumpfile. – sleske Jul 30 '09 at 07:56
  • I think, perhaps, it was a typo and he just meant 'cat dumpfile ... $file > new-dumpfile' otherwise why is called *new-dumpfile* and not just *dumpfile*? – bias Jan 11 '10 at 19:38
  • @EFraim: wait, you downvote my answer because I'm pointing at the obvious data corruption? – mark Sep 22 '10 at 21:44
4

I had a similar but slightly more complex requirement. Several hundred revisions in the past, some very large (>1GB) sample data files were committed to the repository. They were then moved around and eventually deleted from HEAD. However they were still in revision history, making the repository cumbersomely large. I could not use svn list -R, since the files no longer appeared in the working copy.

However, svn list can be given a revision argument. I wasn't sure exactly when the big files had been checked in, but I knew it was sometime after revision 2000. I also had a list of file names. So I used a simple loop and uniq to generate my files-to-delete:

cd $working_copy
for rev in {2000..2437}; do
    svn ls -R -r$rev | grep -f ~/tmp/big-file-names >> ~/tmp/file-paths;
done
cat ~/tmp/file-paths | sort | uniq > ~/tmp/files-to-delete
cd ~/tmp
# You should inspect "files-to-delete" to see if it looks reasonable!
cat dumpfile | svndumpfilter exclude `cat files-to-delete` > dumpfile.new
Jason McVetta
  • 1,383
  • 11
  • 11
0

What about files with the same path in different revisions? For example, if you commit /trunk/foo, then rename it to /trunk/bar, then commit something else at /trunk/foo that you want to obliterate. You don't want to lose the history of what's now /trunk/bar. Maybe svndumpfilter supports peg revisions?

Matt McHenry
  • 20,009
  • 8
  • 65
  • 64