How to efficiently remove last line in a file and append new text for large file?

Question

What is the efficient way of replacing last line in file?

I am expecting my file to be ending with "]" and I would like to replace "]" with new data that will end again with "]".

This is only example a file will be really large ...

Old:
[
a
b
c
]

new:

[
a
b
c
d
e
]

I have full control of how files are written, created.

EDIT:

    // mockData.json
    //[
    //    a
    //    b
    //    c
    //]

    var fileName = "mockData.json";
    var origData = File.ReadAllLines(fileName);
    origData[origData.Length - 1] = " d\n e\n]";
    File.WriteAllLines(fileName, origData);

`I have full control of how files are written, created.` If that's the case, *why* dont you show us how you are doing it, what you have done. — Trevor, Oct 07 '19 at 20:39
I haven't done it properly. 'I have full control' means that if someone suggest specific Encoding it would not be obstacle. I am not getting initial file from third party. — user007, Oct 07 '19 at 20:44
So show us the inefficient way you've done it, then we can go from there. — Sach, Oct 07 '19 at 20:46
I read the the whole file, take out last line and append additional data. I don't have code as I post stackoverflow questions from home, from work I have restricted access. — user007, Oct 07 '19 at 20:48
That sounds like a decent approach. You can use `Insert` or `InsertRange()` as well. — Sach, Oct 07 '19 at 20:51
@user007 thanks for the update, but, we need to be able to reproduce your issue so we can help, currently the *code* doesn't even compile. Where does `toAppend` come from? — Trevor, Oct 07 '19 at 20:59
Added mock code of the version I should be using @Çöđěxěŕ — user007, Oct 07 '19 at 20:59
The code you posted seems fine. Just remember to add the EOF char at the very end! — Rakesh, Oct 07 '19 at 20:59
@Çöđěxěŕ I don't want you to write code for me, code that I've posted gives pattern I am using, not sure why you need exact working version, I am looking for efficient pattern not bug in my code — user007, Oct 07 '19 at 21:01
@user007 if you don't want code then why the question? TBH it's asking for opinions which is off topic as it can generate many answers based on opinions... Would you like instructions typed out or have someone show code and how it's implemented? — Trevor, Oct 07 '19 at 21:05
Possible duplicate of [How to write data at a particular position in c#?](https://stackoverflow.com/questions/8243122/how-to-write-data-at-a-particular-position-in-c) — Damian, Oct 07 '19 at 21:07
Possible duplicate of [Open existing file, append a single line](https://stackoverflow.com/questions/2837020/open-existing-file-append-a-single-line) — Trevor, Oct 07 '19 at 21:08
@Çöđěxěŕ that is the good point, especially when working with files and streams, its too late for me, will edit question properly tomorrow with proper working code — user007, Oct 07 '19 at 21:10
I'm not finding any clear duplicates for appending to a file terminated by an arbitrary token, but this is very similar to wanting to append more elements to an XML or HTML file without rewriting it. [Append XML to file without writing it from scratch?](https://stackoverflow.com/q/45531295/150605) is just such a question, among [others](https://www.google.com/search?q=site%3Astackoverflow.com+c%23+append+XML+file+without+rewriting). — Lance U. Matthews, Oct 07 '19 at 22:28

Kevin Ng · Accepted Answer · 2019-10-07T21:38:03.427

For this code below, the routine is pretty simple since you already know that the last character in your file is a ]. Thus, all you have to do is read the last character of a file and if it is the char ] then you got that file. If that happens, you truncate the last byte from the file and append text to that file. Then you add the char ] to preserve the format. Note that this is for ASCII encoding if your last char is something else that is bigger than a byte then you would have to fix the code a little bit.

using System;
using System.IO;
using System.Text;

public class FSSeek
{
    public static void Main()
    {
        string fileName = "test.txt";
        char lastChar = ']';
        string toBeAppend = "d\ne\n";

        using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.ReadWrite))
        {
            fs.Seek(-1, SeekOrigin.End);
            if ( Convert.ToChar(fs.ReadByte()) == lastChar ){
                fs.SetLength(fs.Length - 1);
                fs.Write(Encoding.ASCII.GetBytes(toBeAppend));
                fs.WriteByte(Convert.ToByte(lastChar));
            }            

        }
    }
}

test.txt content:

[
a
b
c
]

I am using .Net Core, small modification for missing signature: var toBeAppendBytes = Encoding.ASCII.GetBytes(toBeAppend); fs.Write(toBeAppendBytes, 0, toBeAppendBytes.Length);, well done, I started looking into this as someone flagged question as duplicated of writing bytes to specific location — user007, Oct 07 '19 at 21:35

score 0 · Answer 2 · answered Oct 07 '19 at 20:53

0

My suggestion is this :

Read the file, remove last character of content and then write it to the file again.

string fileContent = File.ReadAllText("file.path");
// Remove the last character
fileContent = fileContent.Remove(fileContent.Length - 1);
string newContent = "Your new content";
fileContent += newContent;
// Write content back to the file
File.WriteAllText("file.path", fileContent);

answered Oct 07 '19 at 20:53

Ilia Afzali

429
2
6

1

This looks like it will accomplish the requested task, but I don't think it satisfies the "efficiently" condition of the question. The question states the file to be modified is "really large", and though it's not clear what that means (kilobytes? megabytes? gigabytes?) if we suppose that `file.path` is, say, one gigabyte then I believe this code will end up using a bit more than _three gigabytes_ of memory. Short code, but huge memory usage. – Lance U. Matthews Oct 07 '19 at 22:16

score 0 · Answer 3 · answered Oct 07 '19 at 20:53

0

You can insert a range to a list by InsertRange() like so, and lines.Count - 1 will ensure you're inserting at the line before the last.

var linesToAdd = new List<string>() { "d", "e" };
var lines = File.ReadAllLines(path).ToList();
lines.InsertRange(lines.Count - 1, linesToAdd);
File.WriteAllLines(path, lines);

Obviously this needs error handling, for case such as your original file didn't have a ] at the end.

answered Oct 07 '19 at 20:53

Sach

10,091
8
47
84

Could there be a better way of doing it, I could see there are solutions for reading file backward but encoding should be set properly in order to navigate without errors, If "]" is read I could use that information to replace it with new data ... – user007 Oct 07 '19 at 21:07
Your question isn't well defined, and you refuse to share proper code. What defines 'better'? Better depends on the situation, there is not catch-all 'better' in programming. – Sach Oct 07 '19 at 21:09
give you Up, so Up + Down = 0 :), as you actually posted almost the exact version as the one I got – user007 Oct 07 '19 at 21:38
I didn't downvote, but the question asks for an "efficient way" to insert new lines before the terminating `]`. Storing the entire file in memory (twice actually: first with `ReadAllLines()` and then again with `ToList()`), as this code does, is not efficient in my opinion, although, to be fair, the question does not define in what way "efficient" is measured nor how large this "large file" is. – Lance U. Matthews Oct 07 '19 at 22:06

How to efficiently remove last line in a file and append new text for large file?

3 Answers3