Take 1000 using LINQ

Question

Ok, my linq skills aren't great so im trying to do the following.

Say I have 6000 records (email address) I want to add the first 1000 add to bcc, send, take the next 1000 add to bcc, send, take the 1000 add to bcc, send, etc....

Ive started wrinting,

string[] files = System.IO.Directory.GetFiles(@"C:\Mail\ ", "*.csv");

Parallel.ForEach(files, currentFile =>
{
    string filename = currentFile;
    StreamReader reader = new StreamReader(filename);
    var emailList = new List<String>();
    while (reader.Peek() >= 0)
    {
        emailList.Add(reader.ReadLine());
    }
    \\Here is where I need to do the linq?
    IEnumerable<string> list = emailList

    var message = new System.Net.Mail.MailMessage();
    foreach (var s in list)
    {
        MailAddress mailAddress = new MailAddress(s);
        message.Bcc.Add(mailAddress);
    }

    message.Subject = txtSubject.Text;
    message.From = new System.Net.Mail.MailAddress(txtFrom.Text);
    message.Body = txtMessage.Text;
    message.IsBodyHtml = true;
    var smtp = new System.Net.Mail.SmtpClient();

    smtp.Send(message);

you don't need Linq for this at all - move your email sending code into a separate method and call that after you have enough addresses in your initial while loop — BrokenGlass, Oct 14 '13 at 14:30
@BrokenGlass You never *need* LINQ. It may or may not be an acceptable tool to use to solve the problem. — Servy, Oct 14 '13 at 14:31

Servy · Answer 1 · 2013-10-14T14:39:53.167

6

You seem to be looking for a Batch method. Here is one implementation:

public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int batchSize)
{
    List<T> buffer = new List<T>(batchSize);

    foreach (T item in source)
    {
        buffer.Add(item);

        if (buffer.Count >= batchSize)
        {
            yield return buffer;
            buffer = new List<T>(batchSize);
        }
    }
    if (buffer.Count > 0)
    {
        yield return buffer;
    }
}

You can now write:

foreach(var batch in emailList.Batch(1000))
{
    var message = CreateMessage();
    foreach(var email in batch)
    {
        message.Bcc.Add(email);
    }
    message.Send();
}

On a side note, rather than using a StreamReader to read the file's lines, you can use File.ReadLines to get an IEnumerable<string> representing the lines in the file. It even has the side benefit of streaming the data rather than pulling all of the lines into memory first.

edited Oct 14 '13 at 14:39

answered Oct 14 '13 at 14:33

Servy

202,030
26
332
449

This way, you would be iterating over all e-mail addresses twice. BrokenGlass' implementation only iterates once. Although technically this answers how to achieve this in Linq. – CodingIntrigue Oct 14 '13 at 14:35
@RGraham See the edit at the end; that's a result of the OP using an undesirable mechanism to read the file into memory, which is easily resolved. – Servy Oct 14 '13 at 14:36
Doesn't `ReadLines` use StreamReader internally? I don't think StreamReader puts everything into memory, does it? And can you provide a source because I've been using it in a lot of cases I really shouldn't have!! – CodingIntrigue Oct 14 '13 at 14:42
2

@RGraham `ReadLines` streams the data yes; it fetches the next line when the enumerator asks for it, rather than all up front. The OP's code, despite using a `StreamReader`, doesn't stream the data; instead it uses the stream reader to just pull all of it into a list, thus preventing streaming the data. – Servy Oct 14 '13 at 14:44

score 1 · Answer 2 · answered Oct 14 '13 at 14:33

1

Move your email sending code into a separate method and call that after you have enough addresses in your initial while loop:

var emailList = new List<String>();
while (reader.Peek() >= 0)
{
    emailList.Add(reader.ReadLine());
    if(emailList.Count == 1000)
    {
        SendEmails(emailList);
        emailList = new List<String>();
    }
}
//send the rest, if any
if(emailList.Any()) SendEmails(emailList);

This is much cleaner imo anyway since your email sending method is logically separated from the way you acquire the email addresses.

answered Oct 14 '13 at 14:33

BrokenGlass

158,293
28
286
335

`reader.Peek()` is a real performance killer, `string line; while((line = reader.ReadLine()) != null) { emailList.Add(line);` is a much better way of doing it. – Scott Chamberlain Oct 14 '13 at 14:44
1

@ScottChamberlain Better still is to just `foreach(var line in File.ReadLines(filename))` – Servy Oct 14 '13 at 14:45

score -1 · Answer 3 · edited May 23 '17 at 12:09

-1

If you want to do it with linq I believe this answer will do exactly what you are looking for, modified below where you send in group = 1000

public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> source, int group)
{
    return  source
        .Select((x, i) => new { Index = i, Value = x })
        .GroupBy(x => x.Index / group)
        .Select(x => x.Select(v => v.Value).ToList())
        .ToList();
}

Should This be a comment and not an answer. Please let me know and I will delete and change if that is the case

edited May 23 '17 at 12:09

Community

1
1

answered Oct 14 '13 at 14:39

Harrison

3,843
7
22
49

Please explain the -1, thank you. I can update my answer based on comments not on -1. – Harrison Oct 14 '13 at 17:17

Take 1000 using LINQ

3 Answers3