Multiple thread to process contents of List

Question

SomeClass {
    while(itr.hasNext() {    
       if (MDS.contains(subj)) {

       } else {
          listOfLme.add(new LME(subj.getName(), 
          promptBuilder.toString(), cobBuilder.toString(), openInt));
       }
    }      
} //end class

Hi, I have a loop that adds LME object to a LinkedList. As LME objects are being added to the List, I need multiple threads iterating over what's already in the List and add those objects to database. I have a class that implements Runnable with constructor parameter LME object. Can someone please show me a simple technique how i can use two threads two iterate over listOfLme. If this is possible with an Executor, then I would like to see how this can be achieved.

I don't understand what you try to achieve but a `LinkedList` in a multithreaded environment sounds like a terrible idea. http://stackoverflow.com/questions/2332537/producer-consumer-threads-using-a-queue is maybe a better idea — zapl, Jan 26 '14 at 20:57
@zapl `LinkedList` isn't thread safe: http://docs.oracle.com/javase/7/docs/api/java/util/LinkedList.html: "Note that this implementation is not synchronized." — Mark Elliot, Jan 26 '14 at 21:07

Mark Elliot · Accepted Answer · 2014-01-26T21:08:18.587

2

A standard model for this would use a BlockingQueue instead of just a standard List implementation. BlockingQueues are thread-safe by definition and allow you to add and take from a queue safely.

Instead of having a List<LME> listOfLme, you'd instead create some global state, say:

BlockingQueue<LME> queueOfLme = new ArrayBlockingQueue<LME>(100);

Unlike with your current List, where you add elements, you should call one of:

put(): add to the queue waiting until there is space in the queue for this item
offer(): time-bounded put() (returns false if unsuccessful)

Create a Runnable implementation to process your items:

class Worker implements Runnable {
    public void run() {
        while (true) {
            LME item = null;
            try {
                item = queueOfLme.take()
            } catch (InterruptedException e) {
                break;
            }
            /* do work */
        }
    }
}

Like with adding to the queue, there's some options for pulling items:

take(): returns the next item from the queue or waits indefinitely until one is available
poll(): returns the next item from the queue or waits for the specified interval

Then, create a pool of workers:

int poolSize = 10;
ExecutorService pool = Executors.newFixedThreadPool(poolSize);
for (int i = 0; i < poolSize; i++) {
    pool.submit(new Worker());
}

edited Jan 26 '14 at 21:08

answered Jan 26 '14 at 21:02

Mark Elliot

75,278
22
140
160

I really like this. two questions. 1) can it take same LME item more than once ? 2) where does it make sense to create a pool of worker in my case ? should i wait for all items to be added to BlockingQueue? – M06H Jan 26 '14 at 21:20
(1) a particular instance of LME can only be taken from the queue once; (2) when you want to have multiple threads, say one for adding to the queue and 1 or more processing items in the queue; (3) no, you should start the workers, then add items to the queue (so that the items will be processed as they are added) – Mark Elliot Jan 26 '14 at 21:22
thanks, re: (3) I have a callback method within which I receive updates to subject i've subscribed to. and it is these subject which i add to blockingqueue on retrieval in the callback method. i'm not sure where to write pool.submit(new Worker()); – M06H Jan 26 '14 at 21:29
@user2781389 when your program starts (when you initialize your connection to the topic, say). – Mark Elliot Jan 26 '14 at 21:38

score 0 · Answer 2 · answered Jan 26 '14 at 20:59

Well, you would need the main program and another class implementing Thread or Runnable. The main program would add the LME objects and the thread would iterate over the list and add them to the DB.

Here's how to create the thread: http://docs.oracle.com/javase/tutorial/essential/concurrency/runthread.html

To iterate over the List, either use iterators, for's, or whatever you wish. You can have the secondary thread running accross the list endlessly or you can set it to sleep() and when the main thread adds something to the list, it wakes up the secondary thread so that one adds the new item to the DB whilst the main thread continues it's job.

Multiple thread to process contents of List

2 Answers2