3

Using the standard procedure (for loop) as seen in the examples we found that there is a lot of memory used on the machine. I.e. it seems that all the messages in the queue are loaded into memory and also acknowledged. This approach is here. I assumed the queue would be a generator.

import rabbitpy

with rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2f') as conn:
    with conn.channel() as channel:
        queue = rabbitpy.Queue(channel, 'example')

        # Exit on CTRL-C
        try:
            # Consume the message
            for message in queue:
                message.pprint(True)
                message.ack()

        except KeyboardInterrupt:
            print 'Exited consumer'

Now if we go to the message getter, using one message at a time via .get(), it appears to be less memory hungry. However my code (using an infinite loop) to ensure our consumer has be run when there are no messages in the queue - works better, however in the web control panel, there seems to be no consumer displayed, i.e. somehow this process is not recognized as a consumer - how can we fix this, so that on the web panel we detect a consumer?

queue_read = rabbitpy.Queue(channel, QUEUE_NAME)
while True: 
        body = queue_read.get() ## pop one at a time.
        if body == None:
                time.sleep(3)               
                continue
        body.ack()
        print " [x]  OK - got a message"
disruptive
  • 5,687
  • 15
  • 71
  • 135

1 Answers1

1

It looks like what you need is to set the prefetch property when consuming the queue. Instead of:

for message in queue:
    # handle message

Try instead:

for message in queue.consume(prefetch=5): # or some other reasonable value
    # handle message

The __iter__ method on the Queue class is just a simple call to the consume method itself.

The consume method sets the channel to a consuming state, while the get method merely pulls a message out of the channel and has no idea if you are going to continue consuming in the future. Which is why you aren' t seeing your process in the web panel.

Daniel Timberlake
  • 1,179
  • 6
  • 15