asynchronous http request in ruby

Question

 require 'net/http'

urls = [
  {'link' => 'http://www.google.com/'},
  {'link' => 'http://www.facebook.com/'},
 {'link' => 'http://www.yahoo.com/'}
]

urls.each do |u|
  u['content'] = Net::HTTP.get( URI.parse(u['link']) )
end

print urls

This will work as procedural code.. I just want to hit a server, no issues about the order. How can i do that in ruby. One option is using threads.

Here's an example using threads.

require 'net/http'

urls = [
  {'link' => 'http://www.google.com/'},
  {'link' => 'http://www.facebook.com/'},
  {'link' => 'http://www.yahoo.com/'}
]

urls.each do |u|
  Thread.new do
    u['content'] = Net::HTTP.get( URI.parse(u['link']) )
    puts "Successfully requested #{u['link']}"

    if urls.all? {|u| u.has_key?("content") }
      puts "Fetched all urls!"
      exit
    end
  end
end

Any better solution..??

PS:- i want to hit mixpanel, so that's why I just want to make a http call and dont wait for the response.

score 15 · Accepted Answer · edited Dec 22 '18 at 09:22

15

Lightweight Async handling is the job of Threads (as you said) or Fibers.

Otherwise, you should consider EventMachine which is a very powerful tool.

EDIT: The above URL for Event Machine is dead. Here is their GitHub account, https://github.com/eventmachine/eventmachine . It serves as a good starting point.

edited Dec 22 '18 at 09:22

snowYetis

1,517
16
28

answered Jul 10 '11 at 22:03

apneadiving

114,565
26
219
213

rails, question is tagged with rails 3:) – Mohit Jain Jul 10 '11 at 22:06
RIP EventMachine (at least the posted URL) – snowYetis Dec 22 '18 at 05:12

score 12 · Answer 2 · answered Jul 10 '11 at 22:12

12

Here is a great article covering the topic.

Generally, viable alternatives to using threads for this would be the use of a Fiber or you could use em-http-request. In the latter example you could leave out the callback handling for your particular purpose.

answered Jul 10 '11 at 22:12

emboss

38,880
7
101
108

It's misleading to say that fibers are an alternative to threads. Fibers will not give you concurrency. – pguardiario Aug 26 '12 at 04:05
3

@pguardiario - Fibers absolutely will give you concurrency ... on any IO-bound task. You are correct that for CPU-bound tasks, you want multiple threads, but when most people are concerned about "concurrency" it's usually for parallel network requests with only minimal processing in between. – Dave Dopson Dec 18 '12 at 19:23

score 2 · Answer 3 · answered Mar 01 '16 at 03:18

If its just about plain http requests in async style, probably Unirest is the best fit to achieve it.

Asnc request is as simple as:

response = Unirest.post "http://httpbin.org/post", 
                    headers:{ "Accept" => "application/json" }, 
                    parameters:{ :age => 23, :foo => "bar" } {|response|
response.code # Status code
response.headers # Response headers
response.body # Parsed body
response.raw_body # Unparsed body
}

asynchronous http request in ruby

3 Answers3

Linked