当前位置: 动力学知识库 > 问答 > 编程问答 >

ruby - Limiting concurrent threads

问题描述:

I'm using threads in a program that uploads files over sftp. The number of files that could be upload can potentially be very large or very small. I'd like to be able to have 5 or less simultaneous uploads, and if there's more have them wait. My understanding is usually a conditional variable would be used for this, but it looks to me like that would only allow for 1 thread at a time.

cv = ConditionVariable.new

t2 = Thread.new {

mutex.synchronize {

cv.wait(mutex)

upload(file)

cv.signal

}

}

I think that should tell it to wait for the cv to be available the release it when done. My question is how can I do this allowing more than 1 at a time while still limiting the number?

edit: I'm using Ruby 1.8.7 on Windows from the 1 click installer

网友答案:

Use a ThreadPool instead. See Deadlock in ThreadPool (the accepted answer, specifically).

网友答案:

A word of caution -- there is no real concurrency in Ruby unless you are using JRuby. Also, exception in thread will freeze main loop unless you are in debug mode.

require "thread"

POOL_SIZE = 5

items_to_process = (0..100).to_a

message_queue = Queue.new

start_thread = 
  lambda do
    Thread.new(items_to_process.shift) do |i|
      puts "Processing #{i}"
      message_queue.push(:done)
    end
  end

items_left = items_to_process.length

[items_left, POOL_SIZE].min.times do
  start_thread[]
end

while items_left > 0 
  message_queue.pop
  items_left -= 1
  start_thread[] unless items_left < POOL_SIZE
end
分享给朋友:
您可能感兴趣的文章:
随机阅读: