队列按顺序非并行运行的python多线程处理

这个管用!

start = time.time()
def f(m):
    time.sleep(0.5)
    print("yes")
    return 'yes'


def do_stuff(q):
    while True:
        output = q.get()
        final_result = f(output)
        q.task_done()


q = Queue(maxsize=0)
for message_nbr in range(10):
#     q.put(f())
    q.put(message_nbr)

num_threads = 10

for i in range(num_threads):
    worker = Thread(target=do_stuff, args=(q, ))
    worker.setDaemon(True)
    worker.start()

q.join()
print("time: ", time.time() - start)

答案就在这里:

for message_nbr in range(40):
    q.put(f())

您正在将40个None实例放入队列,因为您正在调用 f() 它不返回而不是传递 f (函数对象)中。这个街区要跑20秒!

更改此代码

def do_stuff(q):
    while True:
        output = q.get()
        q.task_done()

为了这个

def do_stuff(q):
    while True:
        output = q.get()
        output()
        q.task_done()

也是必需的(您需要调用函数!)

决赛:

import time
from queue import Queue
from threading import Thread
start = time.time()
def f():
    time.sleep(0.5)
    print("yes")
    return 'yes'


def do_stuff(q):
    while True:
        output = q.get()
        output()
        q.task_done()


q = Queue(maxsize=100)
for message_nbr in range(40):
    q.put(f)

num_threads = 10

for i in range(num_threads):
    worker = Thread(target=do_stuff, args=(q, ))
    worker.setDaemon(True)
    worker.start()

q.join()
print("time: ", time.time() - start)  # time:  2.183439254760742