从python中的线程内部创建线程的性能影响

问题描述 投票:0回答:1

我有一个使用ThreadPoolExecutor的线程应用程序,我想从每个线程的内部创建n个线程。这样做是个好主意吗?它将如何影响我的资源(如内存和CPU利用率)?我尝试过,但在Google上找不到相关信息。

multithreading python-2.7
1个回答
0
投票
from concurrent import futures
from concurrent.futures.thread import ThreadPoolExecutor
import csv
import time


def casea():
    pools = {}
    for i in range(5000):
        with ThreadPoolExecutor(max_workers=12) as executor:
            pools[executor.submit(middlemen)] = i

def caseb():
    pools = {}
    for i in range(5001,10000):
        with ThreadPoolExecutor(max_workers=12) as executor:
            pools[executor.submit(writecsv, i)] = i


def middlemen():
    for k in range(5000):
        with ThreadPoolExecutor(max_workers=30) as executor:
            pools[executor.submit(writecsv, k)] = k



def writecsv(i):
    nums = dict(zip(range(10000), range(10000)))
    filename = "test_{}.csv".format(i)
    fp = open(filename, 'w+')
    writer = csv.writer(fp, delimiter=',')
    writer.writerows(nums)
    writer.close()
    fp.close()



if __name__ == '__main__':
    t1 = time.time()
    casea()
    t2 = time.time()
    print "Time taken CASE A: {}".format(t2-t1)
    t3 = time.time()
    caseb()
    t4 = time.time()
    print "Time taken CASE B: {}".format(t4-t3)

    # Time taken CASE A: 9.74694705009

    # Time taken CASE B: 35.9461958408

我使用上面的代码进行了一些基准测试,似乎在线程下使用了线程,性能提高了4倍

© www.soinside.com 2019 - 2024. All rights reserved.