我有一个使用ThreadPoolExecutor
的线程应用程序,我想从每个线程的内部创建n个线程。这样做是个好主意吗?它将如何影响我的资源(如内存和CPU利用率)?我尝试过,但在Google上找不到相关信息。
from concurrent import futures
from concurrent.futures.thread import ThreadPoolExecutor
import csv
import time
def casea():
pools = {}
for i in range(5000):
with ThreadPoolExecutor(max_workers=12) as executor:
pools[executor.submit(middlemen)] = i
def caseb():
pools = {}
for i in range(5001,10000):
with ThreadPoolExecutor(max_workers=12) as executor:
pools[executor.submit(writecsv, i)] = i
def middlemen():
for k in range(5000):
with ThreadPoolExecutor(max_workers=30) as executor:
pools[executor.submit(writecsv, k)] = k
def writecsv(i):
nums = dict(zip(range(10000), range(10000)))
filename = "test_{}.csv".format(i)
fp = open(filename, 'w+')
writer = csv.writer(fp, delimiter=',')
writer.writerows(nums)
writer.close()
fp.close()
if __name__ == '__main__':
t1 = time.time()
casea()
t2 = time.time()
print "Time taken CASE A: {}".format(t2-t1)
t3 = time.time()
caseb()
t4 = time.time()
print "Time taken CASE B: {}".format(t4-t3)
# Time taken CASE A: 9.74694705009
# Time taken CASE B: 35.9461958408
我使用上面的代码进行了一些基准测试,似乎在线程下使用了线程,性能提高了4倍