使用线程运行函数并存储其返回值以备后用

问题描述 投票:0回答:3

我正在尝试使用一个函数来获取在后台运行的新下载文件的名称,同时我运行另一个程序来下载特定文件。后台函数将有一个新下载文件的返回值,我打算稍后在代码中使用它。我有下面要尝试做的粗略框架。

with concurrent.futures.ThreadPoolExecutor(max_workers = 1) as executor:
     output = executor.submit(downloadedfiledetector, filedirectory)
     filename = output.result()
#Run code that will download a file
print(filename)

在这里,downloadedfiledetector 是返回下载文件目录的函数,filedirectory 是该函数的参数,其中包含该函数将监视的文件路径。我希望将返回值存储在文件名变量中,以便稍后在程序中使用它。然后在下面,我运行更多的代码,基本上从一个网站下载一个文件,该文件的名称每次都会不同(因此需要一个函数来获取新下载文件的名称)。

当我运行上面的代码时,它根本不运行下载文件的代码,只运行downloadedfiledetector函数。任何帮助表示赞赏!

编辑: 我被要求包括一个代码的工作示例。我知道这肯定不是最好的方法,但我自己写的(大部分)而且我对 Python 还是很陌生。在代码的底部,combinessubtitletovideo 函数是我将使用下载的 .srt 文件并将其与视频组合的地方:

import os
import time
from selenium.webdriver.common.by import By
import undetected_chromedriver as bypass
import concurrent.futures

def downloadedfiledetector(scannedfolder, variablefunction):
    seconds = 0
    dl_wait = True
    filelist1 = []
    filelist2 = []
    while dl_wait and seconds < 120:
        for fname in os.listdir(scannedfolder):
            filelist1.append(fname)
            if fname.endswith('.crdownload') and seconds != 0:
                filelist1.pop()
        time.sleep(1)
        for fname in os.listdir(scannedfolder):
            filelist2.append(fname)
            if fname.endswith('.crdownload') and seconds != 0:
                filelist2.pop()
        if seconds != 0 and len(filelist1) != len(filelist2):
            dl_wait = False
        seconds += 1
    if variablefunction == 1:
        newfilename1 = list(set(filelist2) - set(filelist1))
        newfilename2 = ''.join(str(e) for e in newfilename1)
        time.sleep(1)
        driver.close()
        return newfilename2
    elif variablefunction == 2:
        pass

with concurrent.futures.ThreadPoolExecutor(max_workers = 1) as executor:
    output = executor.submit(downloadedfiledetector, cliplocation, 1)
    filenamesubtitle = output.result()
driver = bypass.Chrome(options=options)
driver.get(subtitleurl)
WebDriverWait(driver, 300).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#root > section > section.container.mx-auto.py-6.mt-4.sm\:mt-6 > div.mx-auto.max-w-4xl.leading-normal.mt-6.sm\:mt-10.sm\:flex.justify-center.px-4.lg\:px-0.text-center > div > div > div > input')))
enterurlbox = driver.find_element(By.CSS_SELECTOR, '#root > section > section.container.mx-auto.py-6.mt-4.sm\:mt-6 > div.mx-auto.max-w-4xl.leading-normal.mt-6.sm\:mt-10.sm\:flex.justify-center.px-4.lg\:px-0.text-center > div > div > div > input')
enterurlbox.send_keys(youtubeurl)
WebDriverWait(driver, 300).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#root > section > section.container.mx-auto.py-6.mt-4.sm\:mt-6 > div.mx-auto.max-w-4xl.leading-normal.mt-6.sm\:mt-10.sm\:flex.justify-center.px-4.lg\:px-0.text-center > button')))
downloadbutton = driver.find_element(By.CSS_SELECTOR, '#root > section > section.container.mx-auto.py-6.mt-4.sm\:mt-6 > div.mx-auto.max-w-4xl.leading-normal.mt-6.sm\:mt-10.sm\:flex.justify-center.px-4.lg\:px-0.text-center > button')
downloadbutton.click()
WebDriverWait(driver, 300).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#root > section > main > section.max-w-3xl.mx-auto.md\:flex.md\:justify-between.pt-4 > div.w-full.px-4.lg\:px-0.mb-6.md\:mb-0 > ul > li.py-6.text-center.block.w-full.px-2.sm\:px-4.sm\:py-3.sm\:flex.justify-between.items-center.border-b.border-gray-200.dark\:border-night-500 > section > div > div > a:nth-child(1)')))
srtdownloadbutton = driver.find_element(By.CSS_SELECTOR, '#root > section > main > section.max-w-3xl.mx-auto.md\:flex.md\:justify-between.pt-4 > div.w-full.px-4.lg\:px-0.mb-6.md\:mb-0 > ul > li.py-6.text-center.block.w-full.px-2.sm\:px-4.sm\:py-3.sm\:flex.justify-between.items-center.border-b.border-gray-200.dark\:border-night-500 > section > div > div > a:nth-child(1)')
srtdownloadbutton.click()
time.sleep(2)
combinesubtitletovideo(filenamesubtitle)
python multithreading python-multithreading
3个回答
0
投票

您可以使用从目标函数返回结果的地图功能:

with concurrent.futures.ThreadPoolExecutor(max_workers = 1) as executor:
     for result in executor.map(downloadedfiledetector, filedirectory):
        print(result) #this gives you the required return value from function

0
投票

尝试这样的事情:

import os
import time
from selenium.webdriver.common.by import By
import undetected_chromedriver as bypass
import concurrent.futures

def downloadedfiledetector(scannedfolder, variablefunction):
    pass # Code exactly like your example, omitted for clarity

def subsequent():
    driver = bypass.Chrome(options=options)
    # Your same code...
    srtdownloadbutton.click()
    # Remove time.sleep() at the end, it does nothing

with concurrent.futures.ThreadPoolExecutor(max_workers = 1) as executor:
    output1 = executor.submit(downloadedfiledetector, cliplocation, 1)
    output2 = executor.submit(subsequent)
    filenamesubtitle = output1.result()
    output2.result()

combinesubtitletovideo(filenamesubtitle)

这将在两个不同的线程中并行执行两个函数。主线程等待他们两个;他们完成的顺序无关紧要。一旦两个线程都完成(返回结果)主线程继续。


0
投票

如果有人遇到过这个或有类似的问题,我解决了我自己的问题。当然,我没有花几个月的时间来解决,但我想我会在这里分享我的过程,以防其他人发现它有用。

我不应该使用并发期货库,我应该使用线程库。通过这种方式,代码实际上变得更加简单易懂:

import threading, os, time

output_file = [] #We innitialize this variable as a list. This will contain the output for the threaded function, as there is not an easy way (to my knowledge at least) to get the return value of a function. Thus, by appending the output of the threaded function to this list, we can access that output outside of the threaded function.

def downloaded_file_detector(scanned_folder, variable_function):
    #input code for detecting the downloaded file. The psuedocode is as follows:
    global output_file #This part may or may not be necessary
    
    while True:
        before_folder_contents = os.listdir(scanned_folder) #create a list of all filenames
        time.sleep(1) #allow for the file download to take place within the interval of 1 second
        after_folder_contents = os.listdir(scanned_folder)
        
        if len(before_folder_contents) != len(after_folder_contents):
            modified_file = ... #here, we perform some function that gets the changed file, i.e. the added element between the two lists
            output_file.append(modified_file)
            break
            
def download_file():
    #I will not write it out in full, but this is the function that downloads a file. It will end once the file has been downloaded.
    
thread_download_detector = threading.Thread(target=downloaded_file_detector, args=(scanned_folder, variable_function))
thread_download_file = threading.Thread(target=download_file)
thread_download_detector.start()
thread_download_file.start()

#The following block of code will effectively pause execution until the threads have completed
while True:
    if len(output_file) != 0:
        break
    else:
        time.sleep(1) #check to see if the two threads have completed ever 1 second

或者,您可以使用 threading.Event 对象并在其中一个线程函数停止运行后设置它,以便查看何时检查下载的文件。这可能是更有效/“pythonic”的做事方式。

我希望这对以后遇到这个问题的人有所帮助。对于此类工作,请查看线程库和 threading.Event 对象,因为这些是(根据我的经验)在多个后续运行的进程之间进行通信的最佳方式。

© www.soinside.com 2019 - 2024. All rights reserved.