当其中一个线程（chrome）异常时，Uwsgi挂起（10-60秒）

Question

我有一台服务器，可以为大约100个客户服务。我使用sqlite本地数据库来存储和检索数据。

码：

class myThread (threading.Thread):
    def __init__(self, threadID, name, q, q_imme):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q
        self.q_imme = q_imme
    def run(self):
        print ("Starting " + self.name)
        worker(self.name, self.q, self.q_imme)
        print ("Exiting " + self.name)


def worker(name, q, q_imme):
    try:
        # Do some chrome-selenium-automation stuff
        while not exitFlag:
            username = None
            donelist = None
            type = 0
            while username is None:
                sleep(1)
                if not q_imme.empty():
                    usernameAndDonelistTuple = q_imme.get(timeout=1)
                    username = usernameAndDonelistTuple[0]
                    type     = usernameAndDonelistTuple[1]
                    donelist = usernameAndDonelistTuple[2]
                elif not q.empty():
                    username = q.get(timeout=1)

            print('{}: start grabbing for {} type: {}'.format(name, username, type))
            time_element_in_dict = {'timestamp': str(datetime.now())}
            if type == 0:  # regular data
                res, data = get_api_info(session, username)

                if res is True:
                    data.update({'status': 'ok'})
                elif res is None:
                    data = {'status': 204}  # No Content
                elif res is False:
                    data = {'status': 503}  # exception

                # TODO: async update database
                data.update(time_element_in_dict)
                userDb[username] = data

                print('{}: done grabbing for {} with status {}'.format(name, username, data['status']))
                if donelist:  # means we need to confirm
                    status = data
                    donelist.put(status)
    except:
        # when we reach here the application hangs!!!, I also manually killed all chrome proceess and it hangs. 

import time
class user_info(Resource):
    def get(self):
        username = request.args.get('username')
        start_time = time.time()
        print('username:{} ~ {}'.format(username, start_time))
        return_data = dict()
        username = username.lower()
        try:
            # first try to grab from database
            #lock.acquire()
            return_data[username] = userDb[username]
            #lock.release()
            print('username: {}'.format(time.time() - start_time))
        except KeyError:
            # grab using chrome
            doneList = doneListsPool.get()
            type = 0
            usernameAndDoneListTuple = (username, type, doneList)
            workQueue_imme.put(usernameAndDoneListTuple)
            #lock.release()
            return_data[username] = doneList.get()  # wait for a thread to finish and confirm
            doneListsPool.put(doneList)  # return doneList back to pool

        return return_data


# threading
exitFlag = 0

# database
userDb = SqliteDict('./usernames.sqlite', encode=json.dumps, decode=json.loads, autocommit=True)

print("1")

threadList = ['1', '2', '3', '4', '5', '6']
NUMBER_OF_THREADS = len(threadList)
NUMBER_OF_PENDING_QUEUES = NUMBER_OF_THREADS*2
queue#lock = threading.#lock()
workQueue = queue.Queue(2000)
workQueue_imme = queue.Queue(100)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue, workQueue_imme)
    thread.start()
    threads.append(thread)
    threadID += 1
    sleep(5)

# create pool of done lists
doneListsPool = queue.Queue()
for i in range(NUMBER_OF_PENDING_QUEUES):
    doneListsPool.put(queue.Queue())

app = Flask(__name__)
api = Api(app)
api.add_resource(user_info_list, '/userinfolist')  # Route_1
api.add_resource(user_info, '/userinfo')  # Route_2

if __name__ == '__main__':
    app.run(host='0.0.0.0', port='5007')

• 甚至1个工作者（线程“工作者”函数）都失败了，除了它结束了chrome进程，但在此期间没有处理请求，也请求只需要数据库，我没有看到“print（'用户名：{} 〜{}'.format（username，start_time））“行打印，直到chrome重新启动。
• Localhost请求也失败

我像这样启动我的服务器：

uwsgi --socket 0.0.0.0:5077 --protocol=http -w wsgi:app --enable-threads --ignore-sigpipe --ignore-write-errors --disable-write-exception --listen 120 --threads 2

• 我认为这个问题在我添加了donelist以确认线程工作者完成的“user_info”函数时就开始了。

Answer 1

您的完成列表会因工作者中的异常而丢失。加：

        if donelist:  # means we need to confirm
            status = data
            donelist.put(status)

到例外。