我正在尝试执行并行化的pyodbc脚本中运行此代码
templst = [lineitem, orders, partsupp, region, cur_cur, T1, T2]
connstr = [DRIVER={libdbodbc17.so};host=lint16muthab.phl.sap.corp:8766;UID=dba;PWD=sql;CharSet=utf8, DRIVER={libdbodbc17.so};host=localhost:8767;UID=dba;PWD=sql;CharSet=utf8,
DRIVER={libdbodbc17.so};host=localhost:8768;UID=dba;PWD=sql;CharSet=utf8,
DRIVER={libdbodbc17.so};host=localhost:8769;UID=dba;PWD=sql;CharSet=utf8, DRIVER={libdbodbc17.so};host=localhost:8770;UID=dba;PWD=sql;CharSet=utf8]
def extract_single(q, cursorconn):
while True:
try:
tableName = q.get_nowait()
time.sleep(3)
qry2 = "Select * FROM %s"% (tableName)
print " extraction done of table:%s done by cursor:%s"%(tableName,cursorconn)
except Queue.Empty:
return
def main():
q = multiprocessing.Queue()
for item in templst:
q.put(item) # add items to queue
process = []
for i in xrange(5):
p = multiprocessing.Process(target=extract_single, args=(q, connstr[i]))
process.append(p)
p.start()
for p in process:
p.join()
if __name__ == '__main__':
main()
输出类似于:
extraction done of table:lineitem done by cursor:DRIVER={libdbodbc17.so};host=lint16muthab.phl.sap.corp:8766;UID=dba;PWD=sql;CharSet=utf8
表的提取完成:游标完成的命令:DRIVER = {libdbodbc17.so}; host = localhost:8767; UID = dba; PWD = sql; CharSet = utf8
extraction done of table:partsupp done by cursor:DRIVER={libdbodbc17.so};host=localhost:8768;UID=dba;PWD=sql;CharSet=utf8
extraction done of table:region done by cursor:DRIVER={libdbodbc17.so};host=localhost:8769;UID=dba;PWD=sql;CharSet=utf8
extraction done of table:cur_cur done by cursor:DRIVER={libdbodbc17.so};host=localhost:8770;UID=dba;PWD=sql;CharSet=utf8
extraction done of table:T2 done by cursor:DRIVER={libdbodbc17.so};host=localhost:8767;UID=dba;PWD=sql;CharSet=utf8
extraction done of table:T1 done by cursor:DRIVER={libdbodbc17.so};host=lint16muthab.phl.sap.corp:8766;UID=dba;PWD=sql;CharSet=utf8
但是当我将提取函数修改为
def extract_single(q, cursorconn):
while True:
try:
tableName = q.get_nowait()
time.sleep(3)
conn = pyodbc.connect(cursorconn, timeout=0)
cursor = connvar.cursor()
qry2 = "Select * FROM %s"% (tableName)
cursor.execute(qry2).fetchall()
print " extraction done of table:%s done by cursor:%s"%(tableName,cursorconn)
except Queue.Empty:
return
我遇到错误,无法打开连接和游标,因此我可以执行此查询并将数据转储到文件中。
2Process Process-1:
Traceback (most recent call last):
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "conn3.py", line 86, in extract_single
Process Process-2:
Traceback (most recent call last):
connvar = pyodbc.connect(cursorconn, timeout=0)
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Error: ('IM005', "[IM005] [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_DBC failed (0) (SQLDriverConnect)")
self.run()
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "conn3.py", line 86, in extract_single
connvar = pyodbc.connect(cursorconn, timeout=0)
Error: ('IM005', "[IM005] [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_DBC failed (0) (SQLDriverConnect)")
Process Process-3:
Traceback (most recent call last):
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/sybopt/software/python/python/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "conn3.py", line 86, in extract_single
connvar = pyodbc.connect(cursorconn, timeout=0)
对于所有连接。请让我知道如何解决此错误。
除了您遇到的问题之外,基于您先前的问题,我还假设cursorconn
是一个已经打开的连接,可以在每个查询中重用。由于它现在似乎只是一个连接字符串,因此您应该在conn = pyodbc.connect(cursorconn, timeout=0)
语句中进行调用while True:
before,以便同一连接可以重用于多个查询,然后应在关闭连接之前您从函数返回。
我认为问题是语句cursor = connvar.cursor()
,应为:cursor = conn.cursor()
。因此,请尝试以下操作:
conn = pyodbc.connect(cursorconn, timeout=0)
while True:
try:
tableName = q.get_nowait()
time.sleep(3) # why is this here?
cursor = conn.cursor()
qry2 = "Select * FROM %s"% (tableName)
cursor.execute(qry2).fetchall()
print " extraction done of table:%s done by cursor:%s"%(tableName,cursorconn)
cursor.close() # should probably add this
except Queue.Empty:
return
finally:
conn.close()