从嵌套的json文件中提取数据并插入sql db。从关键详细信息中,我想在下面获取信息并插入到sql db中;
json文件数据和键/值的样本如下
{
"response": {
"client_log": {
"data": [
{
"city": "LONDON",
"login": "AAAAAAAAAAAAAA",
"state": "MC",
"details": "Please find report below:\r\n\r\n------Report Information------\r\n\r\nEmail Id: [email protected]\r\nServ Id: 1101ar12\r\nServ Num: 11111\r\nServ Details: Super-A\r\nState: LONDON\r\nCity: LONDON\r\n\r\n------Service Information------\r\n\r\nUser Name: John Clark\r\nMobile Number: 000111222\r\n\r\n------Reported Form------\r\n\r\nForm-1: zzzzz\r\nType: 111\r\n\r\nRemarks: Remarks 123.",
"log_number": "1",
"department": "Sales",
"staff_id": "S123",
"staff_name": "EricY",
"timestamp": "2020-02-27 15:57:24"
},
{
"city": "SINGAPORE",
"login": "BBBBBBBBBBBBB",
"state": "XX",
"details": "Please find report below:\r\n\r\n------Report Information------\r\n\r\nEmail Id: [email protected]\r\nServ Id: 903oa112\r\nServ Num: 12345\r\nServ Details: Super-B\r\nState: Sydney\r\nCity: Sydney\r\n\r\n------Service Information------\r\n\r\nUser Name: Peter\r\nMobile Number: 333444555\r\n\r\n------Reported Form------\r\n\r\nForm-2: xxxxxxxxxx\r\nType: 111\r\n\r\nRemarks: Remarks 890.",
"log_number": "1",
"department": "Eng",
"staff_id": "S456",
"staff_name": "YongG",
"timestamp": "2020-02-27 15:57:24"
}
],
"query": "13"
},
"response_time": "0.723494",
"transaction_id": "909122",
"transaction_status": "OK",
}
}
这是我用来提取数据并插入到sql中的代码段代码
myfile = 'sample.json'
with open(myfile, 'r') as f:
mydata = json.load(f)
sql = "INSERT INTO `table1` (`city`, `login`, `state`, `details`, `log_number`, `department`, `staff_id`, `staff_name`, `timestamp`) VALUES ( %(city)s, %(login)s, %(state)s, %(details)s, %(log_number)s, %(department)s, %(staff_id)s, %(staff_name)s, %(timestamp)s )"
cursor.executemany( sql, mydata['response']['client_log']['data'])
db.commit()
db.close()
从上面的代码中,我能够获得包含数据的键的详细信息,但是我得到的详细信息值已显示并插入到SQL中的1个大块数据中(SQL列详细信息...谢谢。
您可以使用str
和/或details
方法轻松解析与split()
键关联的find()
。一行一行,并假设:
左侧的所有内容都是键,而右侧的所有内容都是值。
例如:
import json
myfile = 'sample.json'
with open(myfile, 'r') as f:
mydata = json.load(f)
for entry in mydata['response']['client_log']['data']:
parsed_details = {}
for line in entry['details'].split('\r\n'):
split = line.find(': ') # find() returns -1 if no match is found
if split != -1:
key = line[:split]
value = line[split+2:] # 2 = len(': ')
parsed_details[key] = value
entry['parsed_details'] = parsed_details
或
import json
myfile = 'sample.json'
with open(myfile, 'r') as f:
mydata = json.load(f)
for entry in mydata['response']['client_log']['data']:
parsed_details = {}
for line in entry['details'].split('\r\n'):
try:
key, value = line.split(': ', maxsplit=1)
parsed_details[key] = value
except ValueError:
# This error is only thrown when the line doesn't have ': ' in it,
# which means there aren't enough values to unpack. It is safe to pass.
pass
entry['parsed_details'] = parsed_details
dict
是mydata['response']['client_log']['data']
list
的每一项,现在都有一个parsed_details
键,其值为一个dict
,其键和从details
键中提取的值对,如您所见)在此输出中:
mydata
Out[2]:
{'response': {'client_log': {'data': [{'city': 'LONDON',
'login': 'AAAAAAAAAAAAAA',
'state': 'MC',
'details': 'Please find report below:\r\n\r\n------Report Information------\r\n\r\nEmail Id: [email protected]\r\nServ Id: 1101ar12\r\nServ Num: 11111\r\nServ Details: Super-A\r\nState: LONDON\r\nCity: LONDON\r\n\r\n------Service Information------\r\n\r\nUser Name: John Clark\r\nMobile Number: 000111222\r\n\r\n------Reported Form------\r\n\r\nForm-1: zzzzz\r\nType: 111\r\n\r\nRemarks: Remarks 123.',
'log_number': '1',
'department': 'Sales',
'staff_id': 'S123',
'staff_name': 'EricY',
'timestamp': '2020-02-27 15:57:24',
'parsed_details': {'Email Id': '[email protected]',
'Serv Id': '1101ar12',
'Serv Num': '11111',
'Serv Details': 'Super-A',
'State': 'LONDON',
'City': 'LONDON',
'User Name': 'John Clark',
'Mobile Number': '000111222',
'Form-1': 'zzzzz',
'Type': '111',
'Remarks': 'Remarks 123.'}},
{'city': 'SINGAPORE',
'login': 'BBBBBBBBBBBBB',
'state': 'XX',
'details': 'Please find report below:\r\n\r\n------Report Information------\r\n\r\nEmail Id: [email protected]\r\nServ Id: 903oa112\r\nServ Num: 12345\r\nServ Details: Super-B\r\nState: Sydney\r\nCity: Sydney\r\n\r\n------Service Information------\r\n\r\nUser Name: Peter\r\nMobile Number: 333444555\r\n\r\n------Reported Form------\r\n\r\nForm-2: xxxxxxxxxx\r\nType: 111\r\n\r\nRemarks: Remarks 890.',
'log_number': '1',
'department': 'Eng',
'staff_id': 'S456',
'staff_name': 'YongG',
'timestamp': '2020-02-27 15:57:24',
'parsed_details': {'Email Id': '[email protected]',
'Serv Id': '903oa112',
'Serv Num': '12345',
'Serv Details': 'Super-B',
'State': 'Sydney',
'City': 'Sydney',
'User Name': 'Peter',
'Mobile Number': '333444555',
'Form-2': 'xxxxxxxxxx',
'Type': '111',
'Remarks': 'Remarks 890.'}}],
'query': '13'},
'response_time': '0.723494',
'transaction_id': '909122',
'transaction_status': 'OK'}}
我对SQL不太熟悉,因此在您更新数据库的最后一步方面无济于事。