我刮取了数据,并将刮取的数据保存到五个列表中,我创建了五个行的表,现在我不知道如何将我的刮取数据保存到数据库中。
我的代码是
import requests
from bs4 import BeautifulSoup
import re
import mysql.connector
url = 'https://www.ebay.com/b/Cars-Trucks/6001?_fsrp=0&_sacat=6001&LH_BIN=1&LH_ItemCondition=3000%7C1000%7C2500&rt=nc&_stpos=95125&Model%2520Year=2020%7C2019%7C2018%7C2017%7C2016%7C2015'
res = requests.get(url)
soup = BeautifulSoup(res.text, 'html.parser')
car_titles =[]
title = soup.find_all('h3', class_='s-item__title', limit = 20)
for title_of_car in title:
car_titles.append(title_of_car.text)
car_brands = []
brands = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes1', limit = 20)
for brand in brands:
brand = re.sub(r'Make: ','', brand.text)
car_brands.append(brand)
car_models = []
models = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes2', limit = 20)
for model in models:
model = re.sub(r'Model: ', '', model.text)
car_models.append(model)
car_transmissions = []
transmissions = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes3', limit = 20)
for transmission in transmissions:
transmission = re.sub(r'Transmission: ', '', transmission.text)
car_transmissions.append(transmission)
car_prices = []
char_list = ['\$', '\,', '\.']
prices = soup.find_all('span', class_='s-item__price', limit = 20)
for price in prices:
price = re.sub('|'.join(char_list), '', price.text)
car_prices.append(int(price))
首先通过以下方式获得与mysql数据库的连接
import mysql.connector
config = {
'user': 'username',
'password': 'password',
'host': '127.0.0.1',
'database': 'dbname',
'raise_on_warnings': True
}
cnx = mysql.connector.connect(**config)
# create cursor to execute mysql commands
cursor = cnx.cursor()
# Now create table like this one
cursor.execute("CREATE TABLE car_titles (title VARCHAR(255))")
# insert data to table like this one
for row in car_titles:
cursor.execute("INSERT INTO car_titles (title) VALUES (%s)", row)
cnx.close()
主要是要使用sql查询来创建表和插入数据。所以要用python学习使用mysql
然后创建表格