将刮取的数据保存到数据库中

问题描述 投票:0回答:1

我刮取了数据,并将刮取的数据保存到五个列表中,我创建了五个行的表,现在我不知道如何将我的刮取数据保存到数据库中。

我的代码是

import requests
from bs4 import BeautifulSoup
import re
import mysql.connector

url = 'https://www.ebay.com/b/Cars-Trucks/6001?_fsrp=0&_sacat=6001&LH_BIN=1&LH_ItemCondition=3000%7C1000%7C2500&rt=nc&_stpos=95125&Model%2520Year=2020%7C2019%7C2018%7C2017%7C2016%7C2015'
res = requests.get(url)
soup = BeautifulSoup(res.text, 'html.parser')
car_titles =[]
title = soup.find_all('h3', class_='s-item__title', limit = 20)
for title_of_car in title:
    car_titles.append(title_of_car.text)

car_brands = []
brands = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes1', limit = 20)
for brand in brands:
    brand = re.sub(r'Make: ','', brand.text)
    car_brands.append(brand)

car_models = []
models = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes2', limit = 20)
for model in models:
    model = re.sub(r'Model: ', '', model.text)
    car_models.append(model)

car_transmissions = []
transmissions = soup.find_all('span', class_='s-item__dynamic s-item__dynamicAttributes3', limit = 20)
for transmission in transmissions:
    transmission = re.sub(r'Transmission: ', '', transmission.text)
    car_transmissions.append(transmission)

car_prices = []
char_list = ['\$', '\,', '\.']
prices = soup.find_all('span', class_='s-item__price', limit = 20)
for price in prices:
    price = re.sub('|'.join(char_list), '', price.text)
    car_prices.append(int(price))
python mysql web-scraping mysql-connector
1个回答
0
投票

首先通过以下方式获得与mysql数据库的连接

import mysql.connector

config = {
  'user': 'username',
  'password': 'password',
  'host': '127.0.0.1',
  'database': 'dbname',
  'raise_on_warnings': True
}

cnx = mysql.connector.connect(**config)

# create cursor to execute mysql commands
cursor = cnx.cursor()

# Now create table like this one
cursor.execute("CREATE TABLE car_titles (title VARCHAR(255))")

# insert data to table like this one
for row in car_titles:
    cursor.execute("INSERT INTO car_titles (title) VALUES (%s)", row)

cnx.close()

主要是要使用sql查询来创建表和插入数据。所以要用python学习使用mysql

然后创建表格

© www.soinside.com 2019 - 2024. All rights reserved.