使用beautifulsoup将html表转换为CSV

Question

我对抓取是陌生的，我无法弄清楚如何从某个网站获取所需的数据。这是我的代码：

from lxml import html
import requests
from bs4 import BeautifulSoup
import pandas as pd

website_url = requests.get('https://thereserve2.apx.com/mymodule/reg/prjView.asp?id1=1295').text
soup = BeautifulSoup(website_url,'lxml')
print(soup.prettify())

table = soup.table
table_rows = table.find_all('tr')

for tr in table_rows:
    td = tr.find_all('td')
    row = [i.text for i in td]
    print(row)

df = pd.DataFrame()
df['Rows'] = row
df

输出显示的表格类似于['Column 1','Column 2']，因此应该很容易将其转换为可导出的表格，但由于某些原因它无法正常工作。

Answer 1

import requests
from bs4 import BeautifulSoup
import pandas as pd


website_url = requests.get('https://thereserve2.apx.com/mymodule/reg/prjView.asp?id1=1295').text
soup = BeautifulSoup(website_url,'html.parser')
table = soup.table
table_rows = table.find_all('tr')

data = {}
for tr in table_rows:
    td = tr.find_all('td')
    if len(td) != 2 : continue
    data[td[0].text] = td[1].text.strip()

data_frame = pd.DataFrame([data], columns=data.keys())

data_frame.to_csv('output.csv', index=False)

Answer 2

import pandas as pd

df = pd.read_html(
    "https://thereserve2.apx.com/mymodule/reg/prjView.asp?id1=1295", skiprows=1)[0]

new = pd.DataFrame([df.iloc[:, 4].to_list()], columns=df.iloc[:, 3].to_list())

print(new)

new.to_csv("data.csv", index=False)

输出：view-online