很抱歉,这个问题已经解决了。我正在尝试分析一些Facebook消息。到目前为止,我下载了一个html文件,并使用BeautifulSoup将其转换为一个整洁的列表,现在我正尝试从中创建一个数据框。
我正在查看此资源:https://datatofish.com/list-to-dataframe/,但是没有用。
这里是我现在的格式:
list = ['2019-01-07 12:51 PM', 'name1', 'hi how are you', 'im at home', 'wanna come over?', '2019-01-07 01:02 PM', 'name2', 'hell yeah', '🐟', 'ill bring beer', '2019-01-07 01:06 PM', 'name1', 'awesome', 'and so on']
我尝试了几种不同的方法,但我开始认为自己的能力超出了我的咀嚼能力。我正在学习。
这是我希望得到的输出:
index date time name message
0 2019-01-07 12:51 PM name1 hi how are you
1 2019-01-07 12:51 PM name1 im at home
2 2019-01-07 12:51 PM name1 wanna come over?
3 2019-01-07 12:56 PM name2 hell yeah
我尝试遍历列表并填充列,并点击了日期,名称或消息。
正如我说的,我正在学习,因此,如果不向我指出正确的研究方向,而不是解决方案,那将是惊人的。我将非常感谢。谢谢!
编辑:我尝试了几个现有的消息解析器,但是由于某种原因,它们都在2018年停止了支持。它们也都给我解析错误消息。
这有点难看,但是可以。我很乐意赞成一个更精致的解决方案!
l = iter(['2019-01-07 12:51 PM', 'name1', 'hi how are you', 'im at home', 'wanna come over?', '2019-01-07 01:02 PM', 'name2', 'hell yeah', '🐟', 'ill bring beer', '2019-01-07 01:06 PM', 'name1', 'awesome', 'and so on'])
df = pd.DataFrame()
# get first element in list
x = next(l)
# if element is the last, catch the IterationError and stop
try:
while 1:
# try to convert element to datetime
datetime = pd.to_datetime(x, format="%Y-%m-%d %H:%M %p")
# if successful get next element as name
x = next(l)
name = x
# get next elements as messages while they do not match datetime format
x = next(l)
while 1:
try:
# if datetime conversion is successful break while
pd.to_datetime(x, format="%Y-%m-%d %H:%M %p");
break
except ValueError:
# else add message to dataframe
df = df.append([{"datetime":datetime,"name":name,"msg":x}])
x = next(l)
except StopIteration:
pass
df["date"] = df["datetime"].dt.date
df["time"] = df["datetime"].dt.time
print(df)
datetime msg name date time
0 2019-01-07 12:51:00 hi how are you name1 2019-01-07 12:51:00
0 2019-01-07 12:51:00 im at home name1 2019-01-07 12:51:00
0 2019-01-07 12:51:00 wanna come over? name1 2019-01-07 12:51:00
0 2019-01-07 01:02:00 hell yeah name2 2019-01-07 01:02:00
0 2019-01-07 01:02:00 🐟 name2 2019-01-07 01:02:00
0 2019-01-07 01:02:00 ill bring beer name2 2019-01-07 01:02:00
0 2019-01-07 01:06:00 awesome name1 2019-01-07 01:06:00
0 2019-01-07 01:06:00 and so on name1 2019-01-07 01:06:00
使用正则表达式和列表推导,将列表内容提取并转换为Pandas数据框:
import pandas as pd
import re
datetime_regex = re.compile(r"\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}\sPM")
name_regex = re.compile(r"name\d+")
cols = ["date",
"time",
"name",
"message"
]
l = ['2019-01-07 12:51 PM',
'name1',
'hi how are you',
'im at home',
'wanna come over?',
'2019-01-07 01:02 PM',
'name2',
'hell yeah',
'🐟',
'ill bring beer',
'2019-01-07 01:06 PM',
'name1',
'awesome',
'and so on'
]
tmp = ''.join(l)
datetimes = re.findall(datetime_regex, tmp)
dates = [datetime[:11] for datetime in datetimes]
times = [datetime[11:] for datetime in datetimes]
names = re.findall(name_regex, tmp)
messages = [line
for line in l
if not line.startswith(('2019', 'name1', 'name2'))
]
data = [[[dates[0], times[0], names[0], msg]
for msg in messages[:3]],
[[dates[1], times[1], names[1], messages[3]]],
[[dates[2], times[2], names[2], msg]
for msg in messages[4:]]
]
flatten = [item for sublist in data for item in sublist]
df = pd.DataFrame(flatten, columns=cols)
print(df)
哪个返回:
date time name message
0 2019-01-07 12:51 PM name1 hi how are you
1 2019-01-07 12:51 PM name1 im at home
2 2019-01-07 12:51 PM name1 wanna come over?
3 2019-01-07 01:02 PM name2 hell yeah
4 2019-01-07 01:06 PM name1 🐟
5 2019-01-07 01:06 PM name1 ill bring beer
6 2019-01-07 01:06 PM name1 awesome
7 2019-01-07 01:06 PM name1 and so on