我正在从谷歌图书中搜集一些信息(对NHL团队进行研究),我正在使用RSelenium
开始:
library(tidyverse)
library(RSelenium) # using Docker
library(rvest)
library(httr)
remDr <- remoteDriver(port = 4445L, browserName = "chrome")
remDr$open()
remDr$navigate("https://books.google.com/")
books <- remDr$findElement(using = "css", "[name = 'q']")
books$sendKeysToElement(list("NHL teams", key = "enter"))
bookElem <- remDr$findElements(using = "xpath",
"//h3[@class = 'LC20lb']//parent::a")
links <- sapply(bookElem, function(bookElem){
bookElem$getElementAttribute("href")
})
以上导航到正确的页面并搜索“NHL团队”。但有一点需要注意的是,其中一些书籍有一个“预览”页面,为了得到肉(标题,作者等),必须在“关于这本书”上进一步点击:
for(link in links) {
remDr$navigate(link)
# If statement to get past book previews
if (str_detect(link, "frontcover")) {
# Finding elements for "About this book"
link2 <- remDr$findElements(using = 'xpath',
'//a[@id="sidebar-atb-link" and span[.="About this book"]]')
# Clicking on the "About this book" links
link2_about <- sapply(link2, function(link2){
link2$getElementAttribute('href')
})
duh <- map(link2_about, read_html)
# NHL book title, author
nhl_title <- duh %>%
map(html_nodes, '#bookinfo > h1 > span.fn > span') %>%
map_chr(html_text) %>%
print()
author1 <- duh %>%
map(html_nodes, '#bookinfo div:nth-child(1) span') %>%
map_chr(html_text) %>%
print()
test_df <- cbind(nhl_title, author1) # ONLY binds the last book/author
print(test_df)
} else {
print("lol you thought this would work?") # haven't built this part out yet
}
}
我对map
的使用打印出单个标题/作者,我无法弄清楚如何将它们放入数据帧。每次我使用tibble()
或map_dfr()
我都会收到错误。上面的for
循环列出了标题然后是作者,但没有把任何东西放在一起。如何将所有这些绑定到一个框架中?
答案结果很简单。我只需要在for
循环上面添加一个空白列表,然后在循环内添加它。例如,
blank_list <- list()
for(link in links) {
....
blank_list[[link]] <- tibble(nhl_title, author1)
wow <- bind_rows(blank_list)
print(wow)
}
不要使用do.call()
或其他选项,bind_rows()
比其他选项更快。