我曾经到的HtmlUnit从网页刮图像。我在初学的HtmlUnit。我编码,但不知道如何去获取图片。下面是我的代码。
import java.io.*;
import java.net.URL;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class urlscrap {
public static void main(String[] args) throws Exception
{
//WebClient webClient = new WebClient(Opera);
WebClient webClient = new WebClient();
HtmlPage currentPage = (HtmlPage) webClient.getPage(new URL("http://www.google.com"));
System.out.println(currentPage.asText());
//webClient.closeAllWindows();
}
}
这是否为您工作?
import java.net.URL;
import java.util.List;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlImage;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class urlscrap {
public static void main(String[] args) throws Exception
{
//WebClient webClient = new WebClient(Opera);
WebClient webClient = new WebClient();
HtmlPage currentPage = (HtmlPage) webClient.getPage(new URL("http://www.google.com"));
//get list of all divs
final List<?> images = currentPage.getByXPath("//img");
for (Object imageObject : images) {
HtmlImage image = (HtmlImage) imageObject;
System.out.println(image.getSrcAttribute());
}
//webClient.closeAllWindows();
}
}
看起来你得到的页面,这确实是第一步的文本。你有什么问题?你有找到页面中引用的所有图像有问题吗?我建议找了怎么办DOM在Java中解析,并用它来提取网页上的所有img标签。
如果你不介意切换语言,那么我会建议Python的scrapy。这是迄今为止我已经用刮网页内容,包括图片(它甚至可以自动为你创建缩略图)的最佳框架。就个人而言,我不会使用Java这样的任务。