Python，从字符串中删除所有html标签

Question

我正在尝试使用以下代码的beautifulsoup从网站访问文章内容：

site= 'www.example.com'
page = urllib2.urlopen(req)
soup = BeautifulSoup(page)
content = soup.find_all('p')
content=str(content)

内容对象包含页面中“p”标记内的所有主要文本，但是输出中仍然存在其他标记，如下图所示。我想删除匹配的<>标签对和标签本身所包含的所有字符。这样只留下文字。

我尝试了以下方法，但它似乎不起作用。

' '.join(item for item in content.split() if not (item.startswith('<') and item.endswith('>')))

在sting中删除子串的最佳方法是什么？以某种模式开始和结束，例如<>

Answer 1

你可以使用get_text()

for i in content:
    print i.get_text()

以下示例来自docs：

>>> markup = '<a href="http://example.com/">\nI linked to <i>example.com</i>\n</a>'
>>> soup = BeautifulSoup(markup)
>>> soup.get_text()
u'\nI linked to example.com\n'

Answer 2

使用regEx：

re.sub('<[^<]+?>', '', text)

使用BeautifulSoup :(来自here的解决方案）

import urllib
from bs4 import BeautifulSoup

url = "http://news.bbc.co.uk/2/hi/health/2284783.stm"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)

# kill all script and style elements
for script in soup(["script", "style"]):
    script.extract()    # rip it out

# get text
text = soup.get_text()

# break into lines and remove leading and trailing space on each
lines = (line.strip() for line in text.splitlines())
# break multi-headlines into a line each
chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
# drop blank lines
text = '\n'.join(chunk for chunk in chunks if chunk)

print(text)

使用NLTK：

import nltk   
from urllib import urlopen
url = "https://stackoverflow.com/questions/tagged/python"    
html = urlopen(url).read()    
raw = nltk.clean_html(html)  
print(raw)

Answer 3

你需要使用strings generator：

for text in content.strings:
   print(text)

Answer 4

通过定义匹配所有打开和关闭HTML标记的模式，然后使用该模式转换输入作为抑制器，Pyparsing可以轻松编写HTML剥离器。这仍然留下要转换的q​​azxswpoi HTML实体 - 你可以使用&xxx;来做到这一点：

xml.sax.saxutils.unescape

得到：

source = """
<p><strong>Editors' Pick: Originally published March 22.<br /> <br /> Apple</strong> <span class=" TICKERFLAT">(<a href="/quote/AAPL.html">AAPL</a> - <a href="http://secure2.thestreet.com/cap/prm.do?OID=028198&amp;ticker=AAPL">Get Report</a><a class=" arrow" href="/quote/AAPL.html"><span class=" tickerChange" id="story_AAPL"></span></a>)</span> is waking up the echoes with the reintroduction of a&nbsp;4-inch iPhone, a model&nbsp;its creators hope will lead the company to victory not just in emerging markets, but at home as well.</p> 
<p>&quot;There's significant pent-up demand within Apple's base of iPhone owners who want a smaller iPhone with up-to-date specs and newer features,&quot; Jackdaw Research Chief Analyst Jan Dawson said in e-mailed comments.</p> 
<p>The new model, dubbed the iPhone SE, &quot;should unleash a decent upgrade cycle over the coming months,&quot; Dawson said.&nbsp;Prior to the iPhone 6 and 6 Plus, introduced in 2014, Apple's iPhones were small, at 3.5 inches and 4 inches tall, respectively, compared with models by Samsung and others that approached 6 inches.</p>
<div class=" butonTextPromoAd">
 <div class=" ym" id="ym_44444440"></div>"""

from pyparsing import anyOpenTag, anyCloseTag
from xml.sax.saxutils import unescape as unescape
unescape_xml_entities = lambda s: unescape(s, {"&apos;": "'", "&quot;": '"', "&nbsp;":" "})

stripper = (anyOpenTag | anyCloseTag).suppress()

print(unescape_xml_entities(stripper.transformString(source)))

（将来，请不要提供示例文本或代码作为非可复制粘贴的图像。）

Answer 5

如果您限制使用任何库，您只需使用以下代码删除html标记。

我只是纠正你的尝试。谢谢你的想法

Editors' Pick: Originally published March 22.  Apple (AAPL - Get Report) is waking up the echoes with the reintroduction of a 4-inch iPhone, a model its creators hope will lead the company to victory not just in emerging markets, but at home as well. 
"There's significant pent-up demand within Apple's base of iPhone owners who want a smaller iPhone with up-to-date specs and newer features," Jackdaw Research Chief Analyst Jan Dawson said in e-mailed comments. 
The new model, dubbed the iPhone SE, "should unleash a decent upgrade cycle over the coming months," Dawson said. Prior to the iPhone 6 and 6 Plus, introduced in 2014, Apple's iPhones were small, at 3.5 inches and 4 inches tall, respectively, compared with models by Samsung and others that approached 6 inches.

Answer 6

简单的算法，无需模块和导入的其他库，可以在每种语言中使用。代码是自我记录的：

content="<h4 style='font-size: 11pt; color: rgb(67, 67, 67); font-family: arial, sans-serif;'>Sample text for display.</h4> <p>&nbsp;</p>"


' '.join([word for line in [item.strip() for item in content.replace('<',' <').replace('>','> ').split('>') if not (item.strip().startswith('<') or (item.strip().startswith('&') and item.strip().endswith(';')))] for word in line.split() if not (word.strip().startswith('<') or (word.strip().startswith('&') and word.strip().endswith(';')))])

为了更好地实现文字'>'和'<'需要在循环开始之前在内存中实例化一次。