scrapy xpath中提取多个class值

时间:2022-11-13 12:43:42

xpath中没有提供对class的原生查找方法。但是 * 看到了一个很有才的回答:

This selector should work but will be more efficient if you replace it with your suited markup:
这个表达式应该是可行的。不过如果你把class换成更好识别的标识执行效率会更高

//*[contains(@class, 'Test')]  

But since this will also match cases like class="Testvalue" or class="newTest".

但是这个表达式会把类似 class="Testvalue" 或者 class="newTest"也匹配出来。

//*[contains(concat(' ', @class, ' '), ' Test ')]  

If you wished to be really certain that it will match correctly, you could also use the normalize-space function to clean up stray whitespace characters around the class name (as mentioned by @Terry)

如果您希望确定它能够正确匹配,则还可以使用 normalize-space 函数清除类名周围的空白字符(如@Terry所述)

//*[contains(concat(' ', normalize-space(@class), ' '), ' Test ')]  

Note that in all these versions, the * should best be replaced by whatever element name you actually wish to match, unless you wish to search each and every element in the document for the given condition.

请注意在所有这些版本里,除非你想要在所有元素里搜索带有这些条件的元素,否则你最好把*号替换成你想要匹配的具体的元素名(标签名)。

具体来讲大概就是这样:

html = """
<div class="view_fields">
<div class="row view_row">
<!-- row header -->
<div class="view_field col-xs-4 col-sm-5 col-md-4">Organization
</div>
<!-- row value -->
<div class="view_value col-xs-8 col-sm-7 col-md-8">
<a href="/org/14607">INTERNET HARBOR</a>
</div>
</div>
</div>
""" items = response.xpath('//div[contains(@class,"view_fields")]')

相反:不包含指定的属性或值

如果查找某个属性值是否包含或不包含指定的属性或值时:结合Xpath路径来提取循环列表中的一个HTML标签的InnerText,提取的时候需要判断是这个标签的class属性是否包含某个指定的属性值,利用Xpath的contains可以解决,代码如下:

//选择不包含class属性的节点
var result = node.SelectNodes(".//span[not(@class)]");
//选择不包含class和id属性的节点
var result = node.SelectNodes(".//span[not(@class) and not(@id)]");
//选择不包含class="expire"的span
var result = node.SelectNodes(".//span[not(contains(@class,'expire'))]");
//选择包含class="expire"的span
var result = node.SelectNodes(".//span[contains(@class,'expire')]");

查询值为空的节点

xml.xpath("//[XX[.!='']]")
--------------------------------------//td[.!= '']---------------------------------------------------------------- if each.xpath("./td[2]/text()[.!= '']"): self.positionType = each.xpath("./td[2]/text()").extract()[0] else:
self.positionType = "未知"

博客搬运地址