当前位置: 动力学知识库 > 问答 > 编程问答 >

python - Get xpath() to return empty values

问题描述:

I have a situation where I have a lot of <b> tags:

<b>12</b>

<b>13</b>

<b>14</b>

<b></b>

<b>121</b>

As you can see, the second last tag is empty. When I call:

sel.xpath('b/text()').extract()

Which gives me:

['12', '13', '14', '121']

I would like to have:

['12', '13', '14', '', '121']

Is there a way to get the empty value?


My current work around is to call:

sel.xpath('b').extract()

And then parsing through each html tag myself (the empty tags are here, which is what I want).

网友答案:

This is where it is okay to manually strip the tags and get the text. You can use remove_tags() function provided by w3lib:

>>> from w3lib.html import remove_tags
>>> map(remove_tags, sel.xpath('//b').extract())
[u'12', u'13', u'14', u'', u'121']

Note that w3lib is a Scrapy dependency and is used internally. No need to install it separately.

Also, it would be better to use Scrapy Input and Output Processors here. Continue using sel.xpath('b') and define an input processor. For example, you can define it for specific Fields for the Item class:

from scrapy.contrib.loader.processor import MapCompose
from scrapy.item import Item, Field
from w3lib.html import remove_tags

class MyItem(Item):
    my_field = Field(input_processor=MapCompose(remove_tags)) 
分享给朋友:
您可能感兴趣的文章:
随机阅读: