This is an example of a rule i'm using
Rule(LinkExtractor(allow='(http\:\/\/www\.example\.com\/.*_reviews.*-page-\d+\.html)'), follow=True, callback="parse_data")
Although This extracts the matching links from the
start_urls It does'nt extract matching links within a page as a
yield request at some point in the spider code.
Is this the normal way scrapy functions?