Question

How to get URL page title with scrapy in Python

Answer 1

import scrapy

class Spider(scrapy.Spider):
    name = "SPIDER_NAME"

    start_urls = [
        'https://www.example.com/'
    ]

    def parse(self, response):
        yield {
            'title': response.css('title::text').get()
        }


# Windows 10
# Visual Studio Code - In TERMINAL

scrapy crawl SPIDER_NAME



''' 
run:

{'title': 'Example Domain'}

'''

 

70+ SQL courses for beginners and professionals