How to get HTML from URL and save it into a file with scrapy in Python

1 Answer

import scrapy 

class Spider(scrapy.Spider):
    name = "SPIDER_NAME"

    start_urls = [
        'https://www.example.com/'
    ]

    def parse(self, response):
        filename = 'url-info'
        with open(filename, 'wb') as f:
            f.write(response.body) 


# Windows 10
# Visual Studio Code - In TERMINAL

scrapy crawl SPIDER_NAME



''' 
run:

<!doctype html>
<html>
<head>
    <title>...

'''

70+ SQL courses for beginners and professionals

answered Jun 17, 2020 by avibootz
edited Jun 18, 2020 by avibootz

Most popular tags

How to get HTML from URL and save it into a file with scrapy in Python

1 Answer

Related questions