Python Scraping Data From the Web Additional Scraping Tasks An Intelligent Spider

Brian Haucke
Brian Haucke
13,717 Points

KeyError: 'Spider not found: formSpider.py'

When I try to run formSpider.py from the terminal I get the following error: KeyError: 'Spider not found: formSpider.py'

This is what I am entering into the terminal to run it: scrapy crawl formSpider.py

Here is my code:

'''

from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

'''

2 Answers

from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

try using scrapy crawl horseForm

aah yes! We tell scrappy to crawl by the class name as opposed to the filename. Am just wondering if there is an additional way to run this by the filename. 🤔

Brian Haucke
Brian Haucke
13,717 Points

Haha, yes! That was it. Thank you.