Welcome to the Treehouse Community

The Treehouse Community is a meeting place for developers, designers, and programmers of all backgrounds and skill levels to get support. Collaborate here on code errors or bugs that you need feedback on, or asking for an extra set of eyes on your latest project. Join thousands of Treehouse students and alumni in the community today. (Note: Only Treehouse students can comment or ask questions, but non-students are welcome to browse our conversations.)

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and a supportive community. Start your free trial today.

Python Scraping Data From the Web Additional Scraping Tasks An Intelligent Spider

Brian Haucke
Brian Haucke
13,717 Points

KeyError: 'Spider not found: formSpider.py'

When I try to run formSpider.py from the terminal I get the following error: KeyError: 'Spider not found: formSpider.py'

This is what I am entering into the terminal to run it: scrapy crawl formSpider.py

Here is my code:

'''

from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

'''

2 Answers

from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

try using scrapy crawl horseForm

aah yes! We tell scrappy to crawl by the class name as opposed to the filename. Am just wondering if there is an additional way to run this by the filename. 🤔

Brian Haucke
Brian Haucke
13,717 Points

Haha, yes! That was it. Thank you.