Welcome to the Treehouse Community

Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.

Start your free trial

Python Scraping Data From the Web Additional Scraping Tasks An Intelligent Spider

Brian Haucke
Brian Haucke
13,717 Points
Posted by Brian Haucke
Brian Haucke
13,717 Points

KeyError: 'Spider not found: formSpider.py'

When I try to run formSpider.py from the terminal I get the following error: KeyError: 'Spider not found: formSpider.py'

This is what I am entering into the terminal to run it: scrapy crawl formSpider.py

Here is my code:

'''

from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

'''

2 Answers

rainmaker
rainmaker
14,690 Points 
from scrapy.http import FormRequest

from scrapy.spiders import Spider

class FormSpider(Spider):

name = 'horseForm'
start_urls = ['https://treehouse-projects.github.io/horse-land/form.html']

def parse(self, response):
    formdata ={'firstname': 'Kenneth',
               'lastname': 'Alger',
               'jobtitle': 'Teacher'}
    return FormRequest.from_response(response, formnumber=0,
                                     formdata=formdata,
                                     callback=self.after_post)
def after_post(self, response):
    print('n\n\*******\nForm processed.\n')
    print(response)
    print('\n*******\n')

try using scrapy crawl horseForm

paul nabende
paul nabende
673 Points

aah yes! We tell scrappy to crawl by the class name as opposed to the filename. Am just wondering if there is an additional way to run this by the filename. 🤔

Brian Haucke
Brian Haucke
13,717 Points
Brian Haucke
Brian Haucke
13,717 Points

Haha, yes! That was it. Thank you.