Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.Start your free trial
How do you create a scrapy project?
I'm using pycharm. I have scrapy installed on my project I go to the terminal. I type "scrapy startproject AraneaSpider"
I'm fairly new to using pycharm, I was following the directions in this course. Also I'm using windows.
Then I get this error.
File "C:\Users\ID611\PycharmProjects\web_scraping\venv\Scripts\scrapy.exe\__main__.py", line 5, in <module>
ModuleNotFoundError: No module named 'scrapy.cmdline'; 'scrapy' is not a package
Eric M11,545 Points
Although Scrapy is showing as installed it's likely something has gone wrong with the install. Perhaps with one of Scrapy's dependencies or path.
If you've only installed via the video's instructions you should only have Scrapy as a project install. You can verify this by running
pip show scrapy from a command line outside of Pycharm's venv. Such as a regular powershell terminal. It will be blank.
This means you can try creating a brand new PyCharm project and installing Scrapy again there (the same way as from the video). Be sure to wait for the green install successful message as appears in the video (though it may take a bit longer than in the video).
(venv) C:\Users\ID611\PycharmProjects\web_scraping>pip show scrapy Name: Scrapy Version: 1.5.1 Summary: A high-level Web Crawling and Web Scraping framework Home-page: https://scrapy.org Author: Scrapy developers Author-email: None License: BSD Location: c:\users\id611\pycharmprojects\web_scraping\venv\lib\site-packages Requires: pyOpenSSL, cssselect, queuelib, PyDispatcher, service-identity, parsel, six, w3lib, lxml, Twisted Required-by: You are using pip version 10.0.1, however version 18.1 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Thank you for the fix, I just created a new project and ran the same commands.