Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial
Nathan English
Front End Web Development Techdegree Student 10,817 PointsProblem starting a scrapy project
i'm in the terminal, and I run "scrapy startproject arenaSpider " But then I get
File "C:\Users\ID611\PycharmProjects\web_scraping\venv\Scripts\scrapy.exe_main_.py", line 5, in <module> ModuleNotFoundError: No module named 'scrapy.cmdline'; 'scrapy' is not a package
1 Answer
Alex Koumparos
Python Development Techdegree Student 36,887 PointsHi Nathan,
How did you install scrapy? I'm guessing that you followed the video and installed it from inside PyCharm. I don't use PyCharm or Windows, so I can't replicate that process on my end.
I'm going to suggest, if you haven't tried this already, to completely uninstall the version of scrapy that you have installed. Then try following the official scrapy instructions for Windows users:
Windows
Though it’s possible to install Scrapy on Windows using pip, we recommend you to install Anaconda or Miniconda and use the package from the conda-forge channel, which will avoid most installation issues.
Once you’ve installed Anaconda or Miniconda, install Scrapy with:
conda install -c conda-forge scrapy
If that doesn't work, the problem may be due to having two versions of scrapy installed somewhere on your system and in conflict. Older versions had a different path to cmdline than newer versions, so a version mismatch could explain why the version of scrapy being executed isn't finding the cmdline module.
There have been some people on SO with a similar problem to you (mostly on Unix systems, where the commands used for diagnosis and repair won't be exactly the same as on Windows), like: scrapy error ImportError: No module named scrapy.cmdline. In your case you would use where scrapy instead of which scrapy which should tell you whether the version of scrapy that appears first in your PATH is the same one installed in your virtual environment.
Good luck!
Alex