Python Scraping Data From the Web A World Full of Spiders Everyone Loves Charlotte

Tatiane Lima
Tatiane Lima
6,738 Points

I getting this error: ImportError: No module named request

I'm trying to run this code:

from urllib.request import urlopen
from bs4 import BeautifulSoup

import re

site_links = []


def internal_links(linkURL):
    follow_html = urlopen('https://treehouse-projects.github.io/horse-land/{}'
                          .format(linkURL))
    soup = BeautifulSoup(follow_html, 'html.parser')
    return soup.find('a', href=re.compile('(.html)$'))


def external_links(linkURL):
    follow_html = urlopen('https://{}'.format(linkURL))
    soup = BeautifulSoup(follow_html, 'html.parser')
    return soup.find('a', href=re.compile('^.(https)'))


if __name__ == '__main__':
    urls = external_links('treehouse-projects.github.io/horse-land/index.html')
    while len(urls) > 0:
        page = urls.attrs['href']
        if page not in site_links:
            site_links.append(page)
            print(page)
            print('\n===================================\n')
            urls = external_links(page)
        else:
            break

But i'm getting this error:

/.../scraping_data_from_the_web/soup_follow_scraper.py"
Traceback (most recent call last):
  File "/.../scraping_data_from_the_web/soup_follow_scraper.py", line 1, in <module>
    from urllib.request import urlopen
ImportError: No module named request

Process finished with exit code 1

1 Answer

adrian miranda
adrian miranda
11,468 Points

This works for me with python3. Both on my home PC and when I'm in workspaces. Is it possible you are running this somewhere else, and perhaps getting python2?

That's the error I get when I try to run that line with python2.

Tatiane Lima
Tatiane Lima
6,738 Points

You were right! Thanks for help me.