Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial
Andrew Cottage
20,718 PointsTreehouse Please Offer a Web-scraping Deep Dive
Hello Treehouse staff.
I think it would be most beneficial if you offered a web-scraping class that shows how to do it. I have been using Nokogiri in Ruby to do it, but information even on the web can be confusing. It would be nice to learn from you guys.
6 Answers
Andrew Chalkley
Treehouse Guest TeacherHi Andrew Cottage,
Whilst we've got no immediate plans for a Deep Dive in web scraping, I thought it would be good to do a "shallow paddle" (that's not a real thing).
In this video I use a Rake task to kick off the scraping process. I use the Rubygem Nokogiri to create a simple spider and scraper.
Hope this helps!
Regards
Andrew
Matt Campbell
9,767 PointsWhere'd the beard come from Chalkley???
James Barnett
39,199 PointsIt's like Jim Hoskins junior
Matt Campbell
9,767 PointsExcellent likeness. Well, just got to watch the Hoskins - Chalkley big day out video. :D
James Barnett
39,199 PointsThey should really do a sequel to that now that they've got matching beards.
Matt Campbell
9,767 PointsI will pay to see that.
Jim Hoskins
Treehouse Guest TeacherAndrew Chalkley has a way to go before he can match my beard
Andrew Chalkley
Treehouse Guest TeacherI don't know where it came from. It just turned up one day. I think it's a side effect of living in Portland.
izelnakri
25,803 PointsIn case if any of you find this video too complicated, web scraping with node + cheerio module is much easier and natural. It uses the same Jquery syntax every front-end developer is familiar with.
James Barnett
39,199 PointsJQuery syntax that sounds pretty exciting. I love me some JQuery.
Andrew Chalkley
Treehouse Guest TeacherI was tempted to do that + Jim Hoskins was saying to do it too. But the request was for Ruby / Nokogiri :)
Matt Campbell
9,767 PointsDo one for jQuery and normal sites Andrew Chalkley :)
Natacha S
11,561 PointsGreat video ! You made it sound so easy !
Andrew Chalkley
Treehouse Guest TeacherGlad you liked it!
Andrew Cottage
20,718 PointsThanks Andrew Chalkley . This video answered so many questions that I had when it comes to incorporating a nokogiri scrape inside of a rails project. Keep up the great work!
Andrew Chalkley
Treehouse Guest TeacherI'm glad! Keep pushing yourself!
Joshua Briley
Courses Plus Student 24,645 PointsThis was great. Thanks for sharing this.