Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial
Jorge Guzman
Courses Plus Student 21,390 PointsHow to convert websites into JSON format for building a blog reader app?
Hi guys,
I am trying to build a blog reader, similar to the example in the training, but I do not know how to take any website (blog, rss or whatever) and use it in my app.
Can anybody please guide me through this?
Many thanks in advance,
Jorge
6 Answers
Roger Lüchinger
11,403 PointsHi Jorge and John
Under normal circumstances, I would assume that you use a CMS for your website. If that's the case then there's a good chance your CMS will already have built-in support for outputting your content in JSON format. With some CMS you may have to install a plug-in first, but searching your CMS' documentation for "JSON" should answer your question in any case.
If you cannot output JSON data directly (either because you don't use a CMS, your CMS does not support JSON-Feeds or because you are fetching data from a website you don't control) you are still not out of luck. You could write a so called «scraper» that gets the html page, extracts the content from the page and outputs it as JSON. If PHP is your weapon of choice, you could look at http://php.net/manual/en/class.domxpath.php for getting started with scraping a website.
But nowadays, in the age of "{insert random web-based functionality}-as-a-service", there's an even more convenient way for you to get your web content in JSON format: Have a look at http://www.kimonolabs.com/ - as they are still in beta, they are offering a free plan at the moment, so go there any play around with it. This allows you to create a JSON or XML API from any HTML page on the web in a few minutes.
Roger
P.S. I am in no way affiliated with Kimono Labs and have no benefit of recommending them. I am sure there are other services like that... I just happened to find this a few months ago and have been using it for several projects myself without a reason to complain.
P.P.S. HERE BE DRAGONS: If you scrape content from a website to create an API, this only works as long as the structure of the website does not change. As soon as the structure (not the content, I mean the nesting of HTML elements) changes, the parser may need to be adapted to the new structure. Because of this, built-in APIs are always preferrable to scraping, especially if your application is business-critical. you have been warned ;-)
Lei zheng
7,990 Pointsthank you so much. very educational. exactly what I am looking for.
Jorge Guzman
Courses Plus Student 21,390 PointsHi John,
Oh that's a pity, hopefully somebody will reply. I am googling this thing, if I find something I will share it with you.
Cheers.
Jorge Guzman
Courses Plus Student 21,390 PointsHi Roger,
Wow, many thanks for showing the big picture!. I will explore the options you mention, hopefully I will make it, since I am just a beginner in this programming matters and my background is not technology, but I am really enjoying this learning.
((John, think this is a good answer for us to explore.))
Cheers,
Jorge
Jorge Guzman
Courses Plus Student 21,390 PointsThanks Roger,
I checked the Kimono tool and seems to be a good option.
Cheers man,
Jorge
John Carr
Courses Plus Student 1,984 PointsThanks Roger, Very clear explanation from you. Have you used JSONVIEW extension in chrome to view the json data on a blog like tree houses project? That was what my question was and as to how i would view this on other blogs. In the project Amit got the json info from this link , he also recommended using Jsonview to clean up the info. http://blog.teamtreehouse.com/api/get_recent_summary/ Have you any view on this or how would i find a similar blog to practice on like this.
Erion Vlada
13,496 PointsIt might be a bit late but may also come in handy to someone else looking into this. Its called scrapping, web harvesting etc. The easiest way to do it is use a service called import.io, that is also their website. It is completely free and is very easy to use.
If you want to go down the manual route then you can use a python library called scrappy, very powerful but more work required to get it working.
John Carr
Courses Plus Student 1,984 PointsJohn Carr
Courses Plus Student 1,984 PointsHi jorge i had posted the same question, Unfortunatley i got no response to it.