Welcome to the Treehouse Community
The Treehouse Community is a meeting place for developers, designers, and programmers of all backgrounds and skill levels to get support. Collaborate here on code errors or bugs that you need feedback on, or asking for an extra set of eyes on your latest project. Join thousands of Treehouse students and alumni in the community today. (Note: Only Treehouse students can comment or ask questions, but non-students are welcome to browse our conversations.)
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and a supportive community. Start your free trial today.

Radha Bhambwani
10,182 Pointsnode js and mongodb
I'm new to node js and mongodb. Just testing out some ideas so I am trying to create a web scraper that adds data to a database in mongodb. I have a script that connects to mongodb and dynamically adds data to a database through a node js script. The script runs without any errors, but when I fire up the mongo shell to and run db.posts.find(), I get 0 records. Here's my script:
var MongoClient = require('mongodb').MongoClient;
//requiring the module so we can access them later on
var request = require("request");
var cheerio = require("cheerio");
MongoClient.connect('mongodb://127.0.0.1:27017/makeupdb', function (err, db) {
if (err) {
throw err;
}
else {
console.log("successfully connected to the database");
//define url to download
var url = "http://www.nyxcosmetics.ca/en_CA/highlight-contour";
var prodList = [];
var priceList = [];
var products = [];
request(url, function(error, response, body) {
if(!error) {
//load page into cheerio
var $ = cheerio.load(body);
$(".product_tile_wrapper").each(function(i, elem) {
prodList[i] = $(this).find($(".product_name")).attr("title");
priceList[i] = $(this).find($(".product_price")).attr("data-pricevalue");
});
prodList.join(', ');
for(var i = 0; i < prodList.length; i++) {
var prod = {
name: prodList[i],
price: priceList[i]
};
products.push(prod);
}
console.log(products);
//insert the prods into the database
//in the 'posts' collection
var collection = db.collection('posts');
collection.insert(products);
console.log("products inserted into posts collection");
//Locate all the entries using find
collection.find().toArray(function(err, results) {
console.log(results);
});
} else {
console.log("We've encountered an error!")
}
});
}
db.close();
});

Radha Bhambwani
10,182 PointsThe data is not in JSON format, its in html. product_name and product_price are classes inside inside the html of the url, hence the dot usage.
Jesus Mendoza
23,287 PointsJesus Mendoza
23,287 PointsI have never tried web scrappling beforfe but I think you are using the wrong selectors to find the product_name and product_price. Why are you using a dot in front? Is the data comming in a json file right?