Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial![](https://ecs-static.teamtreehouse.com/assets/views/marketing/shared/community-banner-white-47072046c51352fe6a69f5e691ff5700b28bb11d45197d7bdf066d9ea3f72d0c.webp)
![Radha Bhambwani](https://uploads.teamtreehouse.com/production/profile-photos/1987782/micro_330738_10150348896985240_506780239_9962329_7752608_o.jpg)
Radha Bhambwani
10,182 Pointsnode js and mongodb
I'm new to node js and mongodb. Just testing out some ideas so I am trying to create a web scraper that adds data to a database in mongodb. I have a script that connects to mongodb and dynamically adds data to a database through a node js script. The script runs without any errors, but when I fire up the mongo shell to and run db.posts.find(), I get 0 records. Here's my script:
var MongoClient = require('mongodb').MongoClient;
//requiring the module so we can access them later on
var request = require("request");
var cheerio = require("cheerio");
MongoClient.connect('mongodb://127.0.0.1:27017/makeupdb', function (err, db) {
if (err) {
throw err;
}
else {
console.log("successfully connected to the database");
//define url to download
var url = "http://www.nyxcosmetics.ca/en_CA/highlight-contour";
var prodList = [];
var priceList = [];
var products = [];
request(url, function(error, response, body) {
if(!error) {
//load page into cheerio
var $ = cheerio.load(body);
$(".product_tile_wrapper").each(function(i, elem) {
prodList[i] = $(this).find($(".product_name")).attr("title");
priceList[i] = $(this).find($(".product_price")).attr("data-pricevalue");
});
prodList.join(', ');
for(var i = 0; i < prodList.length; i++) {
var prod = {
name: prodList[i],
price: priceList[i]
};
products.push(prod);
}
console.log(products);
//insert the prods into the database
//in the 'posts' collection
var collection = db.collection('posts');
collection.insert(products);
console.log("products inserted into posts collection");
//Locate all the entries using find
collection.find().toArray(function(err, results) {
console.log(results);
});
} else {
console.log("We've encountered an error!")
}
});
}
db.close();
});
![Radha Bhambwani](https://uploads.teamtreehouse.com/production/profile-photos/1987782/micro_330738_10150348896985240_506780239_9962329_7752608_o.jpg)
Radha Bhambwani
10,182 PointsThe data is not in JSON format, its in html. product_name and product_price are classes inside inside the html of the url, hence the dot usage.
Jesus Mendoza
23,289 PointsJesus Mendoza
23,289 PointsI have never tried web scrappling beforfe but I think you are using the wrong selectors to find the product_name and product_price. Why are you using a dot in front? Is the data comming in a json file right?