Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial
Doug Hawkinson
Full Stack JavaScript Techdegree Student 25,073 PointsCan I retrieve an API endpoint with a fs.readFileSync()? If not, I think I need a synchronous solution.
I am building out an API for use with our BnB guests with the included route file, I am trying to retrieve and display restaurant selections for them to choose from utilizing the Foursquare Places API. I need data from two layers in the API. The first layer brings back base information (genre, name, address, distance (from the BnB) and a field named detailsURL (a link to the Foursquare Venue-specific endpoint). I build a 'places' array with the first pass data and empty fields to be filled by the second pass (phone, price indicator, rating, hours [an array of hours] and canonicalURL (a link to the Foursquare Venue webpage). Looping through the first-pass of the 'places' array (for (pos in places)), I am trying to use fs.readFileSync(uri) to build out the unpopulated fields in the 'places' array. After I get that working I will render the results with a React component which will have active links to the API pages.
I chose fs.readFileSync(uri) because from everything I am discovering, I cannot use axios or other asynchronous methods to bring back my endpoints -- because I am doing it from inside a loop, which is synchronous by nature. (I have proven to myself, using both traditional promises and async / await, trying multiple syntactical approaches that, at least I, cannot do it asynchronously). so I finally decided to try fs.readFileSync(uri).
However I am getting the following error message:
{ Error: ENOENT: no such file or directory, open 'https://api.foursquare.com/v2/venues/4b451a52f964a520b70426e3?client_id=<clientId>&client_secret=<clientSecret>&v=20190302'
at Object.openSync (fs.js:439:3)
at Object.readFileSync (fs.js:344:35)
at goExplore.then.venues (/Users/doug5solas/training/treehouse/fsjsProjects/TH12-Capstone/routes/api/restaurants.js:103:39)
at process._tickCallback (internal/process/next_tick.js:68:7)
errno: -2,
syscall: 'open',
code: 'ENOENT',
path:
'https://api.foursquare.com/v2/venues/4b451a52f964a520b70426e3?client_id=<clientId>&client_secret=<clientSecret>&v=20190302' }
The odd part about it is I can take the URI 'https://api.foursquare.com/v2/venues/4b451a52f964a520b70426e3?client_id=<clientId>&client_secret=<clientSecret>&v=20190302' and paste it into the browser Address Field and it returns exactly the API endpoint in the JSON format that I am looking for. (Clearly the clientId and clientSecret in the example are placeholders). So, I know there is nothing malformed in the href. But that coupled with the text of the error message leads me to believe that fs.readFileSync(uri) is NOT meant to retrieve an API endpoint, only actual file data. If that is true, then I either need another synchronous method of going after an endpoint or a way to do it asynchronously (remember I am in a loop) that will resolve to the data and not a promise.
Here is the relevant code:
const URI = `${keys.foursquareBaseURL}${keys.foursquareEndpoint}${params}${auth}${vDate}`;
const goExplore = async () => {
try {
const results = await axios(URI);
return (venues = results.data.response.venues);
} catch (e1) {
console.log(e1);
}
};
const goDetails = async (URL) => {
try {
const results = await axios(URL);
return (venue = results.data.response.venue);
} catch (e2) {
console.log(e2);
}
};
//*** other code ***
// @route GET api/restaurants/search
// @desc GET local restaurant venues
// @access Private
router.get('/search', passport.authenticate('jwt', {session:false }), async (req, res) => {
// parse restaurant data
const currentUser = req.user._id;
goExplore()
.then(venues => {
// venues retrieved -- parse them
// hang on to this until you are sure you do not need it
// console.log('RESTAURANTS', venues)
const places = [];
venues.forEach(pos => {
places.push({
venueId: pos.id,
genre: pos.categories[0].shortName,
name: pos.name,
formattedAddress: pos.location.formattedAddress,
distance: Math.round((pos.location.distance * mTmConvert * 10) / 10), // rounds to 1 decimal point
phone: '',
price: '',
rating: '',
hours: [],
canonicalURL: '',
detailsURL: `${keys.foursquareVenueDetailsURL}${pos.id}?client_id=${keys.foursquareClientID}&client_secret=${keys.foursquareClientSecret}${vDate}`
});
});
console.log('PLACES1',places);
// places array built, but canonicalURL et al are empty
// Now you need to retrieve the details endpoint and grab the canonicalURL
// NOTE: canonicalURL is the link to the foursquare detail data on venues
// it is provided by venue managers and may be empty
//places.forEach((pos) => {
let pos;
for (pos in places) {
let venue;
let uri = `${places[pos].detailsURL}`;
venue = JSON.parse(fs.readFileSync(uri));
console.log('VENUE',venue);
places[pos].phone = (venue.contact ? venue.contact.formattedPhone : '');
places[pos].price = (venue.price ? venue.price.message : '');
places[pos].rating = (venue.rating ? venue.rating : '');
places[pos].hours = (venue.hours ? venue.hours.timeframes : '');
places[pos].canonicalURL = venue.canonicalUrl;
}
//});
console.log('PLACES2',places);
})
.catch(e3 => {
console.log(e3);
});
}
);
Can anyone point me in the right direction?
3 Answers
Brendan Whiting
Front End Web Development Techdegree Graduate 84,735 PointsThe file system ('fs') module is for reading and writing files on the same machine as this server is running on. It's not for retreiving data from an external url. The error you're getting, no such file or directory, is because it's trying to look in your machine for a file with that name/location which doesn't exist there.
I recommend going back to using fetch/axios. If you have a list of things, and you want to make some request for all of them, one strategy is to use .map() and Promise.all(). Here's an example where I'm using the randomuser API to fetch some data for 3 users.
- I have a list of fake username seed data
- I have a helper function,
fetchUserDatathat callsaxiosand then grabs the nested fields off the response (data.data.results[0].name) - I call .map() on the list of randomUsers, and for each user, pass it into the
fetchUserdatahelper function. This will return a list of promises. I can then pass that into Promise.all which will wait for all the promises to resolve before it resolves itself. I'll end up with an array of all the users' data.
const axios = require('axios');
const randomUsers = ['foo', 'bar', 'foobar']
const fetchUserData = async (user) => {
const data = await axios(`https://randomuser.me/api/?seed=${user}`)
return data.data.results[0].name;
}
Promise.all(randomUsers.map(user => fetchUserData(user)))
.then(names => console.log(names))
/*
[
{ title: 'mrs', first: 'april', last: 'mitchelle' },
{ title: 'ms', first: 'ilona', last: 'niskanen' },
{ title: 'miss', first: 'britney', last: 'sims' }
]
*/
Doug Hawkinson
Full Stack JavaScript Techdegree Student 25,073 PointsFor anyone following this, here is the ultimate solution I employed:
const URI = `${keys.foursquareBaseURL}${keys.foursquareEndpoint}${params}${auth}${vDate}`;
const fetchVenues = async () => {
try {
const results = await axios(URI);
return (venues = results.data.response.venues);
} catch (e1) {
console.log(e1);
}
};
const fetchDetails = async (url, places, i) => {
const data = await axios(url)
venue = data.data.response.venue;
places[i].phone = ( (venue.contact) ? venue.contact.formattedPhone : '' );
places[i].price = ( venue.price ? venue.price.message : '' );
places[i].rating = ( venue.rating ? venue.rating : '' );
places[i].hours = ( venue.popular ? venue.popular.timeframes : '' );
places[i].canonicalURL = venue.canonicalUrl;
return
};
// *** Other code ***
// @route GET api/restaurants/search
// @desc GET local restaurant venues
// @access Private
router.get('/search', passport.authenticate('jwt', {session:false }), async (req, res) => {
// parse restaurant data
const currentUser = req.user._id;
const places = [];
await fetchVenues().then(venues => {
// venues retrieved -- parse them
venues.forEach(venue => {
places.push({
venueId: venue.id,
genre: venue.categories[0].shortName,
name: venue.name,
formattedAddress: venue.location.formattedAddress,
distance: Math.round((venue.location.distance * mTmConvert * 10) / 10), // rounds to 1 decimal point
phone: '',
price: '',
rating: '',
hours: [],
canonicalURL: ''
});
});
})
.catch(e3 => {
console.log(e3);
});
// The places array is now populated with the base data for the venues (the data available from the 'search' query)
// Now retrieve the details endpoint for each venue and grab the:
// phone
// price
// hours
// canonicalURL
// NOTE: canonicalURL is the link to the foursquare detail data on venues
// it is provided by venue managers and may be empty or incomplete
const placesFullyPop = async () => {
let i =0;
for (const item of places) {
url = `${keys.foursquareVenueDetailsURL}${item.venueId}?client_id=${keys.foursquareClientID}&client_secret=${keys.foursquareClientSecret}${vDate}`;
await fetchDetails(url, places, i);
i++;
}
}
await placesFullyPop();
//console.log('PLACES', places)
//
// **************************************************************************
// ***** The fully populated 'places' array is available in this scope. *****
// ***** This is where I will use places to interface with the React UI.*****
// **************************************************************************
//
});
This seems to work just fine. The 'places' array is completely populated and available within the scope I need it. We will see what happens when I hook up the React.
Brendan Whiting
Front End Web Development Techdegree Graduate 84,735 PointsGood work. One thing that you can look at is pure functions, it's a function with only input and output, no side effects. It's not always possible to write code this way, but it's much easier to reason about and much easier to test (also, write tests!). Side effects can also cause some nasty, difficult bugs.
So for example, this line is returning a value but also doing an assignment on the venues variable outside of this function's scope:
return (venues = results.data.response.venues);
This function makes changes to the places variable that's passed in but doesn't return anything:
const fetchDetails = async (url, places, i) => {
const data = await axios(url)
venue = data.data.response.venue;
places[i].phone = ( (venue.contact) ? venue.contact.formattedPhone : '' );
places[i].price = ( venue.price ? venue.price.message : '' );
places[i].rating = ( venue.rating ? venue.rating : '' );
places[i].hours = ( venue.popular ? venue.popular.timeframes : '' );
places[i].canonicalURL = venue.canonicalUrl;
return
};
Also, there's an opportunity to make requests concurrently where you're doing them in a sequential, blocking way right now. This function is looping through each item in places, and waiting for the response before it fires off the next request. It might not matter for a small number of places, but if you have a long list it could become a big performance bottleneck. This is where the strategy of using .map() to make a list of promises and then Promise.all() could come in handy.
const placesFullyPop = async () => {
let i =0;
for (const item of places) {
url = `${keys.foursquareVenueDetailsURL}${item.venueId}?client_id=${keys.foursquareClientID}&client_secret=${keys.foursquareClientSecret}${vDate}`;
await fetchDetails(url, places, i);
i++;
}
}
Doug Hawkinson
Full Stack JavaScript Techdegree Student 25,073 PointsYou know Brendan, I wanted to use array.map(). I just could not interpolate an answer that didn't return syntax errors from your example, rookie that I am. So, I took the fall back position of using for(const item of places). I'd love to see your suggestion of how to make that work.
Doug Hawkinson
Full Stack JavaScript Techdegree Student 25,073 PointsDoug Hawkinson
Full Stack JavaScript Techdegree Student 25,073 PointsBrendan:
Thank you for your comments. It was not clear (to me) that fs is for retrieving local files. If it is stated in the documentation, I missed it. However, I began to suspect something was up based on the error message. Thanks for the clarification.
I will look at the .map and Promise.all() suggestions and try to interpolate and apply them to my situation. I appreciate your patience. You have worked with me in similar situations in the past and hopefully I am getting closer to getting this pounded into my skull. The one things that has me a bit mystified is in your example you are mapping one value (names). I need to map 5 values (phone, price, rating, hours[an array], canonicalURL). But, I'll just wrestle with that.