Welcome to the Treehouse Community

The Treehouse Community is a meeting place for developers, designers, and programmers of all backgrounds and skill levels to get support. Collaborate here on code errors or bugs that you need feedback on, or asking for an extra set of eyes on your latest project. Join thousands of Treehouse students and alumni in the community today. (Note: Only Treehouse students can comment or ask questions, but non-students are welcome to browse our conversations.)

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and a supportive community. Start your free trial today.

JavaScript

Dennis Brown
Dennis Brown
28,742 Points

Data Retention with Node.js and/or Express.js

Hello all!

I have been having a lot of fun with my own personal projects with Node.js, but one thing that has always held me up is dealing with data.

Of course we can pass data with the response and request, and that's awesome, but that data is lost after a new response is sent.

So what if you want to retain some of the data? For instance, you pull a small list of employee IDs, and instead of pulling that same data over and over with each request, you memoize it by attaching it to a value that doesn't go away.

So long story short, is there anything in Node.js or Express.js that allows you to save objects by default? It looks like I can do this with express-session, but I'm very curious what are some well-know practices others have used for this issue?

TIA!

1 Answer

Igor Yamshchykov
Igor Yamshchykov
24,397 Points

Hello Dennis, Indeed you can use session storage or in memory storage (with global variables) to store some data in your node.js app, but this is probably not the best way to that, the reasons are:

  1. In case you are storing data in memory or session you might run into situation when your server is down for some reason and in case this data was modified you won't be able to retrieve those changes.
  2. There also times when you would like your node.js application to work in cluster mode for example ( this is when you run multiple instances of your node.js server and make them process the requests separately, this is like a load balancing ) in this case data that is stored in one app won't be accessible in another, so you'll be in trouble again if you'll need the same data. So the solution I might suggest for you is using Redis or etcd for storing that data, Redis is a simple key - value storage that is often used for caching data. You'll still need to retrieve that data each time your node.js app is processing the request, but as Redis is a key value storage it's extremely fast to access that data and in this case you won't run into troubles when dealing with multiple instances of app or server down. The only thing that might happen is Redis will be down, but in this case you always can retrieve your data from the db again.

I'll try to describe the basic flow of usage for you, so:

  1. Someone performs the request at this stage Redis storage is empty, you need to check if Redis has a key that you're looking for, if not than query the database and after that create and key with value in redis, storing the data you've retrieved from db after that proceed with your request.
  2. During second request you'll check the Redis storage for the key you're looking for, retrieve data from Redis and proceed with your request.

The thing you must be aware of is that if you update data in your db, you also need to either clean Redis or update data in Redis, because otherwise it will return old cached data even though it was updated in your database.

Hope this will help you.

P. S. Also be aware that use of caching for everything is not a good idea, be sure that you use it only when it's really needed ( because even if you'll have 100 simultaneous requests per second your node app will be able to handle it without caching), so use it only in cases when you really have a high load.

Dennis Brown
Dennis Brown
28,742 Points

Very cool!

Thank you very much for this! I had completely forgot about user-specific storage, but thank you for bringing that up!

So I'm guessing there isn't a Node.js built-in native solution for such needs? I'm personally partial to the express-session since posting, but I wanted to research the native possibilities. From what you've said, it sounds like you should always stick to local storage, and databases, yes?

(If so, it sounds like a great excuse to start working with GraphQL more). ;)

Igor Yamshchykov
Igor Yamshchykov
24,397 Points

Maybe there is some kind of native solution, but I personally am not aware of that, I simply use the approach I've described, because it's applicable not only to Node.js, can be used with any server-side app I guess. I think using local storages or db will be quite enough

Dennis Brown
Dennis Brown
28,742 Points

Thanks Igor. I appreciate all of your time. :)