Question about scrollTop() & offset().top

I am trying to create a nav bar that is down 500px from the top of the page, but once you scroll down to it, it becomes fixed at the top.

Originally I had the following code:

$(window).scroll(function() {
    if ($(document).scrollTop() > 500) {
        $nav.addClass('fixed');
    }
    else {
        $nav.removeClass('fixed');
    }
});

and

.fixed {
    position: fixed;
    top: 0;
    z-index: 1;
}

However, my page is responsive and the nav bar will not always be 500px down the page. Therefore, I used .offset().top to trigger the fixed class when scrolling reaches the top of my nav bar.

$(window).scroll(function() {
    var $navTop = $nav.offset().top;
    if ($(document).scrollTop() > $navTop) {
        $nav.addClass('fixed');
    }
    else {
        $nav.removeClass('fixed');
    }
});

The issue is that the nav bar flickers terribly when you scroll. I tried replacing .offset() with .position(), which gets rid of the flicker, but it keeps the nav bar fixed for longer than it should when scrolling back up past its original position.