Am I crazy for not fully using frameworks like Bootstrap and Foundation?

Honestly, I think it goes both ways. It depends on the project and or agency. I personally use bootstrap on a majority of projects on the side. On the other hand, I just interviewed with a company yesterday called The Nerdery, and they do not use frameworks like bootstrap or foundation. I believe frameworks should be used as a tool, but its imperative that you understand how to write your own code and create projects from scratch as well.

I'm building my first website from scratch using plain HTML, CSS, and PHP. As someone who just recently heard of such frameworks, I think they would have saved me loads of time in dealing with a lot of issues, namely spacing and columns. But I'm glad I didn't use them because I want to struggle through these problems and figure them out for myself, with occasional help from the forums when I get really stuck and need to be pointed in the right direction. Maybe in the future, I'll use frameworks because I'll have "gotten the idea" by then of what goes into building sites and why this and that's important, but I can definitely see from your perspective. If you have a lot of experience and have reached a certain level, just take what you need from the frameworks and nothing more - I can only imagine using them for convenience myself.

All depends on what your looking to build. I wouldn't use a CSS framework for a static site. However, if you are building a larger project that will have more pages added in the future a framework is the way to go. I just started using sass and would recommend Bourbon Neat for smaller projects. Larger projects with more complexion I tend to use just the grid portion of Foundation.