What's the point of using text(), when there is already a <p></p> appended to the image?

Paragraph tags (p) don't have an attribute of alt so that's likely why it didn't work.

text() is a jQuery function that, when given an argument, sets the content of an element. So your assumption is correct. :)

we are just assigning an HTML selector to that value?

Correct. To be more specific, we are creating a new element (a paragraph, i.e. $caption) and then, after we append() it, we use the text() function to assign some content to that element ($caption).

In addition, wouldn't it be better practice if we first got the alt attribute and set the caption, and then show the overlay? Or at least less confusing...

If we did that we would have to declare the variable, captionText, outside of the scope of the click event, probably where $caption is declared. The big issue with doing this is that the captionText is dependent upon the image being clicked so it wouldn't work as there would be no context to assign it to. So while it may seem, at a glance, to be less confusing, we would not attain the goal that we wanted.