Luke, you should simply use your best judgement in deciding how to address these. Basically, as a moderator I don't act on these unless the community at large indicates the need to do so. There are a couple of exceptions, such as a post that obviously doesn't meet with the SE guidelines in general, or a post that is generating a lot of discussion instead of answers.
In the first case, if it is obvious that the question or answer doesn't follow our guidelines, I will usually leave a comment asking the person who created the post to modify it to make it fit. If it something that is an abvious and major issue, I will close it and ask them to modify it. If they do so, then I will consider reopening it.
In the situation where there is a lot of discussion, I will usually freeze it for a while and post a comment advising folks to take it to the chat room. Once the freeze is lifted, if the discussions start back up, then I will either extend the freeze or close it to further comments.
Again, just use your best jedgement. If you have already voted on a question then don't worry about the negative feedback it is getting. That is part of the community process. We don't want to create a situation where we start following the herd. If the community as a whole decides to downvote it enough or recommend it for closing, then we act on that, and only that.