Don't fetch if we don't even have a inReplyTo.