This happens semi-frequently to me and others - we reply to a user who
has left the room. Sometimes this is useful (for example, bringing up a
previous topic) but in most cases this is accidental, and almost
guaranteed to happen if you turned off join/leave events.
So I added a reminder when replying - which manifests as a small label
in the chatbar - that the user has left and can't be notified of your
reply.