A transactional email is essentially an email that is triggered based on a specific action (or inaction) that someone has taken.
For example, say you sent an opt-in reminder email to a contact and they clicked on the confirmation link within that email. Clicking that link would trigger another (transactional) email, which would respond thanking that client for their confirmation.
Or, if you sent an email to a client which had important, time-sensitive information included and the client did not open it within a certain length of time, you could make that inaction a trigger to send a follow-up (transactional) email to make sure your client received the important information.