Is it possible for someone other than X or Y to know? It's possible Z knows.
A compromised account could have Z watching the inbox, monitoring over IMAP, or receiving a copy of forwarded emails.
Z help desk or some other team could be looking at your screen if you login from work.
If X or Y suspect there is a problem they should change their passwords, account reset information, and only login from trusted computers/
Is this the kind of answer that you were looking for?

to Email Questions!