First, make sure that there is only one account set up in Outlook for each
message.
Second, edit the account so that the e-mails are deleted from the server as
they are downloaded.
Should fix.