Route by Sender Transport Agent destroys Umlauts
On both Systems where we have installed the Route by Sender Transport Agent, german Umlaut characters are made to unreadable caracters when sending mails between different tenants.
Sending $äöüß€ results in receiving Â§$Ã¤Ã¶Ã¼ÃŸâ‚¬
Mails sended from or to externel addresses are not affected, only mails which are handled by the transport agent.
We now had to uninstall the agent - customers complained about this.
Are there any settings which should have been made?
The CloudPanel Transport Agent doesn't actually modify emails, it only changes the RoutingDomain. Chances are it is the smart host that is causing this because since the transport agent is routing the mail out to a send connector, it will contain a winmail.dat file instead of the actual email. Some smart hosts support decoding this winmail.dat file and some do not.
When it is sent to an external user then Exchange recognizes that the domain isn't local and doesn't use the winmail.dat when forwarded to the smart host.
What smart host are you using?
I realized that mails are sent to a user with another mail domain on the same Exchange server and to external receipients (To and Cc) are receiced correct from the external receipient and the problem is only at the other user on the Exchange server.
So it's possible the smarthost has a problem when processing winmail.dat files.
We are using NoSpamProxy mail gateway.
I opened a ticket at the NoSpamProxy support, the information concerning winmail.dat can be useful for them.
@pncsupport I don't really have any experience with that smart host but typically they shouldn't be processing anything in the email. However, since it contains this winmail.dat file it may be throwing it off. I would like to see what the mail looks like going into the smart host and what it looks like coming out of the smart host so we can see if there are any changes.
I just got an answer from NoSpamProxy, I'll try to translate, they say:
The problem is, that the mail comes to the smarthost as winmail.dat and not as HTML or plain text. The winmail.dat doesn't contain information which characterset should be used - so they have to guess and, depending on the content of the mail, a wrong characterset can be used.
Exchange Server should be configured to not use winmail.dat.
At least that explains, why mails containing only umlauts (öäüßÜÄÖ) are transmitted correct and mails, additional containing the Euro-caharacter € are transmitted wrong.
When installing the Exchange servers we configured them to avoid winmail.dat by setting
Set-RemoteDomain -Identity Default -TNEFEnabled $false
It's possible to add more remote domains - but I can't define a remote domain for mails to the smarthost based on smarthosts IP or so.
I dont know why this setting is not effective when sending mails to the smarthost, I think, Exchange server will not handle his own accepted domains as remote domains.
I found there are more people with this problem (Link below, don't know, if it will be removed by forum).
The problem arises when mails are routed through somewhere to classify/archive/redirect mails or - as in our case - to scan it for spam or malware.
In the article below it's mentioned the transport agent should convert the mails to HTML/plain text.
This is already done by our smarthost - with limitations regarding the missing characterset/codepage info in winmail.dat.
(I don't know the winmail.dat format - do you know, if there is really no characterset information?)
I will ask them if it's possible to convert the mails to be able to scan it but after scanning to forward the original winmail.dat.
I'm not sure it can happen that receiving users not using Outlook can get problems.
But I think rather not, without the smarthost already winmail.dat's are transferred between users of different local maildomains.
I dont think that can be resolved based on my research which is why you have to use a smart host that is capable of decoding the winmail.dat file for processing. For example Mimecast and Mailprotector dont have this issue but we have seen Proodpoint and some others with this issue. The problem is Exchange is treating it as an internal user because it is an internal user but we are forcing it to send out. I don't see a way for the transport agent to change the type during processing
After some mails exchanged with the people from NoSpamProxy they told me, they have found additional information in the TNEF specs and there will be a solution.
So I'm waiting for that.
@pncsupport That is good to hear. I assume maybe they were not decoding the winmail.dat file properly or they are adding the feature to decode it?
I think, they had not decoded it 100% correct, maybe they tested not deeply enough: I guess, they only tested mails with umlauts (öäüßÜÄÖ) only without the Euro character €. But the presence of this character leads to different coding of winmail.dat.
@pncsupport Ah. Well that is good to know. Let us know when they get it fixed and if it works. Thank you!
Update: It's fixed now in NoSpamProxy Version 13.1.20071.1529.
Decoding of winmail.dat was not decoded correct.
@pncsupport Perfect! It is good to know other smart hosts that support this because some simply just don't support decoding of winmail.dat files at all. I wish Microsoft provided a way to overcome this but what we are doing isn't Microsoft's "norm". Thank you for updating us!