Yet another problem with utf-8

I ran this command in phpmyadmin:-

SHOW FULL COLUMNS FROM 0_sys_prefs

Which gave the following results

Profiling [ Edit inline ] [ Edit ] [ Create PHP code ] [ Refresh ]

+ Options
Field     Type     Collation     Null     Key     Default     Extra     Privileges     Comment    
name     varchar(35)     utf8_unicode_ci     NO     PRI             select,insert,update,references    
category     varchar(30)     utf8_unicode_ci     YES     MUL     NULL        select,insert,update,references    
type     varchar(20)     utf8_unicode_ci     NO                 select,insert,update,references    
length     smallint(6)     NULL    YES         NULL        select,insert,update,references    
value     text     utf8_unicode_ci     NO         NULL        select,insert,update,references

So all seems fine there.

However my hosting company list there only locale as:-

LANG=en_US.UTF-8

And state that all languages will run without a problem.......hmmmmmm

I need a technical argument to refute this - if this is the problem?

The error suggests that it is your 'from' email address of your company that has issues, not the name or email address that you are sending to.  Do you have the full headers of the email that bounced that you can post?  Possibilities are:

- FA is not correctly encoding the from email address in sent mail.
- The remote email server does not support utf8 (very unlikely).

Hi campbell, thanks for replying,

Please note that simply removing the accented 'o' in the company name makes the invoice by email function work perfectly; I can change email addresses to either a server defined email addresses or remote email addresses and all works correctly: Therefore the problem must be the accented 'o' in the company name.

Would you agree? Any help very welcome.

I have tried setting class.mail.inc line 41
from
var $charset = 'iso-8859-1';
to
var $charset = 'utf-8'; //also tried utf8
and
var $charset = 'utf8_unicode_ci';
and
var $charset = 'utf8mb4_unicode_ci';

Not sure what you mean about url encoding?

The problem is in company name in company configuration, not in the email address: I have tried quoted format, this returns '&quot' in the email company name, I've also tried psicopedagógico

I assume:-
HTML Entity (decimal)     ó
HTML Entity (hex)     ó
will have the same result as ó unless you can tell me otherwise?

Ok, I have a partial solution which looks promising:-

From my understanding $name is not encoded so accented char will never work, this post at stackexchange pointed me in the right direction, so changing class.mail.inc line 49 from:-

$this->header = "From: $name <$mail>\n";

to

$this->header = "From: =?UTF-8?B?" .base64_encode($name)."?=";

does work, BUT:-

does NOT include the newline \n so the from $name is followed by the Bcc: bccemailaddress@example.com

So I tried to introduce a new line with the following code:-

$this->header = "From: =?UTF-8?B?" .base64_encode($name)."?=", "\n";
and
$this->header = "From: =?UTF-8?B?" .base64_encode($name)."?=". "\n";

but neither of these work as the from is now the base64 code

If anyone knows a little more php than me can help add a new line please do, will sleep on it.

Working solution:-

Replace class.mail.inc line 49 from:-

$this->header = "From: $name <$mail>\n";

with either:-

$this->header = "From: =?UTF-8?B?" .base64_encode($name) ."?=". "<$mail>\n";

OR

$this->header = "From: =?UTF-8?Q?" .imap_8bit($name) ."?=". "<$mail>\n"; //you will need the imap module enabled

depending if you want base64 or imap

You should now be able to use any diacritical in your company name.

@apmuthu I'm struggling with the  mb_send_mail() & mb_encode_mimeheader(), from my understanding this encodes the whole header: When I got it to work it actually used my servers email address as the from, whereas in the above solution the actual servers email address is hidden and the company email address is shown as per FA functionality.  Can you help?

We will have to consult Janusz with this. I will.

/Joe

RFC 5336 - SMTP Extension for Internationalized Email Addresses.

IMAP
====

if (extension_loaded('imap')) {

....
....

}

Looking at this Stack Overflow answer I would suggest adding the 'Content-Type: text/plain; charset=UTF-8' header to the 'body' parameter in the call to the mail function.

It seems from this post that support for unicode email addresses in the local part is relatively new. e.g. Postfix supported it in 2014.  It very much depends on what email software your server (or ISP) has installed to support the sending of email.  I suspect that if you use gmail, or google apps for business then you wouldn't have any problems (assuming the content header is set correctly).  Using Postfix for example you would need to ensure that version 3.0 or greater is installed.  In the debian world this would be Debian Stretch.  Wheezy and Jessie are both on Postfix 2.x.  You can find further information on SMTPUTF8 support and links to relevant RFC on the Postfix SMTPUFT8 page.

To be safe, I would suggest that all email addresses in *your* organization use the 128 lowercase ascii characters as required under the old standard.

Sorry, I mean the additional_headers parameter, not the body.  From the php manual:

"Note: When sending mail, the mail must contain a From header. This can be set with the additional_headers parameter, or a default can be set in php.ini. Failing to do this will result in an error message similar to Warning: mail(): "sendmail_from" not set in php.ini or custom "From:" header missing. The From header sets also Return-Path under Windows."

The content type header should be honored if the mail server supports it.