UnicodeEncodeError: 'ascii' codec can't encode character

[plamer]

Version information

1.0b

Steps to replicate

1. After checking the log I found some errors as the following:

Actual result

2018-01-05 13:20:09,371 [8766] DEBUG Got line: User: dreamtu
2018-01-05 13:20:09,371 [8766] DEBUG Got line: Content-length: 10936
2018-01-05 13:20:09,372 [8766] ERROR Error while processing request from: ('91.215.216.5', 56059)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/SocketServer.py", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib64/python2.7/SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__
    self.handle()
  File "/usr/lib/python2.7/site-packages/oa/server.py", line 45, in handle
    COMMANDS[command.upper()](self.rfile, self.wfile, self.server)
  File "/usr/lib/python2.7/site-packages/oa/protocol/base.py", line 30, in __init__
    self.get_and_handle()
  File "/usr/lib/python2.7/site-packages/oa/protocol/base.py", line 88, in get_and_handle
    message = oa.message.Message(self.ruleset.ctxt, message)
  File "/usr/lib/python2.7/site-packages/oa/message.py", line 140, in __init__
    self.msg = email.message_from_string(self.raw_msg)
  File "/usr/lib64/python2.7/email/__init__.py", line 57, in message_from_string
    return Parser(*args, **kws).parsestr(s)
  File "/usr/lib64/python2.7/email/parser.py", line 82, in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 2382: ordinal not in range(128)

Expected result

Parse the message and check it for spam.

Other notes

Any idea what can be causing this behavior?

[alexkiro]

It looks like for some reason that message is un-parsable by the email library and crashing. Python 3 usually handles these Unicode issues better than Python 2, but we really need to support both versions.

@plamer would it be possible for you to send me (privately) the problematic message? That could speed up the fix significantly :)

[plamer]

@alexkiro I'm sorry but I don't have it - it was from a client and I only have the exim/oad logs.

[geanxyz]

Hi,
have the same problem installing the requirements/python3.txt

`pip install -r requirements/python3.txt
Collecting dnspython3==1.12.0 (from -r requirements/python3.txt (line 1))
Using cached dnspython3-1.12.0.zip
Collecting py3dns==3.1.0 (from -r requirements/python3.txt (line 2))
Using cached py3dns-3.1.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-r83haI/py3dns/setup.py", line 7, in
import DNS
File "/tmp/pip-build-r83haI/py3dns/DNS/init.py", line 27, in
from .Base import DnsRequest
File "/tmp/pip-build-r83haI/py3dns/DNS/Base.py", line 9
SyntaxError: Non-ASCII character '\xc2' in file /tmp/pip-build-r83haI/py3dns/DNS/Base.py on line 10, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-r83haI/py3dns/
`

[MasterSergius]

Hello, I've got the same issue in the same situation - parse message and check it if it's spam. Moreover, email message was stored in *.eml file. Can I avoid this error somehow while there is no fix?