UTF-8 configuration errors

[vaqued]

Hello!;

I am trying to configure the server to allow UTF-8 codification yet I have troubles. I have a django framwork working in this server and I have errors when I try to save files that contain acents or the 'ñ' spanish character in the name.

This is the debug.log of the django with the sample error:

File "/usr/local/lsws/Example/html/defferary/gestionInformes/views.py", line 349, in update_info
    nuevo_docPrestamosAH = DocPrestamosAH.objects.create(
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/query.py", line 677, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/base.py", line 822, in save
    self.save_base(
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/base.py", line 909, in save_base
    updated = self._save_table(
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/base.py", line 1067, in _save_table
    results = self._do_insert(
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/base.py", line 1108, in _do_insert
    return manager._insert(
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/query.py", line 1845, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    for sql, params in self.as_sql():
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1745, in as_sql
    value_rows = [
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1746, in <listcomp>
    [
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1747, in <listcomp>
    self.prepare_value(field, self.pre_save_val(field, obj))
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1695, in pre_save_val
    return field.pre_save(obj, add=True)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/fields/files.py", line 317, in pre_save
    file.save(file.name, file.file, save=False)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/db/models/fields/files.py", line 93, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/core/files/storage/base.py", line 38, in save
    name = self._save(name, content)
  File "/usr/local/lsws/Example/html/lib/python3.10/site-packages/django/core/files/storage/filesystem.py", line 106, in _save
    fd = os.open(full_path, self.OS_OPEN_FLAGS, 0o666)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 76: ordinal not in range(128)

This is my SO:
Linux srv476978 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

These are my locale variables:

locale
LANG=es_ES.UTF-8

LANGUAGE=
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE="es_ES.UTF-8"
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=es_ES.UTF-8

This is the /usr/local/lsws/Example/html/.htaccess content:
AddDefaultCharset UTF-8

Also, in the admin configuration page of the server there is incorporated 'AddDefaultCharset UTF-8' in the Header Operations of the rui '/' (type app) content of the virtual host and the 'Default Charset' configurationis is with 'on' configuration.

The settings configuration of Django for the data saved and the codification is the following:

DEFAULT_CHARSET = 'utf-8'
FILE_UPLOAD_HANDLERS = [
    'django.core.files.uploadhandler.MemoryFileUploadHandler',
    'django.core.files.uploadhandler.TemporaryFileUploadHandler',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.locale.LocaleMiddleware',  # Asegúrate de que este middleware esté aquí
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

Some more intesented data is that the html webpag shows the UTF-8 codification in the Network console: content-type: text/html; charset=utf-8 adddefaultcharset: UTF-8

The code is working in the development Django server and not in the OpenLiteSpeed server.

And this is the database PostgreSQL codification:

postgres=# SHOW client_encoding;
 client_encoding
-----------------
 UTF8
(1 fila)

postgres=# \l
                                                          Listado de base de datos
    Nombre     |  Dueño   | Codificación | Proveedor de locale | Collate |  Ctype  | configuración ICU | Reglas ICU: |      Privilegios
---------------+----------+--------------+---------------------+---------+---------+-------------------+-------------+-----------------------
 defferarybbdd | postgres | UTF8         | libc                | C.UTF-8 | C.UTF-8 |                   |             |
 postgres      | postgres | UTF8         | libc                | C.UTF-8 | C.UTF-8 |                   |             |
 template0     | postgres | UTF8         | libc                | C.UTF-8 | C.UTF-8 |                   |             | =c/postgres          +
               |          |              |                     |         |         |                   |             | postgres=CTc/postgres
 template1     | postgres | UTF8         | libc                | C.UTF-8 | C.UTF-8 |                   |             | =c/postgres          +
               |          |              |                     |         |         |                   |             | postgres=CTc/postgres
(4 filas)

We are thinking that it is something of the OpenLiteSpeed server configuration and we need help.

Regards and thanks a lot!

 

[Cold-Egg]

It may not work if you set "AddDefaultCharset UTF-8" in the .htaccess file.
I assume you are using python app context from OpenLiteSpeed, then you might want to add something like "PYTHONIOENCODING=utf-8" or "AddDefaultCharset=UTF-8" to the App Server Context > Environment. Or add the UTF-8 to the output directly.
Let us know if it works.

 

[lukonii]

One thing you could try is to ensure that your Python environment is also set to use UTF-8. You can do this by adding these lines to the top of your Django settings file:

import sys
sys.setdefaultencoding('UTF-8')

Also, check if your OpenLiteSpeed server is correctly set to handle UTF-8. You might need to add 'utf-8' to the 'DefaultCharset' setting in the server configuration.