liyuping-fisher 发表于 2023-6-26 10:15:18

python3.11求助将制定目录下的ini文件不管是什么编码都改为utf-8

要求:
中文不能为乱码
编码全部为改为utf-8
文件全部改为unix

如下我写了后有的文件的中文是乱码了,应该原来是Ascall码改为utf-8成了乱码
      encoding = 'utf-8'
      for root, dirs, files in os.walk(self.factoryroot_path):
            for file in files:
                if file.endswith('.ini'):
                  file_path = os.path.join(root, file)
                  os.system('dos2unix ' + file_path)
                  # 以读取二进制文件的方式打开 ini 文件
                  with codecs.open(file_path, 'rb') as f:
                        content = f.read().decode(encoding, 'ignore')
                        
                  # 以写入二进制文件的方式重新保存 ini 文件
                  with codecs.open(file_path, 'wb') as f:
                        f.write(content.encode(encoding))

isdkz 发表于 2023-6-26 10:18:00

本帖最后由 isdkz 于 2023-6-26 10:19 编辑

在你的代码中,你已经将所有的ini文件都预设为了utf-8编码来读取,这在处理那些不是utf-8编码的文件时会造成问题,如你提到的中文乱码问题。解决这个问题的一种方法是在读取文件前先判断文件的编码类型,然后使用相应的编码来读取文件。

Python标准库中没有提供检测文件编码的函数,但是我们可以使用第三方库 chardet 来判断文件的编码。下面是一个改进的代码示例:

import os
import codecs
import chardet

for root, dirs, files in os.walk(self.factoryroot_path):
    for file in files:
      if file.endswith('.ini'):
            file_path = os.path.join(root, file)
            os.system('dos2unix ' + file_path)
            # 使用chardet库来判断文件的编码
            rawdata = open(file_path, "rb").read()
            result = chardet.detect(rawdata)
            encoding = result['encoding']

            # 根据检测到的编码来读取文件
            with codecs.open(file_path, 'r', encoding=encoding) as f:
                content = f.read()

            # 以utf-8编码重新保存 ini 文件
            with codecs.open(file_path, 'w', encoding='utf-8') as f:
                f.write(content)


注意,在写文件时我们改为了'w'模式,因为我们要以文本模式写入文件,不再需要用到'wb'模式。使用chardet库后,我们就能正确识别并处理各种编码的文件了,但是注意chardet库可能无法识别出所有类型的编码,尤其是一些非常罕见的编码。

liyuping-fisher 发表于 2023-6-26 10:31:29

能先帮忙看下模块安装错误的问题吗{:5_91:}
python3.11 -m pip install cchardet
Collecting cchardet
Using cached cchardet-2.1.7.tar.gz (653 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: cchardet
Building wheel for cchardet (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─>
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/version.py -> build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/__init__.py -> build/lib.linux-x86_64-cpython-311/cchardet
      running build_ext
      building 'cchardet._cchardet' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/src
      creating build/temp.linux-x86_64-cpython-311/src/cchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Isrc/ext/uchardet/src -I/usr/local/python3.11/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
      src/cchardet/_cchardet.cpp:196:27: fatal error: longintrepr.h: No such file or directory
         #include "longintrepr.h"
                                 ^
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cchardet
Running setup.py clean for cchardet
Failed to build cchardet
Installing collected packages: cchardet
Running setup.py install for cchardet ... error
error: subprocess-exited-with-error

× Running setup.py install for cchardet did not run successfully.
│ exit code: 1
╰─>
      running install
      /usr/local/python3.11/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/version.py -> build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/__init__.py -> build/lib.linux-x86_64-cpython-311/cchardet
      running build_ext
      building 'cchardet._cchardet' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/src
      creating build/temp.linux-x86_64-cpython-311/src/cchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Isrc/ext/uchardet/src -I/usr/local/python3.11/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
      src/cchardet/_cchardet.cpp:196:27: fatal error: longintrepr.h: No such file or directory
         #include "longintrepr.h"
                                 ^
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> cchardet

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

A new release of pip available: 22.3.1 -> 23.1.2
To update, run: pip3 install --upgrade pip

liyuping-fisher 发表于 2023-6-26 11:13:54

isdkz 发表于 2023-6-26 10:18
在你的代码中,你已经将所有的ini文件都预设为了utf-8编码来读取,这在处理那些不是utf-8编码的文件时会造 ...

跪谢大大{:7_113:}
页: [1]
查看完整版本: python3.11求助将制定目录下的ini文件不管是什么编码都改为utf-8