classification
Title: msgfmt should be able to merge more than one po file
Type: enhancement Stage: patch review
Components: Demos and Tools Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: SilentGhost, eric.araujo, ezio.melotti, s-ball
Priority: normal Keywords: patch

Created on 2018-11-28 07:26 by s-ball, last changed 2018-12-03 20:27 by s-ball.

Files
File name Uploaded Description Edit
files.zip s-ball, 2018-12-01 18:33 2 example po files
Pull Requests
URL Status Linked Edit
PR 10875 open s-ball, 2018-12-03 19:33
Messages (8)
msg330575 - (view) Author: (s-ball) * Date: 2018-11-28 07:26
GNU gettext msgfmt can merge several po files into one single mo file.
The current version of msgfmt can only compile one po file to one mo file.

After looking at the code, the enhancement should be simple to implement.

Command line: if one output file is given (option -o) and several input files, then all the input files should be combined.

Implementation:
- main should pass all the parameters to make (*args)
- make should accept one single string for compatibility or an iterable of string. In that latter case, the current processing should be repeated on all input files.

I could propose a patch (but I am afraid it ends being rather large) or a pull request. As a new user here, I do not know what is the best way...
msg330608 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2018-11-28 16:16
The best way is to propose a PR via github (here are some details: https://devguide.python.org/pullrequest/) This sort of change, can only be applied against master (future 3.8), so don't spend time on legacy releases.
msg330701 - (view) Author: (s-ball) * Date: 2018-11-29 16:36
Ok, I have created a fork, and started coding on a local branch. But it will take some time, because I assume that I am supposed to write tests for the msgfmt module...
msg330704 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2018-11-29 17:10
The test for this part of code are located in Lib/test/test_tools/test_i18n.py as you'll quickly notice there aren't any tests specific to msgfmt.py I would actually advice to start by writing some tests for existing behaviour, so you'd know if (when) you break things.
msg330859 - (view) Author: (s-ball) * Date: 2018-12-01 18:33
I have followed SilentGhost's advice and begun by thoroughly testing the current behaviour. I then realized that I was wrong, and that (probably by chance) msgfmt.py partially followed my requirements, but pybabel did not and GNU gettext msgfmt did not really. I wrote 2 tiny po files (joined) and played with them, meaning I tried to combine them with pybabel, msgfmt.py and GNU gettext msg. Current behaviour (Windows shell syntax):

> pybabel compile -o .\file12-fr.mo -l fr -i file1-fr.po -i file2-fr.po

only uses second file (file2-fr.po)

> msgfmt -o file12-fr.mo --no-hash file1-fr.po file2-fr.po

chokes on a repeated key on file2 (the header has "" for key...). It works fine anyway after commenting out the header in any of the files

> python "path\to\Tools\i18n\msgfmt.py" -o file12py-fr.mo file1-fr.po file2-fr.po

unexpectedly produces the expected result and successfully combines both po files into one single mo file

BUT:

> python "path\to\Tools\i18n\msgfmt.py" file1-fr.po file2-fr.po

Produces file1-fr.mo which is the compiled version of file1-fr.po and file2-fr.mo which combines both input files. Definitely not an expected result!

This is caused by the problem identified in issue 9741 (https://bugs.python.org/issue9741)

My initial goal was to be able to use the make function from msgfmt.py in an external script. I then realize that combining multiple po files is not a good idea because the resulting mo file can only contain one single header and the best behaviour is GNU gettext msgfmt one.

I now wonder whether this issue should not be closed because the requirement is not relevant, and it would probably better to propose a fix (including tests and code improvement) for issue 9741.
msg330897 - (view) Author: (s-ball) * Date: 2018-12-02 19:21
After some more thinking about it, my opinion is that the proposed path for issue 9741 does not address at all my requirements. So I will try to propose a pull request addressing both issues here.
msg330977 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2018-12-03 20:11
I am curious about the use cases for this feature!
msg330979 - (view) Author: (s-ball) * Date: 2018-12-03 20:27
Currently my main use case is to be able to compile one or more po file(s) from a Python script, so I just need to be able to repeatedly call make from that script - which was broken per issue 9741

To be honest, I must acknowledge that I initially thought that compiling more than one po file was a common use case, and I only later realized that it was not. But as it was already (partially) allowed by msgfmt.py, I have just fixed the problems and added tests for it.

BTW, I am also the author of last commit, but I have written it on a box where I had forgotten to correctly initialize git
History
Date User Action Args
2018-12-03 20:27:44s-ballsetmessages: + msg330979
2018-12-03 20:11:38eric.araujosetmessages: + msg330977
2018-12-03 20:10:03SilentGhostsetnosy: + ezio.melotti, eric.araujo
2018-12-03 19:33:03s-ballsetkeywords: + patch
stage: patch review
pull_requests: + pull_request10112
2018-12-02 19:21:45s-ballsetmessages: + msg330897
2018-12-01 18:33:52s-ballsetfiles: + files.zip

messages: + msg330859
2018-11-29 17:10:15SilentGhostsetmessages: + msg330704
2018-11-29 16:36:32s-ballsetmessages: + msg330701
2018-11-28 16:16:40SilentGhostsetnosy: + SilentGhost

messages: + msg330608
versions: - Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
2018-11-28 07:26:19s-ballcreate