I find specially nice the completion feature for filenames of IDLE when they include long paths, something that is more mand more frequent with big disks and infinite amounts of files.

But there is a small problem with it. If the name of any of the directories that intervene in the path to the file has any special character in it (meaning by that I suppose any non ASCII character), the completion mechanism ceases to work.

For instance, if the path were:

    d:/Biblioteca/Técnica/informática/Python ...

the cesation of the help would occur after the incorporation of *Técnica*, because its *e* is accented.

There is plenty of cheap software which shows this minor deficiency, but I consider that IDLE for Python 3 should not be in that group.

Would you like to contribute a patch?

I would be delighted, but unfortunately I am a very poor programmer and do not have the slightest idea of how all this works.

The "open_completions" method in AutoComplete.py is where the bug exists. When mode == COMPLETE_FILES, the code searches for characters within the ASCII set, plus a few others contained in FILENAME_CHARS. 

Attached is a patch to also include characters beyond the ASCII set during the backward search. If anyone knows of a better technique, please comment. 

Fransisco, thanks for reporting this issue. If you find more, please let us know.

I think a better technique would be to expand FILENAME_CHARS to include more characters.

Also, shouldn't the space character ' ' be included?

Ahh okay, sorry for the triple post, I have an idea.  On UNIX, the function should accept any character except: \0 /, and on Windows should accept any character except: \0 \ / : * ? " < > |  On classic Macintosh, : is invalid.  However, I do not know what characters are valid on other systems.  (I believe Mac OS X is the same as UNIX, but am not sure.)

Included is a patch with FILENAME_CHARS left in in case anything still needs it.

I agree that chr(32) should be included in FILENAME_CHARS. 

The algorithm for backward searching checks that each character is contained in FILENAME_CHARS. I'm concerned about running time, as expanding FILENAME_CHARS to include all valid Unicode would be a large set to search. Do you know a better algorithm for determining when to perform a file-system completion when in a string?

Also, are there any Unicode code points above 127 that are not valid in a filename? I appreciate that different file systems can have different constraints but I don't know all these subtle points.

Westley, I was responding to msg162168 and didn't see msg162169 yet. 

PEP11 mentions MacOS 9 support was removed in 2.4. Is ":" still invalid in OSX?

I'll need to think about the approach of using an "INVALID_CHARS" list. It looks like it might be a better way to do this. However, here are some issues with the patch.

I received this error:

    AttributeError: 'str' object has no attribute 'relpace'

I fixed this minor mistype, but noticed that file auto-completion no longer works for the following:

>>> "/

Pressing tab after the forward slash should show a drop-down with the root filesystem. Changing INVALID_CHARS to be:

   if os.name == 'posix':
       INVALID_CHARS = '\0'

allows the root filesystem to show.

Is there any necessity at all for the IDLE to test the validity of the filenames? 

I mean: the file specification is provided by the underlying operating system, so by definition it has to be taken as valid. Testing for its validity is superfluous and, in my opinion, a fault in the general conception of the procedure. IDLE's only role should be to present the data it receives in the best possible form and silently integrate the election made by the user.

Excuse me if I speak nonsense, but I got this idea from studying *Autocomplete.py* and its dependencies. It seems to me that the difficulty resides in submitting natural language strings to the judgement of *PyParse* and *Hyperparser*, which are not competent for it because its purpose is to check the validity of Python code and Python *identifiers*. This confussion is what in my opinion causes this bug (and probably others related to the treatment of natural language text files by the IDLE, as it could also happen with issue 14929).

You're right.  The code shouldn't *have* to check if the name is valid.  It should just accept that the name is already valid.  This would simplify things.

Here's the problem: the code needs to find the index of where the string with the filename starts.  The way the code does it now by checking for a quote in a rather obfuscated way (while i and curline[i-1] in FILENAME_CHARS + SEPS:).  So, changing that line to say curline[i-1] != "'" or curline[i-1] != '"' would work (in theory) but I'm really hoping there's a better way.

New changeset 41e85ac2ccef by Martin v. Löwis in branch '3.2':
Issue #14937: Perform auto-completion of filenames in strings even for non-ASCII filenames.
http://hg.python.org/cpython/rev/41e85ac2ccef

New changeset 9aa8af0761ef by Martin v. Löwis in branch 'default':
Merge 3.2: issue #14937.
http://hg.python.org/cpython/rev/9aa8af0761ef

I have now fixed it by looking for the beginning of the string, and not checking for file name characters at all. There was a related issue that the auto-complete window would disappear if you type a non-ascii character; I have fixed that as well.

There's a small logic bug with Martin's patch. Opening up a new editor window and pressing Tab causes a traceback. Attached is a new patch to fix the problem.

New changeset 62030ebb2b01 by Martin v. Löwis in branch '3.2':
Issue #14937: Fix typo. Patch by Roger Serwy.
http://hg.python.org/cpython/rev/62030ebb2b01

Thanks, fixed

This patch (I suspect it is this one) disabled the use of '/' in filenames on windows when using filename completion.

'c:\   <wait, tab, ^space bring up box> in 3.2.3 and 3.3.0>
(If there is no 'r' prefix, it really should require '\\' to be safe.)

"c:/   ditto for 3.2.3.
In 3.3.0, (and, I presume, 3.2.4 if not changed) tab inserts spaces to next tab stop, while ^space is ignored.

Since using '/' is recommended over unsafe '\', this is an undesireable regression.

In my machine with Windows XP and Python 3.3.0 both variants work, the only
difference being that

'c:/
brings up the selection box authomatically and

'c:\

requieres that it be summoned with the tab key, as indicated.

2012/10/12 Terry J. Reedy <report@bugs.python.org>

>
> Terry J. Reedy added the comment:
>
> This patch (I suspect it is this one) disabled the use of '/' in filenames
> on windows when using filename completion.
>
> 'c:\   <wait, tab, ^space bring up box> in 3.2.3 and 3.3.0>
> (If there is no 'r' prefix, it really should require '\\' to be safe.)
>
> "c:/   ditto for 3.2.3.
> In 3.3.0, (and, I presume, 3.2.4 if not changed) tab inserts spaces to
> next tab stop, while ^space is ignored.
>
> Since using '/' is recommended over unsafe '\', this is an undesireable
> regression.
>
> ----------
> resolution: fixed ->
> stage: patch review -> needs patch
> status: closed -> open
> versions: +Python 3.3, Python 3.4
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue14937>
> _______________________________________
>

On Thu, Oct 11, 2012 at 3:06 PM, Terry J. Reedy <report@bugs.python.org> wrote:
>
> Terry J. Reedy added the comment:
>
> This patch (I suspect it is this one) disabled the use of '/' in filenames on windows when using filename completion.
>
> 'c:\   <wait, tab, ^space bring up box> in 3.2.3 and 3.3.0>
> (If there is no 'r' prefix, it really should require '\\' to be safe.)

+1 for requiring \\.

I'll test this tomorrow and report back the behaviour.  On Linux it
seems that the window only pops up when you press tab.  I think
behaviour should be as identical on all platforms as possible.

On 3.4.0a1 on Windows it seems to come up automatically with \ or \\.  A single \ will only pop up with the tab key; I think it's good that way.  Special characters (i.e. æ) work normally.  All this said, I think this issue is fixed.

I am not currently seeing the problem I reported, so closing. If I type past / and hit ^space, I sometimes get an empty listbox, but I do not yet have a consistent failing case. Will open a new issue if/when I get one.