[Creating a new issue from bpo-41045]

I was just just trying to link to someone the documentation for f-strings, but:

1. Searching "fstring" only returns two results about xdrlib0;
2. Searching "f-string" returns many unrelated results1;
3. The first (and closer) result (string -- Common string operations2) yields nothing while using ctrl+f with fstring, f-string, f', f";
4. at the top of that page there are two links in a "see also":

• Text Sequence Type — str3: it mentions raw strings at the beginning, but also yields no results for fstring, f-string, f', f";
• String Methods4: that is another section of the previous page (so ctrl+f doesn't find anything), but has a link to "Format String Syntax"5;

5. The "Format String Syntax" page5 has another link in the middle of a paragraph that points to "formatted string literals", that eventually brings us to the right page6;
6. the "right page"6 has a wall of text with a block of code containing the grammar, luckily followed by a few examples.

I think we should:

1. add index entries for "f-string" and "fstring" in the relevant sections of the docs, so that they appear in the search result;
2. in the Text Sequence Type — str3 section, have a bullet list for f-strings, raw-strings, and possibly u-strings;
3. possibly use the term "f-string" in addition (or instead) of "formatted string literal", to make the pages more ctrl+f-friendly throughout the docs;
4. possibly have another more newbie-friendly section on f-string (compared to the lexical analysis page) in the tutorial, in the stdtypes page7 (e.g. before the printf-style String Formatting" section8), or in the string page2 (e.g. after the "Format String Syntax" section[10]);
5. possibly reorganize and consolidate the different sections about strings, string methods, str.format(), the format mini-language, f-strings, raw/unicode-strings, %-style formatting in a single page or two (a page for the docs and one for the grammar), since it seems to me that over the years these sections got a bit scattered around as they were being added.

I think this is an excellent idea. The main f-string docs being in a section titled "Lexical Analysis" never seemed very user-friendly.

It's basically an accident that the only f-strings docs are in the language reference. Yes, they should be there, and the text there is pretty good for the reference, but there isn't much about them elsewhere outside of the tutorial, so everything links there. Maybe we need a section on them in the library part, of intermediate complexity between what's in the tutorial an what's in the reference. This could be on equal footing with the descriptions of % formatting and .format(), which seem to be quite extensive. We might also be able to share some text between .format() and f-strings, since the !x and :... parts are treated identically AFAICT. This would be another refactoring of some part of the docs.

Hi Ezio,

Would you see this being resolved in part by a HOWTO document?

HOWTOs are generally used to explain how to accomplish a certain task, so I'm not sure if they are a good fit for this situation.

In the list of proposed solutions in my first message, 1-3 should be quite easy to implement. This leaves us with 4-5.

To summarize what we already have (let me know if I missed anything):

• string.rst0 (the string module):
  String constants (string constants)
  Custom String Formatting (string.Formatter)
  Format String Syntax (explains the syntax of {...})
  Format Specification Mini-Language
  Format examples
  Template strings (string.Template)
  Helper functions (string.capwords)
• stdtypes.rst1 (about Python builtin types):
  Text Sequence Type — str (short intro about str)
  String Methods (all the str.* methods)
  printf-style String Formatting (old %-formatting)
• introduction.rst2 (the string section in the tutorial)
• lexical_analysis.rst3
  String and Bytes literals
  String literal concatenation
  Formatted string literals

The lexical analysis is probably fine as is. The introduction in the tutorial should mention f-strings, include a couple of examples, and link to the documentation for more. The stdtypes page is already quite long and crowded, so I'm wary about adding more stuff in there. So, unless we add a separate page somewhere, the string page seems the best candidate, even if technically it should be about the string module.

On top of items 1-3 of my previous message, to solve 4-5 I suggest to:

1. move the printf-style formatting section to string.rst;
2. add a section about f-strings under Format String Syntax in string.rst, focusing on differences and caveats that are unique to f-strings;
3. add some f-string-specific examples after the format examples
4. mention f-string in the string section of the tutorial, and in the text sequence type section of stdtypes.rst;

The string.rst page could then look something like:
String constants (string constants)
Custom String Formatting (string.Formatter)
Format String Syntax (explains the syntax of {...})
Format Specification Mini-Language
Format examples
f-strings
f-strings examples
printf-style String Formatting (old %-formatting)
Template strings (string.Template)
Helper functions (string.capwords)

This should consolidate all the docs in two places: string.rst and lexical_analysis.rst (they have a different target, so the separation is ok). All other places can then refer to this. Once this is done, we can still decide to move these out of string.rst.

Note that there already is something in the tutorial about f-strings (in
inputoutput.rst, labeled tut-f-strings), and the intro has a link to their
reference manual description in the "see also" section.

Hi,

Thanks Ezio for the detailed response.

In the list of proposed solutions in my first message, 1-3 should be quite easy to implement. This leaves us with 4-5.

To the best of my seeing there are only two places in the documentation that have information about f-strings. The tutorial[0] and the reference[1]
I have done PR 21681 that adds index to the tutorial although searching[2][3] does not seem to be better now that the reference has an index.

The lexical analysis is probably fine as is.
I agree.

The introduction in the tutorial should mention f-strings ....

This is there[0] but it _IS_ hard to find, when you don't know it is there.
It may not be very "introduction-y", and if you like I can make a go at trying to reword it.

The stdtypes page is already quite long and crowded ...

True. But is this wrong? In my feeling this is reference documentation. It should be accurate, complete and clear. It may not need to be "textbook-like".

Some thoughts:

• There may be benefit in reorganising stdtypes.rst to improve the flow without changing the actual content. This could mean breaking it up into a number of documents rather than the monolith it is.
• It does feel like printf[4] was plugged in later because str.format()[5] had been explained earlier. (Although I believe printf came before str.format()). A first time reader of the document will find it hard to know the one way that is right when it comes to formatting.
• f-strings should probably also be described here because it _is_ built in, no? It may not be accurate to say it is in /Lib/strings. There is no reference that a developer can just look at to remind/confirm to themselves how to "do it".

So, unless we add a separate page somewhere

This is why I was thinking of a HOWTO. Going back to your original pain in msg371912, someone wanted the f-string documentation and:
-- It was hard to find. (indexing? better table of contents?)
-- It was not very helpful WHEN it was found. (A second entry with f-strings in Lib that is easier for a dev to use?)

But now (and this is mainly to me) it appears that another problem is a need to consistently, clearly, and in one place describe the various elements/nuances/eccentricities of presenting data in Python:

• string
• string.Formatter,
• string.Template,
• str.format()
• f-string
• Format Specification Mini-Language
• maybe even __format__ and conversion?
• etc

I propose one of the following two:

1. A new section in Library documentation just to handle strings, then systematically removing them elsewhere.(Built-In Functions, Built-In Types, Text Processing Services)
2. We take all that Python history and explain it in a HOWTO, and add a developer friendly reference in the Library for f-string. More like Logging Cookbook rather than Logging Howto.

!) above is similar to what you propose ... but different-ish.

[0] https://docs.python.org/dev/tutorial/inputoutput.html#formatted-string-literals
[1] https://docs.python.org/dev/reference/lexical_analysis.html#f-strings
[2] https://docs.python.org/dev/search.html?q=fstring
[3] https://docs.python.org/dev/search.html?q=f-string
[4] https://docs.python.org/dev/library/stdtypes.html#printf-style-bytes-formatting
[5] https://docs.python.org/dev/library/stdtypes.html#str.format

Thanks both for pointing out the additional section in inputoutput.rst that I initially missed.

I have done PR 21681 that adds index to the tutorial although searching[2][3] does not seem to be better now that the reference has an index.

The online docs seem updated, so I'm not sure why it's not working. Maybe you could try moving things around and see if you can make it work? You should be able to build the docs locally and test from there. You can also see if Sphinx raises any warning during the build process, try to move the .. index directive just after the section title (instead of before), or add an empty line between the list of entries and the section label (unnecessary if you move the entries after the title).

This is there[0] but it _IS_ hard to find, when you don't know it is there.

I noticed (or rather, I didn't :)!

It may not be very "introduction-y", and if you like I can make a go at trying to reword it.

I think this section is fine, the only thing that needs to be update is the link at the end of the section to point to the main f-string section that we will add elsewhere.

> The stdtypes page is already quite long and crowded ...
True. But is this wrong? In my feeling this is reference documentation. It should be accurate, complete and clear. It may not need to be "textbook-like".

You might be right.

Some thoughts:

• There may be benefit in reorganising stdtypes.rst to improve the flow without changing the actual content. This could mean breaking it up into a number of documents rather than the monolith it is.

Both in the documentation and in the code, there are advantages in having everything on a single page/file (such as easier ctrl+f search) but also disadvantages. Given the goal of the page, I think keeping it monolithic is fine: it provides a summary of all the built-in types in a single place. Breaking it down into multiple pages will have other issues. If you make a page per type, you will have some pages that are very short, and it will be more difficult to get an overview of all the types, comparing them, and searching through all of them at once.

• It does feel like printf[4] was plugged in later because str.format()[5] had been explained earlier. (Although I believe printf came before str.format()). A first time reader of the document will find it hard to know the one way that is right when it comes to formatting.

If I recall correctly that section has also been there, but initially it was just describing the standard way of doing string formatting, and when str.format() was added, it got renamed and the note at the top was added to link to the str.format() documentation. I guess that the str.format documentation ended up in string.rst because it was related to string.formatter, and the author wanted to keep them together.

• f-strings should probably also be described here because it _is_ built in, no? It may not be accurate to say it is in /Lib/strings. There is no reference that a developer can just look at to remind/confirm to themselves how to "do it".

Agreed, stdtypes.rst would be the right place where to document this, my only concerns were: 1) the size of the page (that as said above, is not such a big deal); 2) keeping str.format() and string.Formatter() together (but this is also not a huge concern).

Also note that str.format() is not documented in stdtypes.rst, however there is an entry for str.format() with a clear link to the formatting docs, so I never had a problem finding it, even if I needed an extra click.

[...]

But now (and this is mainly to me) it appears that another problem is a need to consistently, clearly, and in one place describe the various elements/nuances/eccentricities of presenting data in Python:

• string
• string.Formatter,
• string.Template,
• str.format()
• f-string
• Format Specification Mini-Language
• maybe even __format__ and conversion?
• etc

What about doing the following:

• keep having stdtypes.rst cover and explain all the built-in types and their features;
• move the "Format String Syntax", "Format Specification Mini-Language", "Format examples" sections from string.rst to stdtypes.rst where they belong;
• integrate f-strings in these sections, and add a new section explaining f-string-specific quirks;
• leave the printf-style string formatting in stdtypes.rst, after the format sections
• use string.rst to document the string module and its objects, hence leaving string.Formatter and string.Template here, where they belong (string.Formatter is self contained enough that doesn't need to be with the other format sections);
• leave the inputoutput.rst and lexical_analysis pages as they are;
• update the introduction.rst page to mention f-string;
• once all this is done, update all links to point to the appropriate sections and cross-link all related sections;

I propose one of the following two:

1. A new section in Library documentation just to handle strings, then systematically removing them elsewhere.(Built-In Functions, Built-In Types, Text Processing Services)

Doing this would leave a gap in the built-in types page, and you would still want things things like string.Formatter and string.Template to be documented in string.rst. It also raises the question: where should we put this page?

2. We take all that Python history and explain it in a HOWTO, and add a developer friendly reference in the Library for f-string. More like Logging Cookbook rather than Logging Howto.

One of the reasons why I'm not fond of the HOWTO idea, is that these are built-in features that need to be documented in the main documentation, where the string type is documented. It is my understanding that the HOWTOs are mostly meant as standalone (and somewhat external) additions that cover a specific topic in detail. While having an HOWTO on all the quirks and options for doing string formatting in Python does make sense, we would still have to cover it in the main docs, thus ending up duplicating a lot. For example the argparse tutorial (one of the HOWTOs), shows actual examples of how to use argparse, but the module itself and its API are already extensively documented in the argparse module documentation.

The only way I could see it working is if we turned the "Format Examples" section in an HOWTO and add more examples specific to f-string, however when I wrote that section I think I tried to cover all the features and the end result was compact enough to fit in a single section. I'm also not sure there's much value in adding more examples and documentation about the old string formatting, string.Formatter, and string.Template, since nowadays they are rarely used (especially the last two).

There are also another couple of thoughts I had, that however I haven't fully developed yet (so it's better to deal with them later or in another issue):

• str.format() is still used in a lot of places in the docs, where f-strings could be used instead. These should be updated to use f-string;
• format(), str.format(), and f-string, have quite a lot of overlapping, but I'm not sure what's the best way to document them clearly without unnecessary repetition. I have a feeling that documenting the format mini-language and using f-string for the features shared by all three might be the best option -- additional examples that are specific to format(), str.format(), and f-strings, can then be added separately.

Hi,

The online docs seem updated, so I'm not sure why it's not working. Maybe you could try

So it seems that the .. index::[0] directive creates an index[1]. Both f-index and findex are available in it.
Search is a bit different. A searchindex[2] is generated once when the html is being created. This is then used, locally, when user does search. Still it is not very good. Multiple word queries[3] and hyphenated words don't appear to work[4] well (I also tested this locally, f-string does not get searchindexed while fstring does. Searching also gives the lexical analysis as one of the pages when searching for fstring.)

What about doing the following:

• keep having stdtypes.rst cover and explain all the built-in types and their features;
• move the "Format String Syntax", "Format Specification Mini-Language", "Format examples" sections from string.rst to stdtypes.rst where they belong;
• integrate f-strings in these sections, and add a new section explaining f-string-specific quirks;
• leave the printf-style string formatting in stdtypes.rst, after the format sections
• use string.rst to document the string module and its objects, hence leaving string.Formatter and string.Template here, where they belong (string.Formatter is self contained enough that doesn't need to be with the other format sections);
• leave the inputoutput.rst and lexical_analysis pages as they are;
• update the introduction.rst page to mention f-string;

introduction.rst has a reference to "Formatted string literals" in the "See also:" box[5]. But I can still put an example here. Should we put both str.format() and f-strings, or make this exclusively for f-strings?

• once all this is done, update all links to point to the appropriate sections and cross-link all related sections;

Ok.

Please can we progress as follows: (Each as sequential and independent PR)

1. f-string added to stdtypes.rst. I have done this in PR 21552. We can complete that one and commit it as the first step. It provides both the f-string and the f-string-specific quacks.
2. Add the same wording of f-string to pydoc. I have been studying how to do this. At this moment help(fstring) and help(f-string) do not work.

PS C:\> py -3.9
Python 3.9.0b5 (tags/v3.9.0b5:8ad7d50, Jul 20 2020, 18:35:09) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help(fstring)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'fstring' is not defined
>>> help(f-string)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined
>>>

3. Move "Format String Syntax", "Format Specification Mini-Language", "Format examples" sections from string.rst to stdtypes.rst and ensure references all work well.

4. Add additional f-string-quarks as we discuss into stdtypes.rst.

[0] https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#index-generating-markup
[1] https://docs.python.org/dev/genindex-F.html
[2] https://docs.python.org/dev/searchindex.js
[3] sphinx-doc/sphinx#1486
[4] sphinx-doc/sphinx#1486 (comment)
[5] https://docs.python.org/3.9/tutorial/introduction.html

@ezio.melotti: What is this waiting for at this point? There was some good energy here but progress seems halted, waiting for some kind of decision.

It seems ezio-melotti hesitates to continue the modifications in the PR #21552 (review)

IMO, the PR improves enough the documentation, so it would be nice to merge it, even without the other changes.

I'd like to help but I don't know how I could do that.

@ezio.melotti, @amaajemyfren Are you still interested by this issue? How can I help you?

Eric, do you want to approve and apply these PRs ?