If you add a positional parameter by adding:
parser.add_argument('foo')
then the output becomes:

$ python argparse-help-says-required-args-are-optional.py -h
usage: issue9649.py [-h] --reqarg REQARG [--optarg OPTARG] foo

Do something

positional arguments:
  foo

optional arguments:
  -h, --help            show this help message and exit
  --reqarg REQARG, -r REQARG
                        This is required
  --optarg OPTARG, -o OPTARG
                        This is optional

$

So whatever replaces "optional arguments:" needs to read well with "positional arguments:". Maybe just plain "options:" is good enough, but I think a word to replace "optional" (leaving "arguments:") would be better. I just don't have any useful suggestion :)