classification
Title: compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used as a type, but has not been defined as a type
Type: compile error Stage: resolved
Components: Build, Unicode Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: eric.araujo, jschneid, loewis, michael-o, srid, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2009-05-11 21:24 by srid, last changed 2018-08-22 09:42 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
config.log srid, 2009-06-29 22:47
pyconfig.h srid, 2009-06-29 22:47
wchar.h srid, 2009-06-29 22:49 /usr/include/wchar.h
_mbstate_t.h srid, 2009-07-02 18:24 #include <sys/_mbstate_t.h>
test.c.preprocessed.txt srid, 2009-07-02 18:25 preprocessed output of test program using mbstate_t
test.c_XOPEN_SOURCE_500_preprocessed.txt srid, 2009-07-02 18:29 preprocessed with -D_XOPEN_SOURCE=500
stdsyms.patch jschneid, 2011-07-14 18:18 Workaround for faulty extension detection logic vendor's include file
Messages (20)
msg87588 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-05-11 21:24
cc +DD64 -Ae -D_REENTRANT +Z -c  -DNDEBUG -O  -I. -IInclude -I./Include
  -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
Error 419: "./Modules/python.c", line 34 # 'mbstate_t' is used as a
type, but has not been defined as a type.
            mbstate_t mbs;
            ^^^^^^^^^     
make: *** [Modules/python.o] Error 2


!
!
!

When I get time, I need to create a patch to fix this.. but if somebody
else already figured out what the issue is, that'd be great too.
msg87649 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-05-12 18:25
Adding Martin to the nosy list as this is related to his change:
http://svn.python.org/view/python/branches/py3k/Modules/python.c?view=annotate#l17
msg87653 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-05-12 18:38
I have two questions:
1. does your system provide the mbstate_t type?
2. if so, what header file needs to be included?

Also, please confirm a few things:
a. configure has detected that your system has mbrtowc
b. configure's analysis is correct, i.e. your system has mbrtowc indeed.
msg89890 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-06-29 22:46
Hello Martin,

My apologies for responding so late.


[Martin] Also, please confirm a few things: a. configure has detected 
that your system has mbrtowc (...)

# Yes, as can be seen in the attached config.log

[Martin] (...) b. configure's analysis is correct, i.e. your system has 
mbrtowc indeed.

# Yes, `mbrtowc` is present as the following C programs compiles and 
runs successfully:

  #include <wchar.h>
  int main()
  {
    printf("Init\n");
    mbrtowc(NULL, "", 1, NULL);
  }

[Martin] does your system provide the mbstate_t type? if so, what 
header file needs to be included?

`mbstate_t` seems to exist in /usr/include/wchar.h .. however, 
including <wchar.h> does not seem to work:

bash-2.04$ cc +DD64 -Ae -D_REENTRANT +Z --version              
cc: HP aC++/ANSI C B3910B A.05.55 [Dec 04 2003]
bash-2.04$ cat test.c
#include <wchar.h>
int main()
{
  mbstate_t foo;
  printf("Init\n");
  mbrtowc(NULL, "", 1, NULL);
}
bash-2.04$ cc test.c -o test                     
Error 419: "test.c", line 6 # 'mbstate_t' is used as a type, but has 
not been defined as a type.
      mbstate_t foo;
      ^^^^^^^^^     
bash-2.04$ aCC test.c -o test
Error 403: "test.c", line 8 # Undeclared variable 'mbrtowc'. Perhaps 
'mbtowc' as in "int mbtowc(wchar_t *,const char *,unsigned long)" ["/
usr/include/stdlib.h", line 169] was intended.
      mbrtowc(NULL, "", 1, NULL);
      ^^^^^^^   

cf. http://www.mail-archive.com/lftp-devel@uniyar.ac.ru/msg00602.html
msg89906 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-06-30 05:38
> `mbstate_t` seems to exist in /usr/include/wchar.h

I can't infer that from the copy of wchar.h that you provided. I
see that mbstate_t* is used, but I fail to find any definition
of mbstate_t.

I see that sys/_mbstate_t.h is included. This inclusion is conditional
on  _INCLUDE__STDC_A1_SOURCE, so a first check should be done whether
this is defined. In addition, it would be interesting to know what
_mbstate_t.h contains.

> bash-2.04$ cat test.c
> #include <wchar.h>
> int main()
> {
>   mbstate_t foo;
>   printf("Init\n");
>   mbrtowc(NULL, "", 1, NULL);
> }

It's best to focus on this example. Produce preprocessor output for
it, and attach that to the bug report.

This is standard C, AFAICT, so if the compiler fails to compile
it, something is wrong with the compiler, or you are using it incorrectly.

As a wild guess, try defining _XOPEN_SOURCE to 500, i.e.
-D_XOPEN_SOURCE=500.

> cf. http://www.mail-archive.com/lftp-devel@uniyar.ac.ru/msg00602.html

I don't think this is relevant. Somehow, they managed to #define
mbstate_t to int, breaking the header - this should not happen here.
msg90030 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-07-02 18:27
I've attached the files you requested.

> As a wild guess, try defining _XOPEN_SOURCE to 500, 
> i.e. -D_XOPEN_SOURCE=500.

Yes, this works .. the file compiles. I tried modifying the value of 
_XOPEN_SOURCE to 500 in pyconfig.h (currently set to 600). While fixes 
the mbstate_t error .. it results in in-numerous other errors.
msg90033 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-07-02 19:15
> Yes, this works .. the file compiles. I tried modifying the value of 
> _XOPEN_SOURCE to 500 in pyconfig.h (currently set to 600). 

That's an HPUX bug then, please report it to HP. They should assume
that any feature present in XPG 5 is also present in any later version.

> While fixes 
> the mbstate_t error .. it results in in-numerous other errors.

Well, we absolutely need mbstate_t, so we need to resolve all these
other errors.

Please be prepared for this becoming a many-months project. I don't
know what you need the HP-UX port for - maybe it would be best to hire
somebody who knows HP-UX, C, and Python.
msg139222 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-06-26 21:04
Sridhar, is there still a problem with current 3.2/3?

If you are no longer working on this, I think we should close as languishing/postponed.
msg139307 - (view) Author: Sridhar Ratnakumar (srid) Date: 2011-06-27 16:03
On 2011-06-26, at 2:04 PM, Terry J. Reedy wrote:

> Terry J. Reedy <tjreedy@udel.edu> added the comment:
> 
> Sridhar, is there still a problem with current 3.2/3?
> 
> If you are no longer working on this, I think we should close as languishing/postponed.

I am not working on this yet (its low prio), but will try building 3.2 on HP-UX once 3.2 final is released.
msg139309 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-27 16:11
« Python 3.2 was released on February 20th, 2011. » (from python.org)
msg139313 - (view) Author: Sridhar Ratnakumar (srid) Date: 2011-06-27 16:39
On 2011-06-27, at 9:11 AM, Éric Araujo wrote:

> Éric Araujo <merwok@netwok.org> added the comment:
> 
> « Python 3.2 was released on February 20th, 2011. » (from python.org)

My mistake; I meant to say 3.2.1 final.
msg140344 - (view) Author: Jim Schneider (jschneid) Date: 2011-07-14 17:08
Martin - <sys/_mbstate_t.h> provides a definition for mbstate_t only (at least on HP/UX 11i V2.0).  I can verify that the problem still exists for Python 3.2.1.  I am working on a workaround for this issue, and I will attach a patch once I get it to build.
msg140352 - (view) Author: Jim Schneider (jschneid) Date: 2011-07-14 18:18
I got it to build on HP-UX 11.  However, there are a lot of compiler warnings about type mismatches, the _ctypes, _multiprocessing and termios modules failed to build, and "make test" died after not finding a usable "binascii" module.

To get it to build, I did the following:
1)  Applied the patch I attached to issue 12561
2)  Created a directory sys, and copied /usr/include/sys/stdsyms.h into it.
3)  Did "chmod 644" on sys/stdsyms.h and applied the patch stdsyms.patch that I've attached to this issue to it.
4)  Ran configure with the argument "CPPFLAGS=-I."

At this point, make ran to completion, and produced a python binary.  However, "make test" dies within seconds of starting up.
msg140353 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-07-14 18:19
Jim, the question remains why it fails to compile then. If the type is defined, why does it give an error message "but has not been defined as a type"???
msg140354 - (view) Author: Jim Schneider (jschneid) Date: 2011-07-14 18:28
Martin - sys/_mbstate.h is only included if _INCLUDE__STDC_A1_SOURCE is defined.  The only way this gets defined in the vendor-provided include files is if _XOPEN_SOURCE is defined and is equal to 500, or __STDC_VERSION__ is defined and is greater than or equal to 199901.

I've attached a patch to broaden the _XOPEN_SOURCE case (as the test should clearly have been >=, not ==).  Defining __STDC_VERSION__ to 199901 or greater will also do the job, but it feels more like a hack than just fixing what's broken in the vendor include files.
msg140356 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-07-14 18:35
That's a patch to HP-UX, right? Not one to Python.
msg140361 - (view) Author: Jim Schneider (jschneid) Date: 2011-07-14 18:37
Yes, it is a patch to an HP-provided C compiler system header file.  I cannot provide the actual file it patches, due to copyright limitations.
msg140426 - (view) Author: Jim Schneider (jschneid) Date: 2011-07-15 15:45
I am collecting HP/UX compiler workarounds in issue 12572.  I will be adding patches to it as I produce them, including a patch to fix this on HP/UX.
msg220449 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-13 14:45
Should this be closed in favour of issue 12572?
msg323738 - (view) Author: Michael Osipov (michael-o) * Date: 2018-08-18 22:28
I cannot reproduce this with HP-UX 11.31 and master + 3.7.

I opt to close this one.
History
Date User Action Args
2018-08-22 09:42:38vstinnersetstatus: open -> closed
resolution: out of date
stage: resolved
2018-08-18 23:52:35BreamoreBoysetnosy: - BreamoreBoy
2018-08-18 22:28:07michael-osetnosy: + michael-o
messages: + msg323738
2014-06-13 14:45:01BreamoreBoysetnosy: + BreamoreBoy
messages: + msg220449
2011-07-15 15:47:41vstinnersetnosy: + vstinner
2011-07-15 15:45:51jschneidsetmessages: + msg140426
2011-07-14 18:37:46jschneidsetmessages: + msg140361
2011-07-14 18:35:05loewissetmessages: + msg140356
2011-07-14 18:28:35jschneidsetmessages: + msg140354
2011-07-14 18:19:48loewissetmessages: + msg140353
2011-07-14 18:18:29jschneidsetfiles: + stdsyms.patch
keywords: + patch
messages: + msg140352
2011-07-14 17:08:50jschneidsetnosy: + jschneid
messages: + msg140344
2011-06-27 16:39:47sridsetmessages: + msg139313
2011-06-27 16:11:10eric.araujosetnosy: + eric.araujo
messages: + msg139309
2011-06-27 16:03:36sridsetmessages: + msg139307
2011-06-26 21:04:45terry.reedysetnosy: + terry.reedy

messages: + msg139222
versions: + Python 3.2, - Python 3.1
2009-07-02 19:15:30loewissetmessages: + msg90033
2009-07-02 18:29:25sridsetfiles: + test.c_XOPEN_SOURCE_500_preprocessed.txt
2009-07-02 18:27:53sridsetmessages: + msg90030
2009-07-02 18:25:35sridsetfiles: + test.c.preprocessed.txt
2009-07-02 18:24:56sridsetfiles: + _mbstate_t.h
2009-06-30 05:38:15loewissetmessages: + msg89906
title: compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used as a type, but has not been defined as a type -> compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used as a type, but has not been defined as a type
2009-06-29 22:49:20sridsetfiles: + wchar.h
2009-06-29 22:47:20sridsetfiles: + pyconfig.h
components: + Unicode
2009-06-29 22:47:01sridsetfiles: + config.log

messages: + msg89890
2009-05-12 18:38:56loewissetmessages: + msg87653
2009-05-12 18:25:20sridsetnosy: + loewis
messages: + msg87649
2009-05-11 21:24:33sridcreate