This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add support for WebAssembly System Interface (wasm32-wasi)
Type: enhancement Stage: patch review
Components: Build Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, kumaraditya, pmpp
Priority: normal Keywords: patch

Created on 2022-01-09 14:47 by christian.heimes, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 30507 merged christian.heimes, 2022-01-09 18:46
PR 32033 merged christian.heimes, 2022-03-21 19:07
PR 32266 merged christian.heimes, 2022-04-02 20:23
Messages (4)
msg410153 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-09 14:47
WASI is another WebAssembly platform similar to Emscripten (bpo-40280). Simply speaking Emscripten binaries (wasm32-emscripten) run inside a browser while WASI binaries target standalone runtimes like wasmtime [2][3] on the host. The lines are a bit blurry, as it is possible to run WASI binaries in the browser with help of JS-polyfills. WASI provides compile once, run anyway with JIT/AOT and sandboxing.

WASI is still under development and is lacking several core features:

- threading support and pthread API
- sockets
- signals are emulated (_WASI_EMULATED_SIGNAL and -lwasi-emulated-signal)
- DAC APIs like chmod, umask
- user-related APIs like pwd, grp
- dup(), dup2(), F_DUPFD

For 3.11 I plan to fix our use of #ifdef HAVE_FEATURE to make it easier to experiment with WASI. The pthread APIs need stubs, which I won't commit to 3.11 upstream yet.

[1] https://wasi.dev/
[2] https://github.com/bytecodealliance/wasmtime
[3] https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/
msg410198 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-10 10:59
dup() is required by _PyTokenizer_FindEncodingFilename(). I came up with this hack:

// from wasi-libc libc-top-half/musl/src/internal/stdio_impl.h
struct _IO_FILE {
    unsigned flags;
    unsigned char *rpos, *rend;
    int (*close)(FILE *);
    unsigned char *wend, *wpos;
    // incomplete
};

static int
dummy_close(FILE *fp) {
    return 0;
};

static FILE *
_Py_fdopen_borrow(int fd, const char *mode) {
    FILE *fp = fdopen(fd, mode);
    ((struct _IO_FILE*)fp)->close = dummy_close;
    return fp;
}



keithw on #wasi pointed out that fopencookie() can archive the same outcome without resorting to ABI-specific hack. A trivial implementation is straight forward:


typedef union {
    void *cookie;
    int fd;
} borrowed;

static ssize_t
borrow_read(void *cookie, char *buf, size_t size)
{
	borrowed b;
    b.cookie = cookie;
    return read(b.fd, (void *)buf, size);
}

static ssize_t
borrow_write(void *cookie, const char *buf, size_t size)
{
    errno = ENOTSUP;
    return -1;
}

static int
borrow_seek(void *cookie, off_t *off, int whence)
{
    borrowed b;
    b.cookie = cookie;
    off_t pos;
    pos = lseek(b.fd, *off, whence);
    if (pos == (off_t)-1) {
        return -1;
    } else {
        *off = pos;
        return 0;    
    }
}

static int
borrow_close(void *cookie)
{
    // does not close(fd)
    return 0;
}

FILE *
_Py_fdopen_borrow(int fd, const char *mode) {
    // only support read for now
    if (strcmp(mode, "r") != 0) {
        return NULL;
    }
    cookie_io_functions_t cookie_io = {
        borrow_read, borrow_write, borrow_seek, borrow_close
    };
  	// cookie is just the fd
    borrowed b;
    b.fd = fd;
    return fopencookie(b.cookie, "r", cookie_io);
}
msg410470 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-13 08:46
New changeset a6ca8eee2254762422f90cf94fbaac34f85db780 by Christian Heimes in branch 'main':
bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507)
https://github.com/python/cpython/commit/a6ca8eee2254762422f90cf94fbaac34f85db780
msg416588 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-04-02 21:11
New changeset 3df0e63aabef905b72fad78256f24b9270c63172 by Christian Heimes in branch 'main':
bpo-46315: Use fopencookie only on Emscripten 3.x and newer (GH-32266)
https://github.com/python/cpython/commit/3df0e63aabef905b72fad78256f24b9270c63172
History
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90473
2022-04-02 21:11:46christian.heimessetmessages: + msg416588
2022-04-02 20:23:26christian.heimessetpull_requests: + pull_request30332
2022-03-21 19:07:16christian.heimessetpull_requests: + pull_request30124
2022-01-13 08:46:14christian.heimessetmessages: + msg410470
2022-01-10 10:59:10christian.heimessetmessages: + msg410198
2022-01-10 06:51:55kumaradityasetnosy: + kumaraditya
2022-01-09 22:01:48pmppsetnosy: + pmpp
2022-01-09 18:46:26christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request28712
2022-01-09 14:47:43christian.heimescreate