>>> Actually, you should be able to just remove the asserts.

Aha, thanks! 

Yes: #define PY_SSIZE_T_CLEAN is defined at the top of Objects/exceptions.c, so yes, Py_VaBuildValue is redirected to _Py_VaBuildValue_SizeT, so that 
  PyEval_CallFunction
calls:
  Py_VaBuildValue(format, vargs)
which is preprocessed to:
  _Py_VaBuildValue_SizeT
which calls:
  return va_build_value(format, va, FLAG_SIZE_T);
which means that "s#" is processed with flags==FLAG_SIZE_T, and thus as "Py_ssize_t" within do_mkvalue:
				if (flags & FLAG_SIZE_T)
					n = va_arg(*p_va, Py_ssize_t);
				else
					n = va_arg(*p_va, int);

So, yes, it does look like the three assert lines can simply be removed.