I have a c function to encrypt values which returns an array of bytes.  The function returns proper values outside of python.  When used as a python function, the result is incomplete usually 10-20% of the time.  If I add a sleep(1) call before returning from the function, my success rate goes to 100%.  While this works, it is unacceptable as it will create enormous latency in my application.

static PyObject *method_encrypt(PyObject *self, PyObject *args) {
	char *keyval, *str = NULL, output[512];
	Py_ssize_t count=0;
	PyObject *retval;

	if(!PyArg_ParseTuple(args, "ss", &str, &keyval)) {
		return NULL;
	}

	encryptBlowfishCfb(str, &count, output, keyval);

	retval = Py_BuildValue("y#", output, count);
	//sleep(1);
	return retval;
}

What do you mean by "incomplete"? Does it return less data or invalid data? Could you please paste your implementation of encryptBlowfishCfb(), too?

I have not gone to the extent of comparing the direct output against what python gets, although it appears to be incomplete.  The following is my encrypt function which has been used successfully for many years in a separate c program, so I have high confidence in it.

char * encryptBlowfishCfb(const char * inStr, Py_ssize_t *count, char * output, char * encrkey) {
	MCRYPT td;
	char IV[10];
	int len, i;
	char block_buffer[512];
	time_t t;

	srand((unsigned) time(&t));
	strcpy(IV, "");
	for (i = 0 ; i < 8 ; i++ ) {
		IV[i] =  mset[(rand() % 62)];
	}
	td = mcrypt_module_open(MCRYPT_BLOWFISH, NULL, MCRYPT_CFB, NULL);
	mcrypt_generic_init(td, encrkey, strlen(encrkey), IV);
	memset(block_buffer, 0, sizeof(block_buffer));
	strcpy(block_buffer, inStr);
	i = strlen(inStr);
	mcrypt_generic(td, &block_buffer, i);
	block_buffer[i] = '\0';
	strcpy(&block_buffer[i], IV);
	mcrypt_generic_deinit(td);
	mcrypt_module_close(td);

	len = i + 8 + 1;
	memset(output, 0, 512);
	strncpy(output, block_buffer, len -1);
	*count = i + 8;

	return output;
}

Does mcrypt_generic() output base64 or ASCII-only data? Since you are converting the output to bytes, I assume the output may contain any byte. In that case strcpy() is not safe. You have to use memcpy().

Fun fact: Nintendo had a similar bug many years ago, check out "trucha bug"

Christian -

Thank you for this.  I did as you suggested however, I still have the same problem.  As I pointed out in my original message, the problem does not exist if I insert a sleep(1) statement prior to returning from the function.  Additional to that, I previously had printf('.') statement in place of the sleep statement.  When I ran 10,000 iterations in python and printed my pass/fail totals, I still had a few hundred dots print which tells me that there has to be some sort of timing or buffer transfer issue between c and python.  Does that make sense?

Py_BuildValue("y#", output, count) is equivalent to PyBytes_FromStringAndSize(output, count). The function returns a copy of the input string as a new bytes object. It's very unlikely that the code is broken. It's been around for ages and it's a core feature.

Still no bueno.

David:

If you could give us an example showing the inputs, the actual outputs, and how they differ from what you expect, that would be helpful. Otherwise it's a lot of guessing on our part.

A sleep(1) call affects exactly one aspect of the program: the state of the PRNG rand(). You re-initialize the process globale RNG in every function call with srand((unsigned) time(&t)). time() has a granularity of one second.

Attached are the basic files.  As you can see in the example test9.py, I am generating a random string, encrypting it, decrypting it, and comparing the decrypted result with the original value.

This is intended to run on linux and requires the mcrypt library (libmcrypt-dev on debian based distro).  I'm at a loss and any help is greatly appreciated.

Could you please give us an example for an incorrect output and corresponding correct output as bytes representation?

Christian/Eric -

Thank you both so much for taking time on this.  Christian had pointed out the use of memcpy vs strncpy.  It turns out that while strncpy is designed to copy a specific number of chars, it turns out that it stops on null.  I did make the change Christian suggested however, it turns out that strncpy was also used in the decrypt function which was also subject to the same problem.

Don't feel bad about it. Nintendo made a very similar mistake. The trucha bug made it trivial to bypass DRM of Wii games.