This issue is probably related to issue ??? but I have created it as a separate issue. When shuffle doesn't crash it sometimes (or maybe always - I haven't fully analysed this yet) looses most of the elements in the list that it is supposed to be shuffling. Here is an extract of the code that I'm using:

import io
from io import StringIO 
from lxml import etree 
import random  

filename_xml = 'MockExam5.xml'
with io.open(filename_xml, mode="r", encoding="utf-8") as xml_file:
    xml_to_check = xml_file.read()
doc = etree.parse(StringIO(xml_to_check))
exams = doc.getroot()
questions_element = exams.find("questions")
logmsg(L_TRACE, "There are now " + str(len(questions_element.findall("question"))) + " questions")
logmsg(L_TRACE, "Randomising order of questions in this exam")
random.shuffle(questions_element)
logmsg(L_TRACE, "Finished randomise")
logmsg(L_TRACE, "There are now " + str(len(questions_element.findall("question"))) + " questions")

And here is the log produced by this code:

21-03-24 18:10:11.989 line:  2057 file: D:\XPS_8700 Extended Files\Users\RowanB\Documents\My_Scripts NEW\mockexam\put_exam.py 2 There are now 79 questions
21-03-24 18:10:11.991 line:  2065 file: D:\XPS_8700 Extended Files\Users\RowanB\Documents\My_Scripts NEW\mockexam\put_exam.py 2 Randomising order of questions in this exam
21-03-24 18:10:11.992 line:  2067 file: D:\XPS_8700 Extended Files\Users\RowanB\Documents\My_Scripts NEW\mockexam\put_exam.py 2 Finished randomise
21-03-24 18:10:11.993 line:  2068 file: D:\XPS_8700 Extended Files\Users\RowanB\Documents\My_Scripts NEW\mockexam\put_exam.py 2 There are now 6 questions

How come the shuffle starts off with 79 elements, and finishes with 6?

Thanks - Rowan

Same advice as issue 43616: please provide an example we can run.

This most likely a problem with how you're using lxml, and not a bug in python. But since we can't test it, we can't know for sure.

Are you sure it's "a list"? At least print out `type(questions_element)`. `random.shuffle()` doesn't contain any code _capable_ of changing a list's length. It only does indexed accessing of the list:

...
    for i in reversed(range(1, len(x))):
        # pick an element in x[:i+1] with which to exchange x[i]
        j = randbelow(i + 1)
        x[i], x[j] = x[j], x[i]

That's about all there is to it. Note that, for this purpose, it doesn't matter want `randbelow()` does, because that function never even sees `x`.

The standard library isn't at fault here.  Please file this an an LXML bug.

Reproducer:

    from lxml.etree import Element

    root = Element('outer')
    root.append(Element('zero'))
    root.append(Element('one'))
    root.append(Element('two'))
    print([e.tag for e in root])
    root[1], root[0] = root[0], root[1]
    print([e.tag for e in root])

This outputs:

   ['zero', 'one', 'two']
   ['one', 'two']

Replacing the import with:

   from xml.etree.ElementTree import Element

Gives the expected result:

   ['zero', 'one', 'two']
   ['one', 'zero', 'two']

Interestingly, this isn't an LXML bug.  It is a documented difference from how the standard library works:

    https://lxml.de/tutorial.html#elements-are-lists

So, if you want use random.shuffle(), you need the standard library ElementTree instead of lxml.

This:

    from lxml.etree import Element

    root = Element('outer')
    root.append(Element('zero'))
    root.append(Element('one'))
    root.append(Element('two'))
    root[0] = root[1]
    print([e.tag for e in root])

Produces:

   ['one', 'two']

Yes, this is neither a bug in CPython (or its stdlib) nor in lxml. It's how things work. Don't use these two together.

As an immediate fix to your problem, replace this line:

    random.shuffle(questions_element)

with:

    questions = list(questions_element)
    random.shuffle(questions)
    questions_element[:] = questions