I think this is the expected behavior.

It is expected by IEEE 754 that nan != nan, so that behavior exists:

>>> nan = float('nan')
>>> nan2 = float('nan')
>>> assert nan != nan2
>>> assert nan != nan

However, for "practicality beats purity" (speed) reasons, we compare containers by first checking if the entries are identical (aliases of the same object), before falling back to the actual comparisons. So identical entries make for equal containers, even if the entries don't compare equal.

>>> assert nan is nan       # quick check for identity succeeds,
>>> assert [nan] == [nan]   # therefore these compare equal

>>> assert nan is not nan2  # quick check for identity fails,
>>> assert nan != nan2      # fall back to actual comparison, which fails
>>> assert [nan] != [nan2]  # therefore these do not compare equal

When you serialize and deserialize the container with pickle, you make new entries that are no longer aliases.

From https://docs.python.org/3.9/reference/expressions.html#comparisons:
"The built-in containers typically assume identical objects are equal to themselves."