The set() operators (__or__, __and__, __sub__, __xor__, and their in-place counterparts) require that the parameter also be an instance of set().

They're documented that way:  "This precludes error-prone constructions like set('abc') &  'cbs' in favor of the more readable set('abc').intersection('cbs')."

However, an unintended consequence of this behavior is that they don't inter-operate with user-created types that derive from collections.Set.

That leads to oddities like this:

MySimpleSet() | set()  # This works
set() | MySimpleSet()  # Raises TypeError

(MySimpleSet is a minimal class derived from collections.Set for illustrative purposes -- set attached file)

collections.Set's operators accept any iterable.

I'm not 100% certain what the correct behavior should be.  Perhaps set's operators should be a bit more liberal and accept any collections.Set instance, while collections.Set's operators should be a bit more conservative.  Perhaps not.  It's a little subjective.

It seems to me that at minimum set() and collections.Set() should inter-operate and have the same behavior.

I should add:

I discovered the inconsistency while working on my sortedset class, which provides the same interface as set() but is also indexable like a list (e.g., S[0] always returns the minimum element, S[-1] returns the maximum element, etc.).

sortedset derives from collections.MutableSet, but it's challenging to precisely emulate set() when collections.MutableSet and set() don't work the same way. ;-)

Guido, do you have a recommendation?

No idea, I don't even know what collections.Set is. :-(

In my opinion, the set's operator should be a bit more liberal and accept any collections.Set instances. Given collections.Set is an ABC and isinstance(set, collections.Set) is True, the set methods should(strong recommended) follow all the generalized abstract semantic definition in the ABC. This according to PEP 3119:
"""
In addition, the ABCs define a minimal set of methods that establish the characteristic behavior of the type. Code that discriminates objects based on their ABC type can trust that those methods will always be present. Each of these methods are accompanied by an generalized abstract semantic definition that is described in the documentation for the ABC. These standard semantic definitions are not enforced, but are strongly recommended.
"""

The collections.Set defines __or__() as this (for example):
"""
    def __or__(self, other):
        if not isinstance(other, Iterable):
            return NotImplemented
        chain = (e for s in (self, other) for e in s)
        return self._from_iterable(chain)
"""
which means the "|" operator should accept all iterable. So I think it's better to make set's methods should be more liberal.

Raymond, do you agree with Ray's analysis?

The operator methods in setobject.c should be liberalized to accept instances of collections.Set as arguments.  For speed, they should continue to check PyAnySet_Check(other) first and then if that fails, fall back to testing PyObject_IsInstance(other, collections.Set).  

Internally, the set methods will still need to process "other" as just an iterable container because it cannot rely on elements in "other" as being hashable (for example, the ListBasedSet in the docs does not require hashability) or unique (as perceived by setobject.c it may not work with some set implementing a key-function for an equivalence class whose key-function would be unknown to setobject.c which relies on __hash__ and __eq__).

To implement PyObject_IsInstance(other, collections.Set), there may be a bootstrap issue (with the C code being compiled and runnable before _abcoll.py is able to create the Set ABC).  If so, it may be necessary to create an internal _BaseSet object in setobject.c that can be used in collections.Set.  Alternatively, the code in setobject.c can lazily (at runtime) lookup collections.Set by name and cache it so that we only do one successful lookup per session.

Whatever approach is taken, it should be done with an eye towards the larger problem that Python is filled with concrete isinstance() checks that pre-date ABCs and many of those need to be liberalized (accepting a registered ABC and providing different execution paths for known and unknown concrete types).

> The operator methods in setobject.c should be liberalized to accept
> instances of collections.Set as arguments.

Under this plan, set() and collections.Set will still have slightly different behavior.  collections.Set will be more liberal and accept any iterable.  Are you okay with that?  I don't feel strongly about this point; I just want to make sure it's a conscious decision.

I do feel strongly that set and collections.Set should be able to inter-operate nicely and the proposal satisfies that requirement so I would be happy with it.

> To implement PyObject_IsInstance(other, collections.Set), there may
> be a bootstrap issue (with the C code being compiled and runnable
> before _abcoll.py is able to create the Set ABC). Alternatively, 
> the code in setobject.c can lazily (at runtime) lookup 
> collections.Set by name and cache it so that we only do one
> successful lookup per session.

I favor the lazy lookup approach.

> Whatever approach is taken, it should be done with an eye towards 
> the larger problem that Python is filled with concrete isinstance()
> checks that pre-date ABCs and many of those need to be liberalized
> (accepting a registered ABC and providing different execution paths
> for known and unknown concrete types).

Agreed.  Ideally, the "PyObject_IsInstance(other, collections.Set)" logic would be abstracted out as much as possible so other parts of Python can make similar checks without needing tons of boilerplate code in every spot.

For what it's worth, I don't think we will find as many inconsistency issues with ABCs other than Set.  Set has methods that take another Set and return a third Set.  That forces different concrete implementations of the Set ABC to interact in a way that won't come up for a Sequence or Mapping.

(I suppose that Sequence.extend or MutableMapping.update are somewhat similar, but list.extend and dict.update are already very liberal in what they accept as a parameter.)

Rough cut at a first patch is attached.

Still thinking about whether Set operations should be accepting any iterable or whether they should be tightened to expect other Set instances.  The API for set() came from set.py which was broadly discussed and widely exercised.  Guido was insistent that non-sets be excluded from the operator interactions (list.__iadd__ being on his list of regrets).   That was probably a good decision, but the Set API violated this norm and it did not include named methods like difference(), update(), and intersection() to handle the iterable cases.

Also, still thinking about whether the comparison operators should be making tight or loose checks.

Daniel, do you have time to work on this one?

If so, go ahead an make setobject.c accept any instance of collections.Set and make the corresponding change to the ABCs:

    def __or__(self, other):
        if not isinstance(other, Set):
            return NotImplemented
        chain = (e for s in (self, other) for e in s)
        return self._from_iterable(chain)

The code in the attached prelim.patch has working C code isinstance(x, collections.Set), but the rest of the patch that applies is has not been tested.  It needs to be applied very carefully and thoughtfully because:
* internally, the self and other can get swapped on a binary call
* we can't make *any* assumptions about "other" (that duplicates have actually been eliminated or the the elements are even hashable).

The most reliable thing to do for the case where PyAnySet(obj) is False but isinstance(obj, collections.Set) is true is to call the named method such as s.union(other) instead of continuing with s.__or__ which was designed only with real sets in mind.

Yes, I can take a stab at it.

No need to rush this for the beta.  It's a bug fix and can go in at any time.  The important thing is that we don't break the C code.  The __ror__ magic method would still need to do the right thing and the C code needs to defend against the interpreter swapping self and other.

Would it be sufficient to:

1) Restrict collections.Set()'s operators to accept collection.Set but not arbitrary iterables, and
2) Fix Issue2226 and let set() | MySimpleSet() work via collections.Set.__ror__

Attached is a patch that implements this approach, nominally fixing both this and Issue2226.

This solutions seems much too simple in light of how long I've been thinking about these bugs.  I suspect there are code hobgoblins waiting to ambush me. ;)

If the code were acting exactly as documented, I would consider this a feature request. But "require that the parameter also be an instance of set()" (from original message) is too limited.

>>> set() | frozenset()
set()

So 'set' in "their operator based counterparts require their arguments to be sets." (doc) seems to be meant to be more generic, in which case 'instance of collections.Set' seems reasonable. To be clear, the doc could be updated to "... sets, frozensets, and other instances of collections.Set."

"Both set and frozenset support set to set comparisons. " This includes comparisons between the two classes.

>>> set() == frozenset()
True

so perhaps comparisons should be extended also.

Review of set-with-Set.patch:

Looks good overall. 
I agree that restricting operations to instances of Set rather than Iterable is correct.

Implementing "__rsub__" in terms of - (subtraction) means that infinite recursion is a possibility. It also creates an unnecessary temporary.
Could you just reverse the expression used in __sub__?

Would you add tests for comparisons; Set() == set(), etc.
There are probably tested implicitly in the rest of the test suite, but explicit tests would be good.

Heads up, Issue #16373.

Armin pointed out in http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ that one nasty consequence of the remaining part of issue 2226 and this bug is making it much harder than it should be to use the ItemsView, KeysView and ValuesView from collections.abc to implement third party mappings that behave like the builtin dict.