os.rename() is atomic on Linux, but on Windows it raises an error if the destination does already exist.

Not atomic pseudo-code for Windows:
  if exists(b):
     unlink(b)
  rename(a, b)

Windows offers different functions depending on the version:
 - MoveFileTransacted(): atomic! version >= (Windows Vista, Windows Server 2008)
 - ReplaceFile(): version >= Windows 2000
 - MoveFileEx() with MOVEFILE_REPLACE_EXISTING and MOVEFILE_WRITE_THROUGH flags: not atomic (eg. "If the file is to be moved to a different volume, the function simulates the move by using the CopyFile and DeleteFile functions."), version >= Windows 2000

I don't think that it's possible to write an atomic rename (file) function for any OS, so it's only a "best effort" atomic function. The documentation will give a list of OS on which the operation *is* atomic (eg. Linux).

Note: os.rename() uses MoveFileW() on Windows.