Issue36596
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-04-11 02:07 by cks, last changed 2022-04-11 14:59 by admin.
Messages (4) | |||
---|---|---|---|
msg339915 - (view) | Author: Chris Siebenmann (cks) | Date: 2019-04-11 02:07 | |
The easiest reproduction of this is: import tarfile tarfile.open("/dev/zero", "r:") (If you use plain "r" you get a hang in attempted lzma decoding.) I believe this is probably due to a missing 'elif self.offset == 0:' in the 'except EOFHeaderError' exception handling case that almost all of the other exception handlers have. This appears to be a very long standing issue based on the history of the code. |
|||
msg340488 - (view) | Author: Read Hughes (rthugh02) | Date: 2019-04-18 14:05 | |
GNU description of tar file format: http://www.gnu.org/software/tar/manual/html_node/Standard.html Particular quotes that are relevant: >Physically, an archive consists of a series of file entries terminated by an end-of-archive entry, which consists of two 512 blocks of zero bytes >Each file archived is represented by a header block which describes the file, followed by zero or more blocks which give the contents of the file. At the end of the archive file there are two 512-byte blocks filled with binary zeros as an end-of-file marker The header itself is 257 bytes padded with NUL until it reaches 512. No input other than this, just trying to bring any relevant information to this issue that may help |
|||
msg342764 - (view) | Author: Jeffrey Kintscher (Jeffrey.Kintscher) * | Date: 2019-05-17 20:47 | |
I did some testing with BSD and GNU tar to compare with Python's behavior. jfoo:~ jeff$ tar --version bsdtar 2.8.3 - libarchive 2.8.3 jeff@albarino:~$ tar --version tar (GNU tar) 1.28 Both BSD tar and GNU tar can create an empty tar file that consists of all zero bytes. BSD tar creates a 1 KB file: jfoo:~ jeff$ tar -cf tarfilename.tar -T /dev/null jfoo:~ jeff$ hexdump tarfilename.tar 0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000400 jfoo:~ jeff$ tar -tf tarfilename.tar jfoo:~ jeff$ echo $? 0 while GNU tar creates a 10 KB file: jeff@albarino:~$ tar -cf tarfilename.tar -T /dev/null jeff@albarino:~$ hexdump tarfilename.tar 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0002800 jeff@albarino:~$ tar -tf tarfilename.tar jeff@albarino:~$ echo $? 0 GNU tar will also leave a tar file with 10 KB of zeros when all contents have been deleted (BSD tar doesn't support deletion): jeff@albarino:~$ tar cf empty.tar tarfilename.tar jeff@albarino:~$ hexdump empty.tar 0000000 6174 6672 6c69 6e65 6d61 2e65 6174 0072 0000010 0000 0000 0000 0000 0000 0000 0000 0000 * 0000060 0000 0000 3030 3030 3636 0034 3030 3130 0000070 3537 0031 3030 3130 3537 0031 3030 3030 0000080 3030 3432 3030 0030 3331 3634 3637 3430 0000090 3331 0037 3130 3432 3637 2000 0030 0000 00000a0 0000 0000 0000 0000 0000 0000 0000 0000 * 0000100 7500 7473 7261 2020 6a00 6665 0066 0000 0000110 0000 0000 0000 0000 0000 0000 0000 0000 0000120 0000 0000 0000 0000 6a00 6665 0066 0000 0000130 0000 0000 0000 0000 0000 0000 0000 0000 * 0005000 jeff@albarino:~$ tar --delete -f empty.tar tarfilename.tar jeff@albarino:~$ hexdump empty.tar 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0002800 jfoo:~ jeff$ tar -tf empty.tar jfoo:~ jeff$ echo $? 0 According to the POSIX.1 standard, "[t]he last physical block shall always be the full size, so logical records after the two zero logical records may contain undefined data." (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#). It looks like any file starting with 1,024 bytes of zeros is a valid tar archive per BSD tar, GNU tar, and the POSIX.1 standard. However, BSD tar and GNU tar disagree about files starting with 512 bytes of zeros followed by 512 bytes of garbage. First, I constructed such a file for testing (zr.tar): jfoo:~ jeff$ dd if=/dev/zero of=zr.tar bs=512 count=1 1+0 records in 1+0 records out 512 bytes transferred in 0.000060 secs (8521761 bytes/sec) jfoo:~ jeff$ dd if=/dev/random of=zr.tar bs=512 count=1 oseek=1 1+0 records in 1+0 records out 512 bytes transferred in 0.000056 secs (9138228 bytes/sec) jfoo:~ jeff$ ls -l zr.tar -rw-r--r-- 1 jeff staff 1024 May 17 13:14 zr.tar jfoo:~ jeff$ hexdump zr.tar 0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000200 d7 56 a9 8d 26 11 a4 d8 9a 96 15 04 8d 4b 31 5d 0000210 33 2b 20 ae a2 23 09 8c 60 a1 73 12 a1 ab 73 61 0000220 69 eb 88 bf 8a 7d 6b 9a c5 79 b6 c9 9b a9 5a 6d 0000230 4b 4a 81 a7 71 da 90 24 3f 8f 43 a9 95 a0 20 bb 0000240 93 0f b2 be 7e 4d 80 49 aa 61 19 a2 6b b5 5c f4 0000250 e0 34 7f 99 a0 d3 29 08 9a 25 97 96 d4 d0 07 e4 0000260 90 1c 60 97 9a 23 d3 25 38 54 97 8b 71 a0 83 40 0000270 a6 f9 19 1b 3f 6e bc 5b 06 22 20 fc ff fe 7b eb 0000280 35 9b 52 57 14 83 90 7f d3 e8 f4 72 58 96 16 8c 0000290 09 ad 2a 2f ad fd 43 09 96 eb 7c 8f fc a6 14 d9 00002a0 18 34 38 b6 6a 5a ff 66 6d 46 cb 77 7a 5c 1e 72 00002b0 3e 27 05 3a b0 c4 52 7b c8 cc 26 b9 c3 5f 39 27 00002c0 a3 49 9e f1 3f f8 7e 46 98 df 7c 9d e3 86 c3 72 00002d0 e1 ef 98 7d a1 96 4e 4b 82 bb f4 2b f3 71 6f 16 00002e0 fe 38 2d bc 2b 70 b3 e6 db 1b ad 44 13 06 28 e5 00002f0 3d 05 07 3c 5f 09 5b 90 67 09 0b 5a db 79 b7 27 0000300 8a 4b e5 b3 66 f0 7a 9d a5 c4 e3 a8 b4 b2 d2 c8 0000310 5d d1 27 81 03 25 33 f4 fb 6f 77 b1 df 9d fa cf 0000320 01 a7 70 40 b4 7f 6b ac 04 70 5c 29 06 6a 73 64 0000330 4f 15 92 3b 5e a4 34 95 e0 4b 04 be ca 87 e9 73 0000340 1e 63 98 f3 f1 fd be 7a de fe 84 27 b7 e4 db e0 0000350 fb 04 7f 9d f0 ae af a3 8e 0f c2 a7 80 e0 32 38 0000360 17 1e 47 37 48 9b 99 35 58 9d d5 83 1b 67 d4 e8 0000370 15 0d 00 bb 79 f3 37 59 c3 5e e9 1d 87 79 96 de 0000380 6c 89 35 34 0b b1 12 b2 a8 2d 61 dd f5 9a 19 e7 0000390 c1 c5 24 46 fa 23 f0 db 72 7f a5 18 aa e2 db 04 00003a0 1e cc a6 0f 9e 4e 00 d9 2d eb f9 fc c4 d5 8e 46 00003b0 ab c3 ed 53 98 df a8 81 26 f4 b5 0f b4 7f 12 a4 00003c0 4a aa 14 4c f5 aa dd ba 69 e5 a8 d5 b3 68 0b 9f 00003d0 1a aa 34 a4 60 09 c2 30 22 32 72 dd 2e f9 7a 79 00003e0 88 a3 6a 99 13 4f f4 27 db 02 2e cb a0 ec d8 4d 00003f0 fe 68 44 0c 7b 3a 74 8d 8e cd ba 3e d8 ef cb 97 0000400 GNU tar outputs a warning message, but still returns zero: jeff@albarino:~$ tar -tvf zr.tar tar: A lone zero block at 1 jeff@albarino:~$ echo $? 0 while BSD tar silently accepts the file: jfoo:~ jeff$ tar -tvf zr.tar jfoo:~ jeff$ echo $? 0 Python also accepts the file as valid: >>> tarfile.open("zr.tar", "r") <tarfile.TarFile object at 0x10efa2820> Personally, I think that an error should be returned if the file starts with a zero block followed by a non-zero block. However, changing Python to do that would make its behavior inconsistent with two of the most widely used tar utilities. |
|||
msg343732 - (view) | Author: Jeffrey Kintscher (Jeffrey.Kintscher) * | Date: 2019-05-28 05:00 | |
I recommend closing this issue since the behavior is the same as the BSD and GNU tar utilities. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:13 | admin | set | github: 80777 |
2019-05-28 05:00:37 | Jeffrey.Kintscher | set | type: behavior messages: + msg343732 |
2019-05-17 20:47:58 | Jeffrey.Kintscher | set | nosy:
+ Jeffrey.Kintscher messages: + msg342764 |
2019-04-18 14:05:03 | rthugh02 | set | nosy:
+ rthugh02 messages: + msg340488 |
2019-04-12 03:14:00 | eitan.adler | set | nosy:
+ eitan.adler |
2019-04-11 07:14:24 | hitbox | set | nosy:
+ hitbox |
2019-04-11 02:16:39 | xtreak | set | nosy:
+ lars.gustaebel, serhiy.storchaka |
2019-04-11 02:07:20 | cks | create |