Issue1524938
Created on 2006-07-19 02:46 by markmat, last changed 2009-04-28 21:57 by pitrou.
| Messages (9) | |||
|---|---|---|---|
| msg29202 - (view) | Author: Mark Matusevich (markmat) | Date: 2006-07-19 02:46 | |
Also the gc behavior is consistent with the documentation, I beleave it is wrong. I think, that Gc should be called automatically before any memory allocation is raised. Example 1: for i in range(700): a = [range(5000000)] a.append(a) print i This example will crash on any any PC with less then 20Gb RAM. On my PC (Windows 2000, 256Mb) it crashes at i==7. Also, this example can be fixed by addition of a call to gc.collect() in the loop, in real cases it may be unreasonable. |
|||
| msg29203 - (view) | Author: Rene Dudfield (illume) | Date: 2006-07-19 23:20 | |
Logged In: YES
user_id=2042
Perhaps better than checking before every memory allocation,
would be to check once a memory error happens in an allocation.
That way there is only the gc hit once there is low memory.
So...
res = malloc(...);
if(!res) {
gc.collect();
}
res = malloc(...);
if(!res) {
raise memory error.
}
|
|||
| msg29204 - (view) | Author: Martin v. Löwis (loewis) | Date: 2006-07-23 20:00 | |
Logged In: YES user_id=21627 This is very difficult to implement. The best way might be to introduce yet another allocation function, one that invokes gc before failing, and call that function in all interesting places (of which there are many). Contributions are welcome and should probably start with a PEP first. |
|||
| msg29205 - (view) | Author: Mark Matusevich (markmat) | Date: 2006-07-23 20:11 | |
Logged In: YES user_id=1337765 This is exectly what I meant. For my recollection, this is the policy in Java GC. I never had to handle MemoryError in Java, because I knew, that I really do not have any more memory. |
|||
| msg29206 - (view) | Author: Mark Matusevich (markmat) | Date: 2006-07-23 20:19 | |
Logged In: YES user_id=1337765 Sorry, my last comment was to illume (I am slow typer :( ) |
|||
| msg29207 - (view) | Author: Jim Jewett (jimjjewett) | Date: 2006-08-02 21:52 | |
Logged In: YES user_id=764593 Doing it everywhere would be a lot of painful changes. Adding the "oops, failed, call gc and try again" to to PyMem_* (currently PyMem_Malloc, PyMem_Realloc, PyMem_New, and PyMem_Resize, but Brett may be changing that) is far more reasonable. Whether it is safe to call gc from there is a different question. |
|||
| msg29208 - (view) | Author: Mark Matusevich (markmat) | Date: 2006-08-03 10:02 | |
Logged In: YES user_id=1337765 Another problem related to the above example: there is a time waste due to a memory swap before the MemoryError. Possible solution is to use a dynamic memory limit: GC is called when the limit is reached, then the limit is adjusted according to the memory left. |
|||
| msg29209 - (view) | Author: Martin v. Löwis (loewis) | Date: 2006-08-03 16:43 | |
Logged In: YES user_id=21627 The example is highly constructed, and it is pointless to optimize for a boundary case. In the average application, garbage collection is invoked often enough to reclaim memory before swapping occurs. |
|||
| msg86769 - (view) | Author: Antoine Pitrou (pitrou) | Date: 2009-04-28 21:57 | |
Lowering priority since, as Martin said, it shouldn't be needed in real-life situations. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2009-04-28 21:57:57 | pitrou | set | priority: high -> low nosy: + pitrou messages: + msg86769 type: feature request -> resource usage stage: needs patch |
| 2009-04-27 23:48:03 | ajaksu2 | set | versions: + Python 3.1, Python 2.7, - Python 2.6 |
| 2008-01-05 19:47:02 | christian.heimes | set | priority: normal -> high versions: + Python 2.6 |
| 2006-07-19 02:46:19 | markmat | create | |