This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Optional parameter got passed even when not in the call
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ajindal, eric.smith, ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2021-02-12 13:30 by ajindal, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg386864 - (view) Author: Abhilash Jindal (ajindal) Date: 2021-02-12 13:30
Seeing a very unexpected behavior. A call to __init__ method is receiving extra arguments not present in the caller. Following is a debugger session to demonstrate. Notice that "data_dict" is not being passed in the caller, yet the callee is receiving it!

```python
(Pdb) bt
  /Users/apple/Library/Python/3.8/bin/celery(10)<module>()
-> sys.exit(main())
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/__main__.py(15)main()
-> sys.exit(_main())
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/bin/celery.py(213)main()
-> return celery(auto_envvar_prefix="CELERY")
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/core.py(829)__call__()
-> return self.main(*args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/core.py(782)main()
-> rv = self.invoke(ctx)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/core.py(1259)invoke()
-> return _process_result(sub_ctx.command.invoke(sub_ctx))
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/core.py(1066)invoke()
-> return ctx.invoke(self.callback, **ctx.params)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/core.py(610)invoke()
-> return callback(*args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/click/decorators.py(21)new_func()
-> return f(get_current_context(), *args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/bin/base.py(132)caller()
-> return f(ctx, *args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/bin/worker.py(327)worker()
-> worker.start()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/worker/worker.py(203)start()
-> self.blueprint.start(self)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/bootsteps.py(116)start()
-> step.start(parent)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/bootsteps.py(365)start()
-> return self.obj.start()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/concurrency/base.py(129)start()
-> self.on_start()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/concurrency/prefork.py(107)on_start()
-> P = self._pool = Pool(processes=self.limit,
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/concurrency/asynpool.py(460)__init__()
-> super().__init__(processes, *args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/pool.py(1046)__init__()
-> self._create_worker_process(i)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/concurrency/asynpool.py(477)_create_worker_process()
-> return super()._create_worker_process(i)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/pool.py(1158)_create_worker_process()
-> w.start()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/process.py(124)start()
-> self._popen = self._Popen(self)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/context.py(333)_Popen()
-> return Popen(process_obj)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/popen_fork.py(24)__init__()
-> self._launch(process_obj)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/popen_fork.py(79)_launch()
-> code = process_obj._bootstrap()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/process.py(327)_bootstrap()
-> self.run()
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/process.py(114)run()
-> self._target(*self._args, **self._kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/pool.py(292)__call__()
-> sys.exit(self.workloop(pid=pid))
  /Users/apple/Library/Python/3.8/lib/python/site-packages/billiard/pool.py(362)workloop()
-> result = (True, prepare_result(fun(*args, **kwargs)))
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/app/trace.py(580)_trace_task_ret()
-> R, I, T, Rstr = trace_task(app.tasks[name],
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/app/trace.py(536)trace_task()
-> return task.__trace__(uuid, args, kwargs, request)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/app/trace.py(405)trace_task()
-> R = retval = fun(*args, **kwargs)
  /Users/apple/Library/Python/3.8/lib/python/site-packages/celery/app/trace.py(697)__protected_call__()
-> return self.run(*args, **kwargs)
  /Users/apple/Documents/research/hint/code/scope/core/nodes.py(23)process()
-> processor(idx)
  /Users/apple/Documents/research/hint/code/scope/core/nodes.py(96)__call__()
-> k = next(iter(self.named_in_edges))
> /Users/apple/Documents/research/hint/code/scope/examples/word_count.py(55)__call__()
-> out_row = Row(node_name, self.schema, from_rows=[input_row])
  /Users/apple/Documents/research/hint/code/scope/model/row.py(21)__init__()
-> self.lnode_name = lnode_name
(Pdb) l.
 50  	    def __call__(self, input_row: Row, *args: Any, **kwargs: Any) -> Generator[Row, None, None]:
 51  	        node_name = kwargs['node_name']
 52  	        line, _ = input_row.pget(self.in_col)
 53  	        words = line.split(self.separator)
 54  	        for word in words:
 55  ->	            out_row = Row(node_name, self.schema, from_rows=[input_row])
 56  	            out_row.pset(self.out_col, word.strip(' \t\r\n,!?.'))
 57  	            yield out_row
 58
 59
 60  	class LowerCase(Processor):
(Pdb) d
> /Users/apple/Documents/research/hint/code/scope/model/row.py(21)__init__()
-> self.lnode_name = lnode_name
(Pdb) l.
 17  	    def __init__(self, lnode_name, schema, id = '*', data_dict = {}, read_only = False, from_rows = None):
 18  	        # type(str, Schema, str, Dict[bytes, bytes], bool, List[Row])
 19  	        if lnode_name == 'processor-1':
 20  	            from celery.contrib import  rdb; rdb.set_trace()
 21  ->	        self.lnode_name = lnode_name
 22  	        self._schema = schema
 23  	        self.id = id
 24  	        self.data_dict = data_dict
 25
 26  	        self.parents = RowRelations(self, RelationType.PARENT)
(Pdb) lnode_name
'processor-1'
(Pdb) data_dict
{b'line': b'When will you realize... Vienna waits for you?'}
(Pdb) from_rows
[<scope.model.row.Row object at 0x103201400>]
(Pdb) from_rows[0]
<scope.model.row.Row object at 0x103201400>
(Pdb) id
'*'
(Pdb) read_only
False
(Pdb)
```
msg386865 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-02-12 14:33
It's hard to tell because you didn't show the whole program, but this looks like a mutable default argument problem. Basically you're changing the default value in a different call to Row().

https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects
msg386868 - (view) Author: Abhilash Jindal (ajindal) Date: 2021-02-12 15:17
Aah TIL. Thanks.
History
Date User Action Args
2022-04-11 14:59:41adminsetgithub: 87372
2021-02-12 15:17:12ajindalsetstatus: open -> closed
resolution: not a bug
messages: + msg386868

stage: resolved
2021-02-12 14:33:50eric.smithsetnosy: + eric.smith
messages: + msg386865
2021-02-12 13:30:29ajindalcreate