test error unet_segmentation_3d_ignite_clearml.ipynb

[wyli]

[2023-10-26T01:42:06.351Z] Running ./experiment_management/unet_segmentation_3d_ignite_clearml.ipynb
[2023-10-26T01:42:06.351Z] Checking PEP8 compliance...
[2023-10-26T01:42:07.278Z] Running notebook...
[2023-10-26T01:42:07.278Z] Before:
[2023-10-26T01:42:07.278Z]     "max_epochs = 10\n",
[2023-10-26T01:42:07.278Z] After:
[2023-10-26T01:42:07.278Z]     "max_epochs = 1\n",
[2023-10-26T01:42:15.360Z] MONAI version: 1.3.0+10.g85243f52
[2023-10-26T01:42:15.360Z] Numpy version: 1.22.2
[2023-10-26T01:42:15.360Z] Pytorch version: 2.1.0a0+29c30b1
[2023-10-26T01:42:15.360Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2023-10-26T01:42:15.360Z] MONAI rev id: 85243f52085b0946870fc7808c484c8b639eaf1b
[2023-10-26T01:42:15.360Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2023-10-26T01:42:15.360Z] 
[2023-10-26T01:42:15.360Z] Optional dependencies:
[2023-10-26T01:42:15.360Z] Pytorch Ignite version: 0.4.11
[2023-10-26T01:42:15.360Z] ITK version: 5.3.0
[2023-10-26T01:42:15.360Z] Nibabel version: 5.1.0
[2023-10-26T01:42:15.360Z] scikit-image version: 0.22.0
[2023-10-26T01:42:15.360Z] scipy version: 1.11.1
[2023-10-26T01:42:15.360Z] Pillow version: 9.2.0
[2023-10-26T01:42:15.360Z] Tensorboard version: 2.9.0
[2023-10-26T01:42:15.360Z] gdown version: 4.7.1
[2023-10-26T01:42:15.360Z] TorchVision version: 0.16.0a0
[2023-10-26T01:42:15.360Z] tqdm version: 4.65.0
[2023-10-26T01:42:15.360Z] lmdb version: 1.4.1
[2023-10-26T01:42:15.360Z] psutil version: 5.9.4
[2023-10-26T01:42:15.360Z] pandas version: 1.5.2
[2023-10-26T01:42:15.360Z] einops version: 0.6.1
[2023-10-26T01:42:15.360Z] transformers version: 4.21.3
[2023-10-26T01:42:15.360Z] mlflow version: 2.7.1
[2023-10-26T01:42:15.360Z] pynrrd version: 1.0.0
[2023-10-26T01:42:15.360Z] clearml version: 1.13.2rc1
[2023-10-26T01:42:15.360Z] 
[2023-10-26T01:42:15.360Z] For details about installing the optional dependencies, please visit:
[2023-10-26T01:42:15.360Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2023-10-26T01:42:15.360Z] 
[2023-10-26T01:42:17.879Z] papermill  --progress-bar -k python3
[2023-10-26T01:42:18.135Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-10-26T01:42:18.135Z]   warnings.warn(
[2023-10-26T01:42:50.847Z] 
Executing:   0%|          | 0/35 [00:00<?, ?cell/s]
Executing:   3%|▎         | 1/35 [00:01<00:42,  1.25s/cell]
Executing:   9%|▊         | 3/35 [00:09<01:52,  3.50s/cell]
Executing:  14%|█▍        | 5/35 [00:17<01:53,  3.77s/cell]
Executing:  37%|███▋      | 13/35 [00:26<00:38,  1.74s/cell]
Executing:  43%|████▎     | 15/35 [00:28<00:31,  1.58s/cell]
Executing:  60%|██████    | 21/35 [00:30<00:13,  1.00cell/s]
Executing:  60%|██████    | 21/35 [00:32<00:21,  1.54s/cell]
[2023-10-26T01:42:50.847Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-10-26T01:42:50.847Z]   warnings.warn(
[2023-10-26T01:42:50.847Z] Traceback (most recent call last):
[2023-10-26T01:42:50.847Z]   File "/usr/local/bin/papermill", line 8, in <module>
[2023-10-26T01:42:50.847Z]     sys.exit(papermill())
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2023-10-26T01:42:50.847Z]     return self.main(*args, **kwargs)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2023-10-26T01:42:50.847Z]     rv = self.invoke(ctx)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2023-10-26T01:42:50.847Z]     return ctx.invoke(self.callback, **ctx.params)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2023-10-26T01:42:50.847Z]     return __callback(*args, **kwargs)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2023-10-26T01:42:50.847Z]     return f(get_current_context(), *args, **kwargs)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 250, in papermill
[2023-10-26T01:42:50.847Z]     execute_notebook(
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 128, in execute_notebook
[2023-10-26T01:42:50.847Z]     raise_for_execution_errors(nb, output_path)
[2023-10-26T01:42:50.847Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 232, in raise_for_execution_errors
[2023-10-26T01:42:50.847Z]     raise error
[2023-10-26T01:42:50.847Z] papermill.exceptions.PapermillExecutionError: 
[2023-10-26T01:42:50.847Z] ---------------------------------------------------------------------------
[2023-10-26T01:42:50.847Z] Exception encountered at "In [10]":
[2023-10-26T01:42:50.847Z] ---------------------------------------------------------------------------
[2023-10-26T01:42:50.847Z] MissingSchema                             Traceback (most recent call last)
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/session.py:818, in Session._do_refresh_token(self, old_token, exp)
[2023-10-26T01:42:50.847Z]     817 try:
[2023-10-26T01:42:50.847Z] --> 818     res = self._send_request(
[2023-10-26T01:42:50.847Z]     819         method=Request.def_method,
[2023-10-26T01:42:50.847Z]     820         service="auth",
[2023-10-26T01:42:50.847Z]     821         action="login",
[2023-10-26T01:42:50.847Z]     822         auth=auth,
[2023-10-26T01:42:50.847Z]     823         headers=headers,
[2023-10-26T01:42:50.847Z]     824         refresh_token_if_unauthorized=False,
[2023-10-26T01:42:50.847Z]     825         params={"expiration_sec": exp} if exp else {},
[2023-10-26T01:42:50.847Z]     826     )
[2023-10-26T01:42:50.847Z]     827     try:
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/session.py:408, in Session._send_request(self, service, action, version, method, headers, auth, data, json, refresh_token_if_unauthorized, params)
[2023-10-26T01:42:50.847Z]     407     self._logger.debug("%s: %s [%d bytes, %d headers]", method.upper(), url, size, len(headers or {}))
[2023-10-26T01:42:50.847Z] --> 408 res = self.__http_session.request(
[2023-10-26T01:42:50.847Z]     409     method, url, headers=headers, auth=auth, data=data, json=json, timeout=timeout, params=params)
[2023-10-26T01:42:50.847Z]     410 if self._verbose and self._logger:
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/requests/sessions.py:575, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
[2023-10-26T01:42:50.847Z]     563 req = Request(
[2023-10-26T01:42:50.847Z]     564     method=method.upper(),
[2023-10-26T01:42:50.847Z]     565     url=url,
[2023-10-26T01:42:50.847Z]    (...)
[2023-10-26T01:42:50.847Z]     573     hooks=hooks,
[2023-10-26T01:42:50.847Z]     574 )
[2023-10-26T01:42:50.847Z] --> 575 prep = self.prepare_request(req)
[2023-10-26T01:42:50.847Z]     577 proxies = proxies or {}
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/requests/sessions.py:486, in Session.prepare_request(self, request)
[2023-10-26T01:42:50.847Z]     485 p = PreparedRequest()
[2023-10-26T01:42:50.847Z] --> 486 p.prepare(
[2023-10-26T01:42:50.847Z]     487     method=request.method.upper(),
[2023-10-26T01:42:50.847Z]     488     url=request.url,
[2023-10-26T01:42:50.847Z]     489     files=request.files,
[2023-10-26T01:42:50.847Z]     490     data=request.data,
[2023-10-26T01:42:50.847Z]     491     json=request.json,
[2023-10-26T01:42:50.847Z]     492     headers=merge_setting(
[2023-10-26T01:42:50.847Z]     493         request.headers, self.headers, dict_class=CaseInsensitiveDict
[2023-10-26T01:42:50.847Z]     494     ),
[2023-10-26T01:42:50.847Z]     495     params=merge_setting(request.params, self.params),
[2023-10-26T01:42:50.847Z]     496     auth=merge_setting(auth, self.auth),
[2023-10-26T01:42:50.847Z]     497     cookies=merged_cookies,
[2023-10-26T01:42:50.847Z]     498     hooks=merge_hooks(request.hooks, self.hooks),
[2023-10-26T01:42:50.847Z]     499 )
[2023-10-26T01:42:50.847Z]     500 return p
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/requests/models.py:368, in PreparedRequest.prepare(self, method, url, headers, files, data, params, auth, cookies, hooks, json)
[2023-10-26T01:42:50.847Z]     367 self.prepare_method(method)
[2023-10-26T01:42:50.847Z] --> 368 self.prepare_url(url, params)
[2023-10-26T01:42:50.847Z]     369 self.prepare_headers(headers)
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /usr/local/lib/python3.10/dist-packages/requests/models.py:439, in PreparedRequest.prepare_url(self, url, params)
[2023-10-26T01:42:50.847Z]     438 if not scheme:
[2023-10-26T01:42:50.847Z] --> 439     raise MissingSchema(
[2023-10-26T01:42:50.847Z]     440         f"Invalid URL {url!r}: No scheme supplied. "
[2023-10-26T01:42:50.847Z]     441         f"Perhaps you meant https://{url}?"
[2023-10-26T01:42:50.847Z]     442     )
[2023-10-26T01:42:50.847Z]     444 if not host:
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] MissingSchema: Invalid URL "''/auth.login": No scheme supplied. Perhaps you meant https://''/auth.login?
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] During handling of the above exception, another exception occurred:
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] LoginError                                Traceback (most recent call last)
[2023-10-26T01:42:50.847Z] Cell In[10], line 24
[2023-10-26T01:42:50.847Z]      21 task_name = "UNet segmentation 3d"
[2023-10-26T01:42:50.847Z]      22 project_name = "Monai example"
[2023-10-26T01:42:50.847Z] ---> 24 train_clearml_stats_handler = ClearMLStatsHandler(
[2023-10-26T01:42:50.847Z]      25     task_name=task_name, project_name=project_name, log_dir=log_dir, output_transform=lambda x: x
[2023-10-26T01:42:50.847Z]      26 )
[2023-10-26T01:42:50.847Z]      27 train_clearml_stats_handler.attach(trainer)
[2023-10-26T01:42:50.847Z] 
[2023-10-26T01:42:50.847Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/handlers/clearml_handlers.py:115, in ClearMLStatsHandler.__init__(self, project_name, task_name, output_uri, tags, reuse_last_task_id, continue_last_task, auto_connect_frameworks, auto_connect_arg_parser, *args, **kwargs)
[2023-10-26T01:42:50.847Z]      89 def __init__(
[2023-10-26T01:42:50.848Z]      90     self,
[2023-10-26T01:42:50.848Z]      91     project_name: str | None = "MONAI",
[2023-10-26T01:42:50.848Z]    (...)
[2023-10-26T01:42:50.848Z]     100     **kwargs: Any,
[2023-10-26T01:42:50.848Z]     101 ) -> None:
[2023-10-26T01:42:50.848Z]     102     """
[2023-10-26T01:42:50.848Z]     103     Args:
[2023-10-26T01:42:50.848Z]     104         project_name: ClearML project name, default to 'MONAI'.
[2023-10-26T01:42:50.848Z]    (...)
[2023-10-26T01:42:50.848Z]     112 
[2023-10-26T01:42:50.848Z]     113     """
[2023-10-26T01:42:50.848Z] --> 115     ClearMLHandler.__init__(
[2023-10-26T01:42:50.848Z]     116         self,
[2023-10-26T01:42:50.848Z]     117         project_name=project_name,
[2023-10-26T01:42:50.848Z]     118         task_name=task_name,
[2023-10-26T01:42:50.848Z]     119         output_uri=output_uri,
[2023-10-26T01:42:50.848Z]     120         tags=tags,
[2023-10-26T01:42:50.848Z]     121         reuse_last_task_id=reuse_last_task_id,
[2023-10-26T01:42:50.848Z]     122         continue_last_task=continue_last_task,
[2023-10-26T01:42:50.848Z]     123         auto_connect_frameworks=auto_connect_frameworks,
[2023-10-26T01:42:50.848Z]     124         auto_connect_arg_parser=auto_connect_arg_parser,
[2023-10-26T01:42:50.848Z]     125     )
[2023-10-26T01:42:50.848Z]     126     TensorBoardStatsHandler.__init__(self, *args, **kwargs)
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/handlers/clearml_handlers.py:66, in ClearMLHandler.__init__(self, project_name, task_name, output_uri, tags, reuse_last_task_id, continue_last_task, auto_connect_frameworks, auto_connect_arg_parser)
[2023-10-26T01:42:50.848Z]      64     self.clearml_task = clearml.Task.current_task()
[2023-10-26T01:42:50.848Z]      65 else:
[2023-10-26T01:42:50.848Z] ---> 66     self.clearml_task = clearml.Task.init(
[2023-10-26T01:42:50.848Z]      67         project_name=project_name,
[2023-10-26T01:42:50.848Z]      68         task_name=task_name,
[2023-10-26T01:42:50.848Z]      69         output_uri=output_uri,
[2023-10-26T01:42:50.848Z]      70         tags=tags,
[2023-10-26T01:42:50.848Z]      71         reuse_last_task_id=reuse_last_task_id,
[2023-10-26T01:42:50.848Z]      72         continue_last_task=continue_last_task,
[2023-10-26T01:42:50.848Z]      73         auto_connect_frameworks=auto_connect_frameworks,
[2023-10-26T01:42:50.848Z]      74         auto_connect_arg_parser=auto_connect_arg_parser,
[2023-10-26T01:42:50.848Z]      75     )
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/task.py:563, in Task.init(cls, project_name, task_name, task_type, tags, reuse_last_task_id, continue_last_task, output_uri, auto_connect_arg_parser, auto_connect_frameworks, auto_resource_monitoring, auto_connect_streams, deferred_init)
[2023-10-26T01:42:50.848Z]     561 elif not is_sub_process_task_id:
[2023-10-26T01:42:50.848Z]     562     try:
[2023-10-26T01:42:50.848Z] --> 563         task = cls._create_dev_task(
[2023-10-26T01:42:50.848Z]     564             default_project_name=project_name,
[2023-10-26T01:42:50.848Z]     565             default_task_name=task_name,
[2023-10-26T01:42:50.848Z]     566             default_task_type=task_type,
[2023-10-26T01:42:50.848Z]     567             tags=tags,
[2023-10-26T01:42:50.848Z]     568             reuse_last_task_id=reuse_last_task_id,
[2023-10-26T01:42:50.848Z]     569             continue_last_task=continue_last_task,
[2023-10-26T01:42:50.848Z]     570             detect_repo=False if (
[2023-10-26T01:42:50.848Z]     571                     isinstance(auto_connect_frameworks, dict) and
[2023-10-26T01:42:50.848Z]     572                     not auto_connect_frameworks.get('detect_repository', True)) else True,
[2023-10-26T01:42:50.848Z]     573             auto_connect_streams=auto_connect_streams,
[2023-10-26T01:42:50.848Z]     574         )
[2023-10-26T01:42:50.848Z]     575     except MissingConfigError as e:
[2023-10-26T01:42:50.848Z]     576         if not ENV_IGNORE_MISSING_CONFIG.get():
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/task.py:3529, in Task._create_dev_task(cls, default_project_name, default_task_name, default_task_type, tags, reuse_last_task_id, continue_last_task, detect_repo, auto_connect_streams)
[2023-10-26T01:42:50.848Z]    3527 # create a new task
[2023-10-26T01:42:50.848Z]    3528 if not default_task_id:
[2023-10-26T01:42:50.848Z] -> 3529     task = cls(
[2023-10-26T01:42:50.848Z]    3530         private=cls.__create_protection,
[2023-10-26T01:42:50.848Z]    3531         project_name=default_project_name,
[2023-10-26T01:42:50.848Z]    3532         task_name=default_task_name,
[2023-10-26T01:42:50.848Z]    3533         task_type=default_task_type,
[2023-10-26T01:42:50.848Z]    3534         log_to_backend=True,
[2023-10-26T01:42:50.848Z]    3535     )
[2023-10-26T01:42:50.848Z]    3536     # no need to reload yet, we clear this before the end of the function
[2023-10-26T01:42:50.848Z]    3537     task._reload_skip_flag = True
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/task.py:206, in Task.__init__(self, private, **kwargs)
[2023-10-26T01:42:50.848Z]     202     raise UsageError(
[2023-10-26T01:42:50.848Z]     203         'Task object cannot be instantiated externally, use Task.current_task() or Task.get_task(...)')
[2023-10-26T01:42:50.848Z]     204 self._repo_detect_lock = threading.RLock()
[2023-10-26T01:42:50.848Z] --> 206 super(Task, self).__init__(**kwargs)
[2023-10-26T01:42:50.848Z]     207 self._arguments = _Arguments(self)
[2023-10-26T01:42:50.848Z]     208 self._logger = None
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_interface/task/task.py:161, in Task.__init__(self, session, task_id, log, project_name, task_name, task_type, log_to_backend, raise_on_validation_errors, force_create)
[2023-10-26T01:42:50.848Z]     159 task_id = self._resolve_task_id(task_id, log=log) if not force_create else None
[2023-10-26T01:42:50.848Z]     160 self.__edit_lock = None
[2023-10-26T01:42:50.848Z] --> 161 super(Task, self).__init__(id=task_id, session=session, log=log)
[2023-10-26T01:42:50.848Z]     162 self._project_name = None
[2023-10-26T01:42:50.848Z]     163 self._storage_uri = None
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_interface/base.py:149, in IdObjectBase.__init__(self, id, session, log, **kwargs)
[2023-10-26T01:42:50.848Z]     148 def __init__(self, id, session=None, log=None, **kwargs):
[2023-10-26T01:42:50.848Z] --> 149     super(IdObjectBase, self).__init__(session, log, **kwargs)
[2023-10-26T01:42:50.848Z]     150     self._data = None
[2023-10-26T01:42:50.848Z]     151     self._id = None
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_interface/base.py:41, in InterfaceBase.__init__(self, session, log, **kwargs)
[2023-10-26T01:42:50.848Z]      39 def __init__(self, session=None, log=None, **kwargs):
[2023-10-26T01:42:50.848Z]      40     super(InterfaceBase, self).__init__()
[2023-10-26T01:42:50.848Z] ---> 41     self._session = session or self._get_default_session()
[2023-10-26T01:42:50.848Z]      42     self._log = log or self._create_log()
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_interface/base.py:119, in InterfaceBase._get_default_session(cls)
[2023-10-26T01:42:50.848Z]     116 @classmethod
[2023-10-26T01:42:50.848Z]     117 def _get_default_session(cls):
[2023-10-26T01:42:50.848Z]     118     if not InterfaceBase._default_session:
[2023-10-26T01:42:50.848Z] --> 119         InterfaceBase._default_session = Session(
[2023-10-26T01:42:50.848Z]     120             initialize_logging=False,
[2023-10-26T01:42:50.848Z]     121             config=config_obj,
[2023-10-26T01:42:50.848Z]     122             api_key=ENV_ACCESS_KEY.get(),
[2023-10-26T01:42:50.848Z]     123             secret_key=ENV_SECRET_KEY.get(),
[2023-10-26T01:42:50.848Z]     124         )
[2023-10-26T01:42:50.848Z]     125     return InterfaceBase._default_session
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/session.py:157, in Session.__init__(self, worker, api_key, secret_key, host, logger, verbose, config, http_retries_config, **kwargs)
[2023-10-26T01:42:50.848Z]     154     from clearml.config import ConfigWrapper
[2023-10-26T01:42:50.848Z]     155     self.config = ConfigWrapper._init()
[2023-10-26T01:42:50.848Z] --> 157 self._connect()
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/session.py:224, in Session._connect(self)
[2023-10-26T01:42:50.848Z]     217     raise MissingConfigError()
[2023-10-26T01:42:50.848Z]     219 super(Session, self).__init__(
[2023-10-26T01:42:50.848Z]     220     **self.__token_manager_kwargs,
[2023-10-26T01:42:50.848Z]     221     token_expiration_threshold_sec=token_expiration_threshold_sec,
[2023-10-26T01:42:50.848Z]     222     req_token_expiration_sec=req_token_expiration_sec
[2023-10-26T01:42:50.848Z]     223 )
[2023-10-26T01:42:50.848Z] --> 224 self.refresh_token()
[2023-10-26T01:42:50.848Z]     226 local_logger = self._LocalLogger(self._logger)
[2023-10-26T01:42:50.848Z]     228 # update api version from server response
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/token_manager.py:112, in TokenManager.refresh_token(self)
[2023-10-26T01:42:50.848Z]     111 def refresh_token(self):
[2023-10-26T01:42:50.848Z] --> 112     self._set_token(self._do_refresh_token(self.__token, exp=self.req_token_expiration_sec))
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] File /usr/local/lib/python3.10/dist-packages/clearml/backend_api/session/session.py:857, in Session._do_refresh_token(self, old_token, exp)
[2023-10-26T01:42:50.848Z]     854         raise LoginError("Response data mismatch: No 'token' in 'data' value from res, receive : {}, "
[2023-10-26T01:42:50.848Z]     855                          "exception: {}".format(res, ex))
[2023-10-26T01:42:50.848Z]     856 except Exception as ex:
[2023-10-26T01:42:50.848Z] --> 857     raise LoginError('Unrecognized Authentication Error: {} {}'.format(type(ex), ex))
[2023-10-26T01:42:50.848Z] 
[2023-10-26T01:42:50.848Z] LoginError: Unrecognized Authentication Error: <class 'requests.exceptions.MissingSchema'> Invalid URL "''/auth.login": No scheme supplied. Perhaps you meant https://''/auth.login?

cc @pollfly