Omnisci Docker Installation slow response to open jupyterlab

Dear Expert omnisci,

I wish you help me, I has install omnisci using docker compose and follow all tutorial from here : https://docs.omnisci.com/data-science/get-started-jupyter#installing-jupyter-with-omniscidb

but the problem is, while we want to open jupyter notebooks, its wait too long, maybe need time 1-2 minutes to open jupyter notebooks workspace, and if I open folder in workspace is also need time abour 40 seconds, also backward folder.

what must I do to optimize this?
Thank you.

Best Regards,
Dedy

Hi @dsugiarto,

to be sure that you aren’t experiencing some sort of network problems, because of the size of jupyter-lab JS’s size, cound you open the debug console of your browser and take a look to the netwrok tab?

You can press F12 and you should get an output like that (I used Chrome)

Regards

Hi @dsugiarto,

Can you check the versions you have? Is it OmniSci 5.4.1, and Jupyter Hub and Lab 0.3? Are you using the CPU or GPU version of Jupyter Lab? The JS size shouldn’t be too bad in 0.3, as it was in an earlier version.

Is it running on a server or your local machine? During that 40s navigating a directory, please check top to see what the cpu is doing.

Thanks,
Mike

hi @candido.dessanti

here my screenshoot about it, it always like this,

after that wait some time and goto workspace.
its always waitting TTFB about 12 minutes, I using VPN, but if I access omnisci its fast, but if open jupyterlab too long.

here omnisci is faster than i access jupyter above

Hi @mike.hinchey

yes I use Docker ee cuda version 5.4.1 and jupyter hub and lab v0.3
image

I using gpu version
my workstation is DGX Workstation ram 256G, 40 Core cpu, and have NVDIA GPU also,

is there’s any misconfiguration?

Hi @dsugiarto,

The only way I was able to reproduce your problem, placing my workstation on a sub 10Mbit network and calling thru the internet with a laptop connected thrue mobile on a 3g network with a quite low poerformance

I measured with speedtest.net and got those result

I did a reload on JupyterLab page disabling caching, so the background server was already started

I’m not sure I reproduced exactly your problem, because while I’m having a lot of TTFB, but also longer transfer time than yours.

Could you try to open a browser into your remote workstation and open jupyterlab? this way we can exclude you are face a netwrok problem

hi @candido.dessanti,

I will try it later because I still cant to connect directly my dgx worksation, LINUX GUI is not detected in monitor. I still work on it, if I has test I will give information about it.

I think its not depend by network, because my omnisci web server is fast to access and responsive, but if open jupyterlab on omnisci link icon this long time to response, starting kernel and movement between folder in jupyter is too long, is that docker problem?

Regards,
Dedy

Hi @dsugiarto,

If you followed the instructions to setup jupyterlab with Omnisci, also the webserver is installed into a container, and everything (immerse or Jupiter) is served by the container.

When I intentionally degraded the performance of my network, I was able to use Immerse, while the load time was around 12 seconds, instead of the usual 3, while the jupyterlab was unresponsive, taking over 1 minute to load and showing the “it’s taking to much time to load” message.

If you are connecting to the workstation with SSH, you should be able to tunnel the X-window protocol and open a local browser in the workstation.

In the meantime, I will try to dig further into alternative methods to find out what’s going wrong with your installation.

Hi @candido.dessanti

yes my omnisci is also in container also my jupyter but its strange if omnisci web and movement of the page is fast, but jupyter movement of page and load is so long, in docker logs its every process need time just about 4.5ms, but waitting time is to long.

i dont know i still cant open it, i try access it using cable to connect my laptop to hub in my dgx server but same still slow in jupyter access and movement page.

thank you for your effort I very appreciate.

finally i can connect to my dgx server because before lightdm is stopped, but if i try directly access from web browser in dgx workstation, its still long time to access jupyter, so not problem with network.

1 Like

I am assuming you have followed the guide to building the environment from the ground up.

Anyway, assuming I have just started the container with the docker-compose command, I should have those containers running.

root@zion16:/var/lib/omnisci# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
920dbe88fbb1 omnisci/omnisci-jupyterhub:v0.3 “/docker-entrypoint.…” 5 days ago Up About a minute 8000/tcp jupyterhub
9dddd7eab65a omnisci/omnisci-ee-cuda:v5.4.1 “/bin/sh -c '/omnisc…” 5 days ago Up 45 minutes 0.0.0.0:6273-6274->6273-6274/tcp, 0.0.0.0:6278->6278/tcp omnisciserver

get the logs of the container running jupyterhub with docker 920dbe88fbb1 logs command

[I 2020-11-13 08:39:22.457 JupyterHub app:2 120] Using Authenticator: jhub-omnisci-authenticator.omnisci_user_auth.OmniSciUserAuthenticator
[I 2020-11-13 08:39:22.457 JupyterHub app:2120] Using Spawner: dockerspawner.dockerspawner.DockerSpawner-0.11.1
[I 2020-11-13 08:39:22.461 JupyterHub app:1257] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
[I 2020-11-13 08:39:22.478 JupyterHub proxy:460] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2020-11-13 08:39:22.497 JupyterHub app:1563] Not using whitelist. Any authenticated user will be allowed.
[I 2020-11-13 08:39:22.522 JupyterHub app:2337] Hub API listening on http://172.19.0.2:8081/jupyter/hub/
[W 2020-11-13 08:39:22.523 JupyterHub proxy:516] Found proxy pid file: /srv/jupyterhub/jupyterhub-proxy.pid
[W 2020-11-13 08:39:22.523 JupyterHub proxy:528] Proxy no longer running at pid=39
[W 2020-11-13 08:39:22.524 JupyterHub proxy:642] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else…
[I 2020-11-13 08:39:22.524 JupyterHub proxy:645] Starting proxy @ http://:8000/jupyter/
08:39:22.923 - info: [ConfigProxy] Proxying http://*:8000 to (no default)
08:39:22.924 - info: [ConfigProxy] Proxy API at http://127.0.0.1:8001/api/routes
[I 2020-11-13 08:39:23.451 JupyterHub app:2362] Starting managed service cull_idle
[I 2020-11-13 08:39:23.451 JupyterHub service:316] Starting service ‘cull_idle’: [‘python’, ‘/srv/jupyterhub/cull_idle_servers.py’, ‘–timeout=3600’]
[I 2020-11-13 08:39:23.454 JupyterHub service:121] Spawning python /srv/jupyterhub/cull_idle_servers.py --timeout=3600
[I 2020-11-13 08:39:23.465 JupyterHub proxy:319] Checking routes
[I 2020-11-13 08:39:23.465 JupyterHub proxy:399] Adding default route for Hub: /jupyter/ => http://172.19.0.2:8081
[I 2020-11-13 08:39:23.468 JupyterHub app:2422] JupyterHub is now running at http://:8000/jupyter/
[I 2020-11-13 08:39:23.580 JupyterHub log:174] 200 GET /jupyter/hub/api/users (cull_idle@172.19.0.2) 11.82ms

Then login into Immmerse with the browser and open jupyter-lab.

with docker ps command, you should get the id of Jupiter lab container

root@zion16:/var/lib/omnisci# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1745b67785a1 omnisci/omnisci-jupyterlab-gpu:v0.3 “/omnisci/bin/docker…” About a minute ago Up About a minute 8888/tcp jupyterlab-candido
920dbe88fbb1 omnisci/omnisci-jupyterhub:v0.3 “/docker-entrypoint.…” 5 days ago Up 4 minutes 8000/tcp jupyterhub
9dddd7eab65a omnisci/omnisci-ee-cuda:v5.4.1 “/bin/sh -c '/omnisc…” 5 days ago Up 6 minutes 0.0.0.0:6273-6274->6273-6274/tcp, 0.0.0.0

Continue to the second part

seond part of the message

Then check for logs in the containers, just to check if there is something that could lead us to identify your performance problem.

My logs are the following (I called the user in the db as I called the user in my laptop)

Executing the command: jupyter labhub --ip=0.0.0.0 --port=8888 --notebook-dir=/home/jovyan --NotebookApp.default_url=/lab --NotebookApp.allow_origin=* --NotebookApp.terminals_enabled=True
[W 2020-11-13 08:42:11.957 SingleUserLabApp configurable:168] Config option `open_browser` not recognized by `SingleUserLabApp`.  Did you mean `browser`?
[I 2020-11-13 08:42:13.446 SingleUserLabApp extension:158] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 2020-11-13 08:42:13.446 SingleUserLabApp extension:159] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 2020-11-13 08:42:14.352 SingleUserLabApp singleuser:561] Starting jupyterhub-singleuser server version 1.1.0
[I 2020-11-13 08:42:14.355 SingleUserLabApp log:174] 302 GET /jupyter/user/candido/ -> /jupyter/user/candido/lab? (@172.19.0.2) 0.82ms
[W 2020-11-13 08:42:14.361 SingleUserLabApp _version:56] jupyterhub version 1.0.0 != jupyterhub-singleuser version 1.1.0. This could cause failure to authenticate and result in redirect loops!
[I 2020-11-13 08:42:14.361 SingleUserLabApp notebookapp:1924] Serving notebooks from local directory: /home/jovyan
[I 2020-11-13 08:42:14.361 SingleUserLabApp notebookapp:1924] The Jupyter Notebook is running at:
[I 2020-11-13 08:42:14.361 SingleUserLabApp notebookapp:1924] http://1745b67785a1:8888/jupyter/user/candido/
[I 2020-11-13 08:42:14.361 SingleUserLabApp notebookapp:1925] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2020-11-13 08:42:14.367 SingleUserLabApp singleuser:542] Updating Hub with activity every 300 seconds
[I 2020-11-13 08:42:14.417 SingleUserLabApp log:174] 302 GET /jupyter/user/candido/ -> /jupyter/user/candido/lab? (@192.168.1.23) 0.91ms
[I 2020-11-13 08:42:14.436 SingleUserLabApp log:174] 302 GET /jupyter/user/candido/lab? -> /jupyter/hub/api/oauth2/authorize?client_id=jupyterhub-user-candido&redirect_uri=%2Fjupyter%2Fuser%2Fcandido%2Foauth_callback&response_type=code&state=[secret] (@192.168.1.23) 2.63ms
[I 2020-11-13 08:42:14.573 SingleUserLabApp auth:981] Logged-in user {'kind': 'user', 'name': 'candido', 'admin': False, 'groups': [], 'server': '/jupyter/user/candido/', 'pending': None, 'created': '2020-11-08T06:28:23.262102Z', 'last_activity': '2020-11-13T08:42:14.570068Z', 'servers': None}
[I 2020-11-13 08:42:14.574 SingleUserLabApp log:174] 302 GET /jupyter/user/candido/oauth_callback?code=[secret]&state=[secret] -> /jupyter/user/candido/lab? (@192.168.1.23) 48.36ms
[I 2020-11-13 08:42:14.597 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab? (candido@192.168.1.23) 8.53ms
[I 2020-11-13 08:42:15.550 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256935556 (candido@192.168.1.23) 21.72ms
[I 2020-11-13 08:42:15.556 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/kernelspecs?1605256935557 (candido@192.168.1.23) 4.74ms
[I 2020-11-13 08:42:15.559 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/terminals?1605256935559 (candido@192.168.1.23) 1.34ms
[I 2020-11-13 08:42:15.660 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/?1605256935564 (candido@192.168.1.23) 25.37ms
[I 2020-11-13 08:42:15.663 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/kernelspecs?1605256935663 (candido@192.168.1.23) 1.48ms
[I 2020-11-13 08:42:15.664 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256935662 (candido@192.168.1.23) 1.04ms
[I 2020-11-13 08:42:15.666 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/terminals?1605256935664 (candido@192.168.1.23) 0.83ms
[I 2020-11-13 08:42:15.803 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/metrics?1605256935820 (candido@192.168.1.23) 7.40ms
[I 2020-11-13 08:42:16.277 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/nbconvert?1605256936031 (candido@192.168.1.23) 259.56ms
[I 2020-11-13 08:42:16.490 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/nbconvert?1605256936052 (candido@192.168.1.23) 212.54ms
[I 2020-11-13 08:42:16.492 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936076 (candido@192.168.1.23) 1.81ms
[I 2020-11-13 08:42:16.494 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936075 (candido@192.168.1.23) 1.41ms
[I 2020-11-13 08:42:16.498 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936077 (candido@192.168.1.23) 1.62ms
[I 2020-11-13 08:42:16.506 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936077 (candido@192.168.1.23) 2.24ms
[I 2020-11-13 08:42:16.519 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936078 (candido@192.168.1.23) 3.30ms
[I 2020-11-13 08:42:16.523 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936080 (candido@192.168.1.23) 2.54ms
[I 2020-11-13 08:42:16.525 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936081 (candido@192.168.1.23) 1.88ms
[I 2020-11-13 08:42:16.528 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936082 (candido@192.168.1.23) 1.72ms
[I 2020-11-13 08:42:16.533 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936083 (candido@192.168.1.23) 2.54ms
[I 2020-11-13 08:42:16.536 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936084 (candido@192.168.1.23) 2.12ms
[I 2020-11-13 08:42:16.542 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936085 (candido@192.168.1.23) 2.14ms
[I 2020-11-13 08:42:16.547 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936086 (candido@192.168.1.23) 1.91ms
[I 2020-11-13 08:42:16.550 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936087 (candido@192.168.1.23) 1.95ms
[I 2020-11-13 08:42:16.554 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936089 (candido@192.168.1.23) 1.51ms
[I 2020-11-13 08:42:16.556 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936088 (candido@192.168.1.23) 1.46ms
[I 2020-11-13 08:42:16.559 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936090 (candido@192.168.1.23) 1.47ms
[I 2020-11-13 08:42:16.562 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936091 (candido@192.168.1.23) 1.50ms
[I 2020-11-13 08:42:16.567 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936092 (candido@192.168.1.23) 3.45ms
[I 2020-11-13 08:42:16.570 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936093 (candido@192.168.1.23) 2.03ms
[I 2020-11-13 08:42:16.574 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936093 (candido@192.168.1.23) 2.93ms
[I 2020-11-13 08:42:16.577 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936101 (candido@192.168.1.23) 1.96ms
[I 2020-11-13 08:42:16.580 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936094 (candido@192.168.1.23) 1.96ms
[I 2020-11-13 08:42:16.589 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936102 (candido@192.168.1.23) 1.69ms
[I 2020-11-13 08:42:16.593 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1605256936103 (candido@192.168.1.23) 1.79ms
[I 2020-11-13 08:42:16.597 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256936107 (candido@192.168.1.23) 1.27ms
[I 2020-11-13 08:42:16.603 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/?content=1&1605256936119 (candido@192.168.1.23) 5.93ms
[I 2020-11-13 08:42:16.615 SingleUserLabApp handlers:144] Uploading file to /.jupyterscratch/.omniscisession
[I 2020-11-13 08:42:16.617 SingleUserLabApp log:174] 201 PUT /jupyter/user/candido/api/contents/.jupyterscratch/.omniscisession?1605256936118 (candido@172.19.0.2) 2.95ms
[I 2020-11-13 08:42:16.621 SingleUserLabApp handlers:144] Uploading file to /OmniSci_2020-11-13_08:42:16.ipynb
[I 2020-11-13 08:42:16.633 SingleUserLabApp log:174] 201 PUT /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08:42:16.ipynb?1605256936118 (candido@172.19.0.2) 12.71ms
[I 2020-11-13 08:42:16.637 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/workspaces/lab?1605256936118 (candido@172.19.0.2) 1.52ms
[I 2020-11-13 08:42:16.641 SingleUserLabApp log:174] 204 PUT /jupyter/user/candido/lab/api/workspaces/lab?1605256936118 (candido@172.19.0.2) 1.39ms
[I 2020-11-13 08:42:16.645 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/workspaces/lab?1605256936118 (candido@192.168.1.23) 2.00ms
[I 2020-11-13 08:42:16.672 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples?1605256936690 (candido@192.168.1.23) 4.18ms
[I 2020-11-13 08:42:16.674 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08%3A42%3A16.ipynb?content=0&1605256936692 (candido@192.168.1.23) 1.63ms
[I 2020-11-13 08:42:16.677 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_07%3A59%3A29.ipynb?content=0&1605256936691 (candido@192.168.1.23) 1.47ms
[I 2020-11-13 08:42:16.678 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb?content=0&1605256936693 (candido@192.168.1.23) 0.94ms
[I 2020-11-13 08:42:16.680 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb?content=0&1605256936695 (candido@192.168.1.23) 0.95ms
[I 2020-11-13 08:42:16.682 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb?content=0&1605256936697 (candido@192.168.1.23) 1.05ms
[I 2020-11-13 08:42:16.693 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py?content=0&1605256936698 (candido@192.168.1.23) 1.43ms
[I 2020-11-13 08:42:16.697 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples?content=1&1605256936714 (candido@192.168.1.23) 2.60ms
[I 2020-11-13 08:42:16.882 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256936909 (candido@192.168.1.23) 1.07ms
[I 2020-11-13 08:42:16.989 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_07%3A59%3A29.ipynb?type=notebook&content=1&1605256936986 (candido@192.168.1.23) 6.41ms
[W 2020-11-13 08:42:16.997 SingleUserLabApp manager:510] Notebook omnisci_examples/SQL Editor.ipynb is not trusted
[I 2020-11-13 08:42:17.002 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb?type=notebook&content=1&1605256936989 (candido@192.168.1.23) 11.95ms
[I 2020-11-13 08:42:17.006 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08%3A42%3A16.ipynb?type=notebook&content=1&1605256936985 (candido@192.168.1.23) 3.32ms
[W 2020-11-13 08:42:17.013 SingleUserLabApp manager:510] Notebook omnisci_examples/Ibis + Altair.ipynb is not trusted
[I 2020-11-13 08:42:17.019 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb?type=notebook&content=1&1605256936987 (candido@192.168.1.23) 12.15ms
[W 2020-11-13 08:42:17.029 SingleUserLabApp manager:510] Notebook omnisci_examples/Renderers.ipynb is not trusted
[I 2020-11-13 08:42:17.038 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb?type=notebook&content=1&1605256936988 (candido@192.168.1.23) 18.87ms
[I 2020-11-13 08:42:17.041 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py?format=text&type=file&content=1&1605256936990 (candido@192.168.1.23) 1.47ms
[I 2020-11-13 08:42:17.056 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_07%3A59%3A29.ipynb/checkpoints?1605256937056 (candido@192.168.1.23) 1.53ms
[I 2020-11-13 08:42:17.058 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_07%3A59%3A29.ipynb/checkpoints?1605256937057 (candido@192.168.1.23) 1.21ms
[I 2020-11-13 08:42:17.073 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_07%3A59%3A29.ipynb/checkpoints?1605256937059 (candido@192.168.1.23) 1.95ms
[W 2020-11-13 08:42:17.095 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb/checkpoints?1605256937120 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.096 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.096 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb/checkpoints?1605256937120 (candido@192.168.1.23) 2.13ms
[W 2020-11-13 08:42:17.098 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb/checkpoints?1605256937116 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.098 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.099 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/SQL%20Editor.ipynb/checkpoints?1605256937116 (candido@192.168.1.23) 2.17ms
[I 2020-11-13 08:42:17.146 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08%3A42%3A16.ipynb/checkpoints?1605256937151 (candido@192.168.1.23) 1.76ms
[I 2020-11-13 08:42:17.148 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256937123 (candido@192.168.1.23) 1.59ms
[I 2020-11-13 08:42:17.152 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08%3A42%3A16.ipynb/checkpoints?1605256937152 (candido@192.168.1.23) 1.64ms
[I 2020-11-13 08:42:17.155 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/contents/OmniSci_2020-11-13_08%3A42%3A16.ipynb/checkpoints?1605256937174 (candido@192.168.1.23) 1.50ms
[W 2020-11-13 08:42:17.165 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py/checkpoints?1605256937179 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.165 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.166 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py/checkpoints?1605256937179 (candido@192.168.1.23) 1.15ms
[W 2020-11-13 08:42:17.194 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py/checkpoints?1605256937181 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.194 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.195 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/debounce.py/checkpoints?1605256937181 (candido@192.168.1.23) 1.72ms
[W 2020-11-13 08:42:17.220 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb/checkpoints?1605256937241 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.220 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.221 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb/checkpoints?1605256937241 (candido@192.168.1.23) 2.07ms
[W 2020-11-13 08:42:17.223 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb/checkpoints?1605256937244 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.223 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.223 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Ibis%20%2B%20Altair.ipynb/checkpoints?1605256937244 (candido@192.168.1.23) 1.84ms
[I 2020-11-13 08:42:17.256 SingleUserLabApp kernelmanager:172] Kernel started: 891a221a-e7ac-4483-a720-a71304e0f983
[I 2020-11-13 08:42:17.257 SingleUserLabApp log:174] 201 POST /jupyter/user/candido/api/sessions?1605256937264 (candido@192.168.1.23) 19.79ms
[I 2020-11-13 08:42:17.268 SingleUserLabApp kernelmanager:172] Kernel started: a0ef80b7-e5e7-4c35-8ff1-fc85e6096c09
[I 2020-11-13 08:42:17.270 SingleUserLabApp log:174] 201 POST /jupyter/user/candido/api/sessions?1605256937266 (candido@192.168.1.23) 12.02ms
[I 2020-11-13 08:42:17.282 SingleUserLabApp kernelmanager:172] Kernel started: fa1ac6a5-af78-47fd-96fd-15cdd98d19af
[I 2020-11-13 08:42:17.283 SingleUserLabApp log:174] 201 POST /jupyter/user/candido/api/sessions?1605256937268 (candido@192.168.1.23) 12.04ms
[I 2020-11-13 08:42:17.286 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256937271 (candido@192.168.1.23) 1.48ms
[W 2020-11-13 08:42:17.384 SingleUserLabApp log:174] 404 GET (candido@192.168.1.23) 27.20ms
[W 2020-11-13 08:42:17.432 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb/checkpoints?1605256937460 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.433 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.433 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb/checkpoints?1605256937460 (candido@192.168.1.23) 2.31ms
[W 2020-11-13 08:42:17.436 SingleUserLabApp web:1667] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb/checkpoints?1605256937462 (192.168.1.23): Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.436 SingleUserLabApp handlers:608] Permission denied: omnisci_examples/.ipynb_checkpoints
[W 2020-11-13 08:42:17.437 SingleUserLabApp log:174] 403 GET /jupyter/user/candido/api/contents/omnisci_examples/Renderers.ipynb/checkpoints?1605256937462 (candido@192.168.1.23) 2.02ms
[I 2020-11-13 08:42:17.530 SingleUserLabApp log:174] 200 PATCH /jupyter/user/candido/api/sessions/6e863334-695b-4c99-ae59-8d49afa05511?1605256937517 (candido@192.168.1.23) 2.45ms
[I 2020-11-13 08:42:17.536 SingleUserLabApp log:174] 200 PATCH /jupyter/user/candido/api/sessions/6266de9a-0015-4cfd-85d9-bf5da496cc8f?1605256937521 (candido@192.168.1.23) 2.32ms
[I 2020-11-13 08:42:17.548 SingleUserLabApp kernelmanager:172] Kernel started: b1b61d68-518b-414d-a7d2-e690e39a426c
[I 2020-11-13 08:42:17.549 SingleUserLabApp log:174] 201 POST /jupyter/user/candido/api/sessions?1605256937526 (candido@192.168.1.23) 11.93ms
[I 2020-11-13 08:42:17.560 SingleUserLabApp kernelmanager:172] Kernel started: 341d2db4-036d-4d11-b480-b911761f2173
[I 2020-11-13 08:42:17.561 SingleUserLabApp log:174] 201 POST /jupyter/user/candido/api/sessions?1605256937524 (candido@192.168.1.23) 11.56ms
[I 2020-11-13 08:42:17.564 SingleUserLabApp log:174] 200 PATCH /jupyter/user/candido/api/sessions/9dc49f58-eef3-4a5c-adeb-ab5756aa5ade?1605256937528 (candido@192.168.1.23) 1.57ms
[I 2020-11-13 08:42:17.568 SingleUserLabApp log:174] 204 PUT /jupyter/user/candido/lab/api/workspaces/lab?1605256937536 (candido@192.168.1.23) 1.24ms
[I 2020-11-13 08:42:17.574 SingleUserLabApp log:174] 200 PATCH /jupyter/user/candido/api/sessions/1bb1eb34-e44f-45fd-a455-7ca04a1387bc?1605256937599 (candido@192.168.1.23) 1.44ms
[I 2020-11-13 08:42:17.579 SingleUserLabApp log:174] 200 PATCH /jupyter/user/candido/api/sessions/a8456e00-9c54-48c4-9730-4a9ceb82d374?1605256937606 (candido@192.168.1.23) 1.42ms
[I 2020-11-13 08:42:17.607 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/metrics?1605256937621 (candido@192.168.1.23) 13.59ms
[I 2020-11-13 08:42:17.705 SingleUserLabApp build_handler:49] Build is up to date
[I 2020-11-13 08:42:17.706 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/lab/api/build?1605256936009 (candido@192.168.1.23) 1691.96ms
[I 2020-11-13 08:42:18.703 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/891a221a-e7ac-4483-a720-a71304e0f983/channels?session_id=b7957bf8-64ec-4b3c-97c2-4ad84191b99a&token=[secret] (candido@192.168.1.23) 1138.13ms
[I 2020-11-13 08:42:18.778 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/891a221a-e7ac-4483-a720-a71304e0f983/channels?session_id=ef1ef4d1-7b27-4957-afc8-853cff53e845&token=[secret] (candido@192.168.1.23) 9.26ms
[I 2020-11-13 08:42:18.880 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/a0ef80b7-e5e7-4c35-8ff1-fc85e6096c09/channels?session_id=8321a867-06ce-4815-9874-1285263329d7&token=[secret] (candido@192.168.1.23) 13.17ms
[I 2020-11-13 08:42:18.912 SingleUserLabApp log:174] 200 GET /jupyter/user/candido/api/sessions?1605256938925 (candido@192.168.1.23) 2.53ms
[I 2020-11-13 08:42:18.959 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/a0ef80b7-e5e7-4c35-8ff1-fc85e6096c09/channels?session_id=728f36a2-42b4-4ef4-a876-0214ca6e1c73&token=[secret] (candido@192.168.1.23) 2.77ms
[I 2020-11-13 08:42:19.050 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=048f3fbb-42f8-4255-b5e2-e3c87b0a2f54&token=[secret] (candido@192.168.1.23) 12.20ms
[I 2020-11-13 08:42:19.116 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=52301bb8-02f7-4dce-8e68-3cab78cc0b01&token=[secret] (candido@192.168.1.23) 3.58ms
[I 2020-11-13 08:42:19.202 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=b104773d-6997-4a3f-b0a8-6ab608f7f49a&token=[secret] (candido@192.168.1.23) 3.58ms
[I 2020-11-13 08:42:19.275 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=b38c0c6a-6aef-4e13-b342-5bbf6cf846d7&token=[secret] (candido@192.168.1.23) 2.93ms
[I 2020-11-13 08:42:19.361 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=0af838d2-4278-40b8-b81f-ea45f4ddfe76&token=[secret] (candido@192.168.1.23) 2.62ms
[I 2020-11-13 08:42:19.436 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/fa1ac6a5-af78-47fd-96fd-15cdd98d19af/channels?session_id=c0790819-dcf1-4d2f-9a4f-d8051ccd8c9d&token=[secret] (candido@192.168.1.23) 5.84ms
[I 2020-11-13 08:42:19.523 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/b1b61d68-518b-414d-a7d2-e690e39a426c/channels?session_id=94216b5e-bc75-4d18-aa7f-f910456d8d7a&token=[secret] (candido@192.168.1.23) 9.56ms
[I 2020-11-13 08:42:19.625 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/b1b61d68-518b-414d-a7d2-e690e39a426c/channels?session_id=9cf37f9f-82d2-4e29-a700-04957ef22f4d&token=[secret] (candido@192.168.1.23) 3.51ms
[I 2020-11-13 08:42:19.716 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=be7c10f5-6a19-4e4f-bcad-ce2cb78ee093&token=[secret] (candido@192.168.1.23) 11.30ms
[I 2020-11-13 08:42:19.798 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=c5dda378-18cc-48d0-8ce4-c0e07cc6bc1e&token=[secret] (candido@192.168.1.23) 3.38ms
[I 2020-11-13 08:42:19.893 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=c7714398-5fe9-4998-9ad5-2254f798928d&token=[secret] (candido@192.168.1.23) 3.53ms
[I 2020-11-13 08:42:19.958 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=bd1e92d9-1db4-45a0-aa60-01fd19c4640c&token=[secret] (candido@192.168.1.23) 3.35ms
[I 2020-11-13 08:42:20.035 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=9a85d4bd-ca7d-459c-be4c-4bba24cb7156&token=[secret] (candido@192.168.1.23) 5.61ms
[I 2020-11-13 08:42:20.116 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=3ba50384-405d-4ac0-b117-91ff8431bbc5&token=[secret] (candido@192.168.1.23) 6.15ms
[I 2020-11-13 08:42:20.195 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=f81c6ca2-213f-48b4-a15d-2cb5cdac6458&token=[secret] (candido@192.168.1.23) 3.36ms
[I 2020-11-13 08:42:20.278 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=e678225f-eb9e-4cf8-918f-0a0e98c59f82&token=[secret] (candido@192.168.1.23) 3.35ms
[I 2020-11-13 08:42:20.343 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=e1693a14-9adc-4aaa-a670-50086e57545b&token=[secret] (candido@192.168.1.23) 3.65ms
[I 2020-11-13 08:42:20.429 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=d26609ca-fa2a-48f1-9e88-c4ff9bd88f82&token=[secret] (candido@192.168.1.23) 3.51ms
[I 2020-11-13 08:42:20.888 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=b7508ebf-5dd6-41cb-baa3-29d999e63aa7&token=[secret] (candido@192.168.1.23) 3.43ms
[I 2020-11-13 08:42:21.080 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=e1214bca-9cc8-46e0-a83e-2942715f1dbe&token=[secret] (candido@192.168.1.23) 22.84ms
[I 2020-11-13 08:42:21.199 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=c1dfb5ac-2465-4589-ba36-56fc54895a0b&token=[secret] (candido@192.168.1.23) 3.60ms
[I 2020-11-13 08:42:21.274 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=2595e9da-c730-46bd-8624-03f17f06af45&token=[secret] (candido@192.168.1.23) 3.39ms
[I 2020-11-13 08:42:21.362 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=4b151eba-072b-4a0e-a44a-14f5db7a8e1c&token=[secret] (candido@192.168.1.23) 5.44ms
[I 2020-11-13 08:42:21.434 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=5b54be07-f236-47aa-bc0b-9616f2030678&token=[secret] (candido@192.168.1.23) 2.85ms
[I 2020-11-13 08:42:21.882 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=32c0af07-61d4-4b3a-8d68-de0b17c201c0&token=[secret] (candido@192.168.1.23) 3.41ms
[I 2020-11-13 08:42:21.972 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=b520f598-bcd2-4b0b-a656-7f589d40a294&token=[secret] (candido@192.168.1.23) 3.31ms
[I 2020-11-13 08:42:22.257 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=da9ede05-65ac-4f08-9b9c-568c5c6ab7c1&token=[secret] (candido@192.168.1.23) 3.39ms
[I 2020-11-13 08:42:22.543 SingleUserLabApp log:174] 101 GET /jupyter/user/candido/api/kernels/341d2db4-036d-4d11-b480-b911761f2173/channels?session_id=5a72b226-129d-4abb-91d2-5e2a55a568e8&token=[secret] (candido@192.168.1.23) 3.34ms

As you can see for each call you will get the timings of every call, and eventually Errors and Warings.

Hi @candido.dessanti, I has install omnisci in another server, HP Workstation
RAM 50Gb
CPU 4 Core
Ubuntu 16.04
Docker 19.03
its fast speed jupyter is fast and responsive,

In my DGX Workstation
RAM 250Gb
CPU 40 Core
Ubuntu 18.04
Docker 19.03
its slow, is ubuntu 18.04 is there’s some performance issue with docker? should I downgrade to 16.04 or 20.04? I still not test in another server that have ubuntu 18.04 on it.

I will attach my jupyterlab logs in below, thank you.
jupyterlab-logs.txt (38.5 KB)
jupyterhub.log.txt (364.9 KB)

hi @dsugiarto,

from the logs looks you have modified the configuration to allow the login of super users; it should be a problem, I commented out and changed this line on the docker-compose.yml file

  OMNISCI_ALLOW_SUPERUSER_ROLE: "true"               # Authorize super users access to Jupyter without needing to be given the OmniSci role specified in OMNISCI_JUPYTER_ROLE

anyway it doesn’t change anything.

I also tried to re-create the docker containers in ubuntu 18.04 (I was on 16.04) and everythign work flawlessy and we are using the same version of docker (Docker version 19.03.6, build 369ce74a3c).

I looked at your logs comparing them with my ubuntu 18.04 and I cannot see anything special (raised log to the debug level to match yours).

Here my logs on a Ubuntu 18.04 system.

jupyter-lab.zip (11,2 KB) jupyter-hub.zip (5,8 KB)

I had to install everything except docker, maybe is better to restart the docker part from scratch, because looking at the logs I can see any macro problem on jupyter containers.

hi @candido.dessanti,

In HP workstation its also enable superuser to use jupyter and fast,

ok I will try remove and install docker again tomorrow, so its not causing by docker and ubuntu version ya?

thank you I will try to reinstall docker in my dgx.

Hi,

Yes, enabling the superuser doesn’t change anything, as enabling the debug level for the logs, but I had to enable both of them to match your configuration.

I installed on Ubuntu 16.04 and 18.04 and everything works (except for the examples that looks outdated) with the expected performance. Besides the first logon, where the system has to build the jupiterlab container, it takes just a few seconds (around 3) to open the jupyter notebook interface.

On my installations I have Jaeger that’s not working. Sent you a PM

regards