"Out of box" omnisci-datascience seems broken, Altair.Chart() fails in included examples

Examples code to generate charts using ibis in the Jupyter notebooks distributed with omnisci-datascience all fail with “Javascript Error: Unexpected token < in JSON at position 0”,

e.g. from the very first notebook seen by users “Altair_Ibis_Vega_for_Interactive_Exploration”

flights_by_state = alt.Chart(
t,
title=“Total Number of Flights by State”
).mark_bar().encode(
…)
flights_by_state

fails with

Javascript Error: Unexpected token < in JSON at position 0

Uncommenting the preceding debug line does not give anything useful, at least to me, but here is the screen shot…

Unclear what I’ve done wrong. I’d hope the demo code/notebooks “just work” in a stock install of both products.

omnisci-datascience installed for individual user install on macOS Big Sur 11.2.3 (Intel) (latest version of omnisci-datascience as of today, but how is anybody supposed to know/report the version of omnisci-datascience installed? How about at least dropping a version file into the omnisci-datascience directory?)
Omnisci - Version 5.5.1 (5.5.1)

Hi @darryl ,

I tried the same example on Linux and I cannot reproduce your issue.

do the example starts with a connection to metis?

import altair as alt
import ibis
import ibis_vega_transform

conn = ibis.omniscidb.connect(
    host='metis.mapd.com', user='mapd', password='HyperInteractive',
    port=443, database='mapd', protocol= 'https'
)
t = conn.table("flights_donotmodify")

Have you tried to reference the load database? (you should change the name of the table)

Regards,
Candido

Hi Darryl - I lead the product team and my sincere apologies for the frustration here.

It seems we’ve had an unfortunate regression in the pkg version of the Mac installer - thanks for bringing it to our attention. At first glance it looks like there is something that broke in the renderer that allows the combination of Ibis + Vega + Altair charts to show in a jupyterlab notebook. We’re digging deeper into why that happened but it is likely due to jupyterlab moving to 3.x and a regression testing miss on our part.

Also, we distribute signed versions of the data science installer pkg, so we’ll take a look at what broke there as well

I’m happy to jump on a zoom with you to provide a workaround till we fix this. One admittedly clunky immediate workaround is to use a terminal window to start up jupyter lab outside of Immerse in a browser (i.e not clicking the icon). You can then hit the running jupyter server like so

  • Open up terminal (doh). You should see a (base) prefix to your prompt, indicating that the installer has updated the conda environment. If this hasn’t happened, try running source ~/.zshrc
  • navigate to the ~/omnisci-examples directory that is the standard installer path - cd ~/omnisci-examples
  • type jupyter lab at the prompt and either click on the url that the server shows in the terminal log, or alternatively hit localhost:8888. This should open up jupyterlab outside of the Immerse/Electron context that seems to be having issues. Try the notebooks again - they should not have the javascript error message you highlighted
1 Like

Why are you trying to reproduce this problem on Linux not Mac?

Basic access to the database works fine. I am literally running the OmniSci notebook page I mentioned, everything works in that page works OK until the failure I quoted.

Do you folks release QA test against these notebooks?

1 Like

Thanks Venkat I’ll give that a try later today.

(crossing threads) The omnisci-datascience package is signed by OmniSci as a developer but is not notarized by Apple. A requirement for install approval on macOS Catalina, Big Sur, and later. I was trying to use the word “notarized” carefully. Examining your installer with Suspicious Package (https://mothersruin.com/software/SuspiciousPackage) clearly confirms this problem.

The whole current out of box experience on macOS is pretty well, I have a perfect word for it.

I hear you and the same word crossed my mind as well - thanks for taking the trouble to write this up, we’ll do better.

Because I hadn’t access to a Mac this morning, and I thought it was better than nothing and took the chance to take a look at the latest release on AWS.

In the past got that error occasionally (something intermittent on a particular version of ubuntu) but it was on a very early release, so I wanted to be sure that it wasn’t a regression on the core.

I hope the workaround that Venkat supplied works for you, and you will get a slick experience with the data science tool from now on.

Regards,
Candido

Thanks Venkat this seems to have gotten me going, a comments inline that might help others.

I’m not sure what you mean by “standard installer path”. (The PATH variable is not being set by .zshrc to include any omnisci-examples directory if that’s what you mean, but I suspect it’s not).

There are
/Users//Documents/omnisci-examples
/Users//omnisci-datascience/pkgs/omnisci-examples-0.3.0-5/opt/omnisci-examples
/Users//omnisci-datascience/opt/omnisci-examples

And since I think all you want here is to pick up the sincluded notebooks… cd’ing to /Users//Documents/omnisci-examples worked for me. It ran Jupyter lab in that directory and in that resulting Jupyter browser window I could connect to OmniSci and run the examples using ibis_vega_transform() that failed before.


I don’t think it’s a good idea, and is fairly anti-social, to go sticking garbage in user’s command prompts. (Yes I know this is Conda, their bad,… personally I would disable changeps1).

Ah - yes, that (the conda init stuff in your ~/.zshrc or ~/.bash_profile) a bit of a loop we have to jump through at this point. The issue is that in order to use our data science tools seamlessly you ultimately need to have a functioning conda environment with bits installed, and conda also needs to activate that environment with the bits (hence the init stuff).

This might all be irrelevant :slight_smile: since you’re having trouble, but What happens on the OmniSci side is that we detect the presence of the installed bits under ~/omnisci-datascience and Immerse shows the icon for Jupyter - which underneath the hood essentially launches jupyter lab just like you would at a command prompt. The dependency here is that it in turn needs the bits installed, which in turn needs conda to have successfully 'init’d the base environment with the tools (including jupyter lab itself).

It’s not really something we usually want to own (since python data science environments are well beyond our remit), but we did try to make it ‘convenient’ by packaging the collection of dependencies into a single pkg file (the one that unfortunately seems to have the issues you pointed out).

@darryl - we’ve narrowed down the cause finally - it’s a weird problem where the path setting is getting mangled when you start up jupyter inside Immerse.

Here’s a quick, still somewhat clunky workaround while we roll out the fix properly

  1. Startup the Immerse application from a terminal window - basically go to an empty terminal and type /Applications/OmniSci.App/Resources/MacOs/OmniSci
  2. Close the main OmniSci window (not minimize), and then click on OmniSci in the applications tray. You’ll see the Jupyter icon show up. Now clicking that icon and opening up Jupyter should get everything to work.

I know it feels like standing up and turning around three times and jumping once in the air, but it’s due to a subtle bug we’ve just tracked down and will fix. Thanks for your patience

@Venkat_Krishnamurthy Thanks! I’ll play more, but the other work around had already helped me get going and do a quick test for something I needed.

This integration is neat, I think it can be a helpful out of box experience. All emphasizing the value of on OmniSci on Mac, and I’d love to see the new updates of that. Oh Apple, c’mon please give us NVIDIA GPU support :slight_smile: