[BUG] Cannot build Dockerfile.jupyter about sparkmagic HOT 3 CLOSED

Answer generated by a 🤖

Answer

I understand that you're encountering an issue when trying to build Dockerfile.jupyter for sparkmagic. The error message indicates that the jupyter-nbextension command is not found. This is likely due to the removal of this command in version 7.0.0 of the notebook package, as you've correctly identified.

To address this issue, one potential solution is to downgrade the notebook package to version 6.5.5 in the Dockerfile. This can be done by adding a line to install this specific version before the jupyter nbextension enable --py --sys-prefix widgetsnbextension command is run.

However, please note that the requirements.txt file of sparkmagic specifies notebook>=4.2, which means it requires at least version 4.2 of notebook. Downgrading to version 6.5.5 could potentially cause conflicts if other parts of the application rely on features introduced in versions later than 6.5.5.

Here's how you can modify the Dockerfile:

# Add this line before the jupyter nbextension enable command
RUN pip install notebook==6.5.5

This should allow the Docker image to be built successfully. However, I would recommend checking if this change causes any other issues in your application due to the notebook>=4.2 requirement in requirements.txt.

If you find this solution to be correct and believe that this is a bug that could impact other users, I encourage you to make a pull request with the necessary changes to the Dockerfile.

If this solution does not align with your experience, or if you encounter any other issues, I would suggest upgrading to the latest version of sparkmagic and trying again.

Sources

• Dockerfile.jupyter
• sparkmagic/requirements.txt

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

from sparkmagic.

I may have found the issue while trying to run a sparkmagic-powered Jupyter notebook (on my machine, not in a docker container). The jump to notebook 7.0.0 has introduced major changes to Jupyter's API, including the removal of the jupyter-nbextension command. Downgrading to notebook 6.5.5 fixed the issue for me.

from sparkmagic.

@agrounds Thanks for reporting this and finding a workaround 🙌

from sparkmagic.