At the beginning of September, 2020, at least 8.5 million publicly accessible Jupyter notebooks were stored on Github, and 12,500 new notebooks were being committed to public Github repositories each day[1]. These notebooks represent an enormous potential educational resource. There are, however, significant obstacles that prevent the collaborative use of notebooks for education. Specifically: 1) The notebooks are stored as JSON files, which combine human readable content written in Commonmark markdown with metadata about the notebook (language kernel, attributes of notebook cells, etc.) and the notebook cell output. This prevents authors and collaborators from using the most effective work-flows for software development, such as change tracking and merging, to extend and adapt notebooks. 2) The notebooks are computer programs that need to run in a specific environment. Since there is no easy way to "pin" the specification of that environment for a particular notebook, it is difficult to set up automated testing to guarantee that the notebooks work across Linux, MacOS and Windows computers. These barriers currently make it difficult to foster the collaborative, iterative workflow that would enable an educational community centered around open-source software to thrive.

At the University of British Columbia, the Department of Earth, Ocean and Atmospheric Sciences (EOAS) has begun a 3-year project to transform 15 undergraduate and graduate courses to use Jupyter notebooks as a core component of course instruction. Support for the OCESE project (Open-source computing for earth science education) covers staff and student salaries and teaching buy-outs for course development, faculty training and course evaluation[2]. This project is part of a wider effort within the Faculty of Science to use Jupyter notebooks in a broad range of courses, including the Masters in Data Science program and new computer science and statistics courses that are part of the developing Data Science minor[3]. Together, these initiatives will produce and make use of hundreds of notebooks and will require collaborative development to guarantee that modules, developed for example for the first year data science course, can be repurposed for EOAS courses that build on that preparation.

As part of OCESE we have adopted tools from several rapidly growing open-source projects to address the obstacles to collaboration and environment specification. We are writing our notebooks using a new, extensible markdown version (MyST Markdown[4]) that provides a complete specification of notebook content and metadata in a form that can be edited either with Jupyter or standard code editors, and can be version-controlled and merged using git. We use Jupytext[5] to pair the MyST notebooks with python script equivalents for code formatting, editing, and testing, and for moving code between notebooks and python modules. Course notebooks are assembled into a single, cross-referenced and indexed web-site using Jupyterbook[6], which provides Latex-like equation and figure referencing, figure captioning, and math and bibliography environments for notebook collections. To completely specify the Jupyterbook execution environment we have adapted the Docker container stacks from the Pangeo project[7], which uses conda-lock[8] to exactly pin the python environment required to run a notebook collection, and uses Github to build, test and push the Docker containers to Dockerhub.

In this talk we will show how all of this technology comes together using three examples: 1) an undergraduate upper-year course in remote sensing hosted on a Kubernetes cluster in the commercial cloud 2) A graduate workshop on distributed computing hosted on departmental compute nodes and 3) A 4th year student capstone project transformed into a Jupyterbook as part of a student portfolio on Github. We will also outline how we are training faculty and graduate students to use these tools for teaching and research, and how we will evaluate the transformed courses to determine the effectiveness of Jupyter notebooks for student learning.

[1] https://nbviewer.jupyter.org/github/parente/nbestimate/blob/master/estimate.ipynb
[2] https://eoas-ubc.github.io
[3] https://tlef.ubc.ca/funded-proposals/entry/713/
[4] https://myst-parser.readthedocs.io/en/latest/
[5] https://jupytext.readthedocs.io/en/latest/?badge=latest
[6] https://jupyterbook.org/intro.html
[7] https://github.com/pangeo-data/pangeo-docker-images
[8] https://github.com/conda-incubator/conda-lock