How to automate running of Jupyter Notebook cells periodically

Question

I want to integrate my jupyter notebook with my website, where I have written the code to fetch real-time data from MySQL server and do real-time visualisation using plotly. But every time I'm having to run all the cells of my Kernel. Is there a way I can automate the running of the Jupyter notebook cells periodically say everyday 1 hour?

So you are **NOT** using sql server? Then why tag it? – SMor Aug 05 '18 at 12:21 — SMor, Aug 05 '18 at 12:21
Hi, , I, am using SQL server of my website. – Debadri Dutta Aug 05 '18 at 17:39 — Debadri Dutta, Aug 05 '18 at 17:39

score 2 · Accepted Answer · answered Aug 05 '18 at 10:22

2

My suggestion would be to

Setup a cron job with the periodicity you want.
Use runipy to run all the cells in the notebook. It has a lot of functionality like saving the run as html report. This will particularly be useful in your case as you want to visualise plotly plots.

I can provide the commands here, but they are pretty straight forward and can easily be followed from the links.

answered Aug 05 '18 at 10:22

Deepak Saini

2,810
1
19
26

So I can directly link it to my website? And like it will run throughout the day on its own,whatever I've coded in it? – Debadri Dutta Aug 05 '18 at 17:40
The runipy site notes that it "started before Jupyter's [execute API](https://nbconvert.readthedocs.io/en/latest/execute_api.html), which is now the recommended way to run notebooks from the command-line." – fact_finder Oct 12 '22 at 12:33
As @fact_finder notes, runipy is now deprecated, and points uses to the execute API of nbconvert. – jeffmcc Jan 10 '23 at 19:03

score 0 · Answer 2 · edited Apr 22 '22 at 20:55

0

import time
 
# Wait for 3600 seconds
while True:
    time.sleep(3600)
    #YOUR CODE HERE

If you want not to wait for the cell in running mode you can run the code above inside a thread

edited Apr 22 '22 at 20:55

Hami

704
2
9
27

answered Aug 05 '18 at 10:11

Afshin Amiri

3,438
1
20
21

score 0 · Answer 3 · answered Aug 06 '18 at 03:39

Please setup a function that runs for an interval of 1 hour using the Javascript setInterval function. Inside the interval function you can call the jupyter object method, to run all cells.

%%html
<script>
    // AUTORUN ALL CELLS ON NOTEBOOK-LOAD!
    require(
        ['base/js/namespace', 'jquery'], 
        function(jupyter, $) {
            $(jupyter.events).on("kernel_ready.Kernel", function () {
                setInterval(function(){
                  console.log("Auto-running all cells-below...");
                  jupyter.actions.call('jupyter-notebook:run-all-cells-below');
                  // jupyter.actions.call('jupyter-notebook:save-notebook');
                }, 60000); // 60000 for 1 hour interval gap
            });
        }
    );
</script>

Note: I have added the setInterval part of the script by myself, the major code,comes from this SO answer

`// 60000 for 1 hour interval gap` is not correct, should be `3600000` (60 * 60 * 1000 = hour in milliseconds) — Evgeniia Vakarina, Mar 16 '21 at 12:19

How to automate running of Jupyter Notebook cells periodically

3 Answers3

Linked