Jupyter notebook running slow in Microsoft Azure

Question

I am quite new to using Microsoft Azure for running jupyter notebooks. I noticed that it can take 30-45 seconds to polar plot 2 numpy arrays, which is small relatively small (<300 datapoints per array). When I have to execute several of these plots, the time adds up, so I am wondering if this is related to a particular compute instance or network latency? Any insight would be greatly appreciated, thank you!

Answer 1

Notebook will be slow when the data loading limits are high, below is one of the case where I faced similar issue.

• Tried to display some 40000 columns, I faced some serious unresponsive issue and slowness.
• As soon as I changed the code to display only 40 or 80 columns, the response was good.

Below are some root causes for this:

• Clean all the data which is related to dataframes like pandas etc.

• From the below block we can get memory and cpu usage, so that it will help us to clear the unwanted data:

  #!/usr/bin/env python
  import psutil
  # gives a single float value
  psutil.cpu_percent()
  # gives an object with many fields
  psutil.virtual_memory()
  # you can convert that object to a dictionary 
  dict(psutil.virtual_memory()._asdict())
  # you can have the percentage of used RAM
  psutil.virtual_memory().percent
  79.2
  # you can calculate percentage of available memory
  psutil.virtual_memory().available * 100 / psutil.virtual_memory().total
  20.8
  

• We will have some variable inspectors, if they are enabled the notebook might get slow because of some dataframes like pandas. GIT Issue

  If you want to disable it --> Edit --> nbextensions config.

Refer to these SO (SO1, SO2, SO3, SO4) links for detailed explanations.