kuda.ai | code. guitar. life.

I work with notebooks a lot, especially when it comes to prototyping.

The term "notebook" represents either a jupyter notebook or a jupyter lab notebook.

Whenever I discover interesting things in notebooks, I will publish them here.

Run shell commands from cells

You can run shell commands (ls, pwd, env, ssh, etc.) from your notebook cell. First type a bang, then type the shell command.

!env | grep CONDA

Assign output of shell commands to a variable

out = !ls /Users/davidkuda/.ipython/profile_default/startup

for i in out:
    print(i)

Load things from a python src file

If you happen to always do the same things at the beginning of each notebook or a session, you could simplify it by defining all these things in a python src file, and then run that file.

Let's say that by default, you always have the same imports:

import json

import numpy as np
import pandas as pd
import seaborn as sns

Save the same lines in namespace.py, and make them available in your notebook session:

%run namespace.py

Make sure that you get the path to the folder right, e.g.

%run /Users/user/code/project/notebooks/namespace.pu

Load things on startup

Create any .py file in:

cd $HOME/.ipython/profile_default/startup

As long as all the scripts can be executed without raising an exception, your namespace will be updated with whatever you define there.

Example:

echo 'NAME = "DAVID"' >> $HOME/.ipython/profile_default/startup/consts.py

Now, from any ipython session (i.e. in every notebook), the variable NAME is available to you.

Use jq to format json

It's common to work with JSON. Sometimes even with complex json.

jq is a fantastic tool to add syntax highlight and nice formatting to your json strings.

You can save a json string as a variable, for example the return value of an API. You can then pass this value to jq directly from your notebook.

s = '{"hello":"world"}'
!echo '{s}' | jq

You could even wrap this into a python function and invoke the python function instead. That way, you save a few characters to type.

def jqit(t: str, filter: str = "."):
    !echo '{t}' | jq '{filter}'

jqit('{"hello":"world"}')

Result:

{
  "hello": "world"
}

For such a small json string, this is of course over-engineered.

But I often work with APIs that return huge json strings, especially when I work with APIs from NoSQL applications.

import requests

def jqit(res: requests.models.Response, filter: str = "."):
    t = res.text
    !echo '{t}' | jq '{filter}'