1

I have seen answers to this question and the answer is always that these files are located in

/home/ec2-user/Sagemaker/

however when I'm in Terminal in Sagemaker Studio I don't see this. It seems that Sagemaker Studio has a different directory than whichever Sagemaker service other askers are using. This is what my command line looks like when I do pwd in the directory:

sagemaker-user@studio$ pwd
/home/sagemaker-user/sagepipeline_nonomethod/source_dir
sagemaker-user@studio$

When I do

os.chdir('/home/sagemaker-user/sagepipeline_nonomethod/source_dir')

inside my jupyter notebook however I get an error that says there is no such directory. So there must be some prefix to the directory that I am missing...

Leigh
  • 518
  • 5
  • 12

2 Answers2

2

In short, you should be able to access the directory at /root/sagepipeline_nonomethod/source_dir within your notebook. Explaination follows:

If you open a terminal in the Launcher under Utilities and files, you will arrive at a system terminal. It's the compute environment living in SageMaker Studio, instead of the container where your notebook (kernel) lives.

The /home/sagemaker-user directory is a mountpoint of an EFS, where your Jupyter file browser points to.

sagemaker-user@studio$ df -h
Filesystem         Size  Used Avail Use% Mounted on
overlay             32G  5.3M   32G   1% /
tmpfs               64M     0   64M   0% /dev
tmpfs              1.9G     0  1.9G   0% /sys/fs/cgroup
shm                395M     0  395M   0% /dev/shm
/dev/nvme0n1p1     160G   26G  135G  16% /opt/.sagemakerinternal
127.0.0.1:/200005  8.0E   40M  8.0E   1% /home/sagemaker-user
devtmpfs           1.9G     0  1.9G   0% /dev/tty
tmpfs              1.9G     0  1.9G   0% /proc/acpi
tmpfs              1.9G     0  1.9G   0% /sys/firmware

To access the terminal of the container environment, go to Launcher > Notebooks and compute resources > Open image terminal

The same EFS is mounted on /root

root@datascience-1-0-ml-t3-medium-1abf3407f667f989be9d86559395:~# df -h
Filesystem         Size  Used Avail Use% Mounted on
overlay             32G   40K   32G   1% /
tmpfs               64M     0   64M   0% /dev
tmpfs              1.9G     0  1.9G   0% /sys/fs/cgroup
shm                395M     0  395M   0% /dev/shm
127.0.0.1:/200005  8.0E   40M  8.0E   1% /root
/dev/nvme0n1p1     160G   24G  137G  15% /opt/.sagemakerinternal
devtmpfs           1.9G     0  1.9G   0% /dev/tty
tmpfs              1.9G     0  1.9G   0% /proc/acpi
tmpfs              1.9G     0  1.9G   0% /sys/firmware
lsc
  • 235
  • 1
  • 9
1

This occurs because your files and notebooks in SageMaker Studio exist separately to the machine running the kernel to execute your code.

SageMaker Studio is, in essence, just a theme for Jupyter with a few added features. In the below diagram, you can see that your Studio interface and files are hosted by the "JupyterServer App", while the instances that execute your code are separate machines running containers.

The Amazon SageMaker Studio Notebooks documentation provides a helpful overview of this architecture. See the diagram below.

Your question appears to be specific to an issue you're having, however. If you create a cell in your notebook and execute !pwd and !ls, you should find that the files available match those in the SageMaker Studio sidebar.

fucalost
  • 313
  • 1
  • 8