Reaching Memory Limit

Helpdesk Science Console JupyterLab
1

I recently ran into the issue of reaching the memory limit for the server I was using (Small). This caused the server to “crash” and I had to restart it. There was no warning or indication that I had reached the memory limit (and I wasn’t expecting to reach it, since I wasn’t doing anything I thought would need > 8 GB of memory).

I don’t know if this is feasible, but would it be possible to have something to show that the memory limit has been reached? I’m testing code and it would be helpful since I can cause my code to fail in 100 different ways. It would be helpful to know why something failed.

1 Like
2

If the server crashes, it is 99% likely caused by memory overload. There are several ways to monitor memory use.

  • use top or htop from the terminal.
  • The memory usage by a notebook is shown at the bottom status bar of the form: Mem: 256.54 / 6912.00 MB.
  • This docs link suggest other tools you can install to do detailed profiling of python jobs.
3

@azoghbi you seriously underestimate my ability to find the 1% of the cases that wouldn’t involve memory overload. :wink: