Cannot initialize Instant Client in jupyter notebook (in AWS Sagemaker), works fine in script on the same instance

[rahul-ve]

Hi all,

We followed the docs and got python-oracledb working fine in thick mode.
But when we try to do the same in Jupyter notebook on AWS Sagemaker, it fails when trying to initialize the Instant client.

• In CLI on same instance

# ./test_oracle_db.py 
ODPI [00592] 2023-09-07 06:27:55.414: ODPI-C 5.0.0
ODPI [00592] 2023-09-07 06:27:55.414: debugging messages initialized at level 64
ODPI [00592] 2023-09-07 06:27:55.414: Context Parameters:
ODPI [00592] 2023-09-07 06:27:55.414: Environment Variables:
ODPI [00592] 2023-09-07 06:27:55.414:     LD_LIBRARY_PATH => "/root/oracle_files/odb_client/instantclient_21_11"
ODPI [00592] 2023-09-07 06:27:55.414: check module directory
ODPI [00592] 2023-09-07 06:27:55.415: module name is /opt/conda/lib/python3.10/site-packages/oracledb/thick_impl.cpython-310-x86_64-linux-gnu.so
ODPI [00592] 2023-09-07 06:27:55.415: load in dir /opt/conda/lib/python3.10/site-packages/oracledb
ODPI [00592] 2023-09-07 06:27:55.415: load with name /opt/conda/lib/python3.10/site-packages/oracledb/libclntsh.so
ODPI [00592] 2023-09-07 06:27:55.415: load by OS failure: /opt/conda/lib/python3.10/site-packages/oracledb/libclntsh.so: cannot open shared object file: No such file or directory
ODPI [00592] 2023-09-07 06:27:55.415: load with OS search heuristics
ODPI [00592] 2023-09-07 06:27:55.415: load with name libclntsh.so
ODPI [00592] 2023-09-07 06:27:55.424: load by OS successful
ODPI [00592] 2023-09-07 06:27:55.424: validating loaded library

• in Jupyter Notebook
  

ODPI [00376] 2023-09-07 06:18:08.781: ODPI-C 5.0.0
ODPI [00376] 2023-09-07 06:18:08.781: debugging messages initialized at level 64
ODPI [00376] 2023-09-07 06:18:08.781: Context Parameters:
ODPI [00376] 2023-09-07 06:18:08.781:     Oracle Client Lib Dir: /root/oracle_files/odb_client/instantclient_21_11
ODPI [00376] 2023-09-07 06:18:08.781: Environment Variables:
ODPI [00376] 2023-09-07 06:18:08.781:     LD_LIBRARY_PATH => "/root/oracle_files/odb_client/instantclient_21_11"
ODPI [00376] 2023-09-07 06:18:08.781: load in parameter directory
ODPI [00376] 2023-09-07 06:18:08.781: load in dir /root/oracle_files/odb_client/instantclient_21_11
ODPI [00376] 2023-09-07 06:18:08.781: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so
ODPI [00376] 2023-09-07 06:18:08.787: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.787: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.19.1
ODPI [00376] 2023-09-07 06:18:08.794: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.794: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.18.1
ODPI [00376] 2023-09-07 06:18:08.800: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.800: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.12.1
ODPI [00376] 2023-09-07 06:18:08.806: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.806: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.11.1
ODPI [00376] 2023-09-07 06:18:08.812: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.812: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.20.1
ODPI [00376] 2023-09-07 06:18:08.818: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory
ODPI [00376] 2023-09-07 06:18:08.818: load with name /root/oracle_files/odb_client/instantclient_21_11/libclntsh.so.21.1
ODPI [00376] 2023-09-07 06:18:08.819: load by OS failure: libnnz21.so: cannot open shared object file: No such file or directory

---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
Cell In[28], line 1
----> 1 oracledb.init_oracle_client(lib_dir="/root/oracle_files/odb_client/instantclient_21_11")

File src/oracledb/impl/thick/utils.pyx:476, in oracledb.thick_impl.init_oracle_client()

File src/oracledb/impl/thick/utils.pyx:500, in oracledb.thick_impl.init_oracle_client()

File src/oracledb/impl/thick/utils.pyx:421, in oracledb.thick_impl._raise_from_info()

DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory". See https://python-oracledb.readthedocs.io/en/latest/user_guide/initialization.html for help

Anyone come across the same issue.

Kind regards
Rahul

[cjbj]

On Linux don't pass lib_dir to init_oracle_client(). (Technically you can, but it doesn't do what you want, and it can lead to clashes). You must make sure LD_LIBRARY_PATH or ldconfig are run and respected by the OS, before starting Python. If you are using LD_LIBRARY_PATH, then make sure Jupyter is being invoked in the terminal that sets it. I can see that it is not really set correctly because of the libnnz failure, which is typical. Although the log shows LD_LIBRARY_PATH is set, something in your configuration is ignoring it.

[cjbj]

The fact that it's now failing on libclntsh.so confirms the hypothesis that the environment is restricting Jupyter's access for dynamically loading external libraries. I don't have any pearls of wisdom about Sagemaker.

What Thick mode feature requirements prevent you using Thin mode?

[rahul-ve]

Thanks for your help @cjbj, we are narrowing this down. Will reach out to Sagemaker team as well.
Database version: 11.2.0.4.0. We tried Thin mode first, says it is not supported.

[cjbj]

Thanks for the info and good luck. DB 11.2 is never going to be supported in Thin mode, sorry.

[rahul-ve]

Hope this helps the next person.
We needed the below to allow for dynamic linking of modules.
Change /root/oracle_files/odb_client/instantclient_21_11 to the location where the files were unzipped.

%%sh

sh -c "echo /root/oracle_files/odb_client/instantclient_21_11 > /etc/ld.so.conf.d/oracle-instantclient.conf"
ldconfig