Dependencies - Request for environment .yml file

[DennisDannecker]

Hello,

I am trying to run the denoiser on a couple of electron microscopy micrographs of mine.

I am running into one dependency error after the other.

I would like to ask you to provide an yml file in order to create a conda environment in which I can run this denoiser script.

So far I am not even able to just use the inference let alone the training.

While it is helpful that you listed the relevant packages, a version control is very important in order to have the script to run.

Thank you very much for your contribution and your help.

Best

Dennis

[DennisDannecker]

I am getting the following warnings and errors:

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
• https://github.com/tensorflow/io (for I/O related ops)
  If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:25: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:25: The name tf.logging.DEBUG is deprecated. Please use tf.compat.v1.logging.DEBUG instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:587: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:591: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:594: RunConfig.init (from tensorflow.contrib.learn.python.learn.estimators.run_config) is deprecated and will be removed in a future version.
Instructions for updating:
When switching to tf.estimator.Estimator, use tf.estimator.RunConfig instead.
WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:596: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Please use tf.global_variables instead.
WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:597: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/miniconda3/envs/jeffrey/lib/python3.6/site-packages/tensorflow_core/python/util/tf_should_use.py:198: initialize_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use tf.variables_initializer instead.
WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:601: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:502: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/miniconda3/envs/jeffrey/lib/python3.6/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:2784: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:188: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:543: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:550: The name tf.losses.mean_squared_error is deprecated. Please use tf.compat.v1.losses.mean_squared_error instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/miniconda3/envs/jeffrey/lib/python3.6/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:518: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:518: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

WARNING:tensorflow:From /Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py:614: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

---

ValueError Traceback (most recent call last)
in
20
21 #Initialize the denoising neural network
---> 22 noise_remover = Denoiser(checkpoint_loc=checkpoint_loc, visible_cuda="0")
23
24 #Denoise a 512x512 crop from the image

/Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py in init(self, checkpoint_loc, visible_cuda)
613
614 saver = tf.train.Saver()
--> 615 saver.restore(sess, tf.train.latest_checkpoint(checkpoint_loc))
616
617 self.sess = sess

/Data/erc-3/ddannecker/miniconda3/envs/jeffrey/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py in restore(self, sess, save_path)
1275 return
1276 if save_path is None:
-> 1277 raise ValueError("Can't load save_path when it is None.")
1278
1279 checkpoint_prefix = compat.as_text(save_path)

ValueError: Can't load save_path when it is None.

[Jeffrey-Ede]

I used TensorFlow 1.x.

I developed this code as part of my PhD in Physics, which I finished in 2021. Thus, I no longer have my original python environment to create a yml config file.

[DennisDannecker]

Thank you for your quick response.

I see. Interesting, because I installed TensorFlow 1.15 once I realized that the contrib function was deprecated and not available in 2.x.

Mmmh...the error points to TensorFlow, but maybe another dependency is of the wrong version also.
Let's see, I play around with the environment, maybe I can fix it.

Thank you!

[DennisDannecker]

Another question:

As for the location to the latest checkpoint.
Which file exactly is one supposed to pass as the location of the latest checkpoint.
The Google-Drive mount lists several files.
So far I have downloaded only the checkpoint files (general or low dose) and used their path (absolute path or relative).
Is that the file that is required by tensorflow?

The checkpoint file is most likely not the right required file, it only carries the path information to the checkpoints.
So which one of the files listed in the google drive is to be provided to the denoiser script in the checkpoint location variable?

My first goal would be to simply run the example script with the randomly generated 1500x1500 image (np.array).
However, I always run into this error:

ValueError Traceback (most recent call last)
in
10
11 #Initialize the denoising neural network
---> 12 noise_remover = Denoiser(checkpoint_loc=checkpoint_loc, visible_cuda="0")
13
14 #Denoise a 512x512 crop from the image

/Data/erc-3/ddannecker/denoising_with_jeffrey/Electron-Micrograph-Denoiser/denoiser.py in init(self, checkpoint_loc, visible_cuda)
613
614 saver = tf.train.Saver()
--> 615 saver.restore(sess, tf.train.latest_checkpoint(checkpoint_loc))
616
617 self.sess = sess

/Data/erc-3/ddannecker/miniconda3/envs/jeffrey/lib/python3.6/site-packages/tensorflow/python/training/saver.py in restore(self, sess, save_path)
1743 return
1744 if save_path is None:
-> 1745 raise ValueError("Can't load save_path when it is None.")
1746 logging.info("Restoring parameters from %s", compat.as_text(save_path))
1747 try:

ValueError: Can't load save_path when it is None.

Indicating that the latest checkpoints cannot be found.
I am afraind I am not able to properly provided the trained model to the denoiser.
Before I continue the training and thereby maybe be able to provide the actual checkpoints, I would like to ask you to maybe provide a bit more detail in terms of how does one provide the pre-trained model to the denoiser. Continuing or Re-training the model would require me downloading your dataset, which I would like to avoid, at least for now.