Errors to generate a result

Community Cloud
1

Hi, I am deploying Streamlit on the cloud using a GAN model. However, I do not know why I cannot get any results as the process seems to be paused and I cannot find the reason.

The github link is: GitHub - tongtz/photo2avatar
App link is: https://share.streamlit.io/tongtz/photo2avatar/main/app.py

The log is as following:

e[35mWARNINGe[0m:tensorflow:From main.py:91: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

e[35mWARNINGe[0m:tensorflow:From main.py:91: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

2022-04-12 22:08:15.413952: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-04-12 22:08:15.423631: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200210000 Hz
2022-04-12 22:08:15.426197: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5647d8422360 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-04-12 22:08:15.426243: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-04-12 22:08:15.428388: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/appuser/venv/lib/python3.7/site-packages/cv2/../../lib64:/home/appuser/venv/lib/python3.7/site-packages/cv2/../../lib64:
2022-04-12 22:08:15.428426: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN e[35mERRORe[0m (303)
2022-04-12 22:08:15.428453: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (localhost): /proc/driver/nvidia/version does not exist
e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/UGATIT.py:481: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/UGATIT.py:105: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/ops.py:49: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.keras.layers.Conv2D` instead.
e[35mWARNINGe[0m:tensorflow:From /home/appuser/venv/lib/python3.7/site-packages/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/ops.py:105: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/ops.py:61: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/ops.py:100: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/ops.py:145: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/utils.py:71: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.


##### Information #####
# light :  False
# gan type :  lsgan
# dataset :  sample
# max dataset number :  0
# batch_size :  1
# epoch :  100
# iteration per epoch :  10000
# smoothing :  True

##### Generator #####
# residual blocks :  4

##### Discriminator #####
# discriminator layer :  6
# the number of critic :  1
# spectral normalization :  True

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
---------
Variables: name (type shape) [size]
---------
generator_B/conv/conv2d/kernel:0 (float32_ref 7x7x3x64) [9408, bytes: 37632]
generator_B/conv/conv2d/bias:0 (float32_ref 64) [64, bytes: 256]
generator_B/ins_norm/beta:0 (float32_ref 64) [64, bytes: 256]
generator_B/ins_norm/gamma:0 (float32_ref 64) [64, bytes: 256]
generator_B/conv_0/conv2d/kernel:0 (float32_ref 3x3x64x128) [73728, bytes: 294912]
generator_B/conv_0/conv2d/bias:0 (float32_ref 128) [128, bytes: 512]
generator_B/ins_norm_0/beta:0 (float32_ref 128) [128, bytes: 512]
generator_B/ins_norm_0/gamma:0 (float32_ref 128) [128, bytes: 512]
generator_B/conv_1/conv2d/kernel:0 (float32_ref 3x3x128x256) [294912, bytes: 1179648]
generator_B/conv_1/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/ins_norm_1/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/ins_norm_1/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_0/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_0/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_0/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_1/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_1/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_1/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_2/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_2/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_2/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_3/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/resblock_3/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_B/resblock_3/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_B/CAM_logit/kernel:0 (float32_ref 256x1) [256, bytes: 1024]
generator_B/CAM_logit/bias:0 (float32_ref 1) [1, bytes: 4]
generator_B/conv_1x1/conv2d/kernel:0 (float32_ref 1x1x512x256) [131072, bytes: 524288]
generator_B/conv_1x1/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/MLP/linear_0/dense/kernel:0 (float32_ref 1048576x256) [268435456, bytes: 1073741824]
generator_B/MLP/linear_0/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/MLP/linear_1/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_B/MLP/linear_1/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/MLP/gamma/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_B/MLP/gamma/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/MLP/beta/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_B/MLP/beta/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock0/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock0/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock0/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock0/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock0/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock0/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock1/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock1/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock1/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock1/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock1/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock1/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock2/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock2/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock2/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock2/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock2/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock2/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock3/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock3/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock3/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock3/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_B/adaptive_resblock3/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_B/adaptive_resblock3/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_B/up_conv_0/conv2d/kernel:0 (float32_ref 3x3x256x128) [294912, bytes: 1179648]
generator_B/up_conv_0/conv2d/bias:0 (float32_ref 128) [128, bytes: 512]
generator_B/layer_ins_norm_0/rho:0 (float32_ref 128) [128, bytes: 512]
generator_B/layer_ins_norm_0/gamma:0 (float32_ref 128) [128, bytes: 512]
generator_B/layer_ins_norm_0/beta:0 (float32_ref 128) [128, bytes: 512]
generator_B/up_conv_1/conv2d/kernel:0 (float32_ref 3x3x128x64) [73728, bytes: 294912]
generator_B/up_conv_1/conv2d/bias:0 (float32_ref 64) [64, bytes: 256]
generator_B/layer_ins_norm_1/rho:0 (float32_ref 64) [64, bytes: 256][2022-04-12 22:08:17.501031] 
generator_B/layer_ins_norm_1/gamma:0 (float32_ref 64) [64, bytes: 256]
generator_B/layer_ins_norm_1/beta:0 (float32_ref 64) [64, bytes: 256]
generator_B/G_logit/conv2d/kernel:0 (float32_ref 7x7x64x3) [9408, bytes: 37632]
generator_B/G_logit/conv2d/bias:0 (float32_ref 3) [3, bytes: 12]
generator_A/conv/conv2d/kernel:0 (float32_ref 7x7x3x64) [9408, bytes: 37632]
generator_A/conv/conv2d/bias:0 (float32_ref 64) [64, bytes: 256]
generator_A/ins_norm/beta:0 (float32_ref 64) [64, bytes: 256]
generator_A/ins_norm/gamma:0 (float32_ref 64) [64, bytes: 256]
generator_A/conv_0/conv2d/kernel:0 (float32_ref 3x3x64x128) [73728, bytes: 294912]
generator_A/conv_0/conv2d/bias:0 (float32_ref 128) [128, bytes: 512]
generator_A/ins_norm_0/beta:0 (float32_ref 128) [128, bytes: 512]
generator_A/ins_norm_0/gamma:0 (float32_ref 128) [128, bytes: 512]
generator_A/conv_1/conv2d/kernel:0 (float32_ref 3x3x128x256) [294912, bytes: 1179648]
generator_A/conv_1/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/ins_norm_1/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/ins_norm_1/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_0/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_0/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_0/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_1/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_1/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_1/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_2/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_2/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_2/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_3/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res1/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res1/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/resblock_3/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res2/instance_norm/beta:0 (float32_ref 256) [256, bytes: 1024]
generator_A/resblock_3/res2/instance_norm/gamma:0 (float32_ref 256) [256, bytes: 1024]
generator_A/CAM_logit/kernel:0 (float32_ref 256x1) [256, bytes: 1024]
generator_A/CAM_logit/bias:0 (float32_ref 1) [1, bytes: 4]
generator_A/conv_1x1/conv2d/kernel:0 (float32_ref 1x1x512x256) [131072, bytes: 524288]
generator_A/conv_1x1/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/MLP/linear_0/dense/kernel:0 (float32_ref 1048576x256) [268435456, bytes: 1073741824]
generator_A/MLP/linear_0/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/MLP/linear_1/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_A/MLP/linear_1/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/MLP/gamma/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_A/MLP/gamma/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/MLP/beta/dense/kernel:0 (float32_ref 256x256) [65536, bytes: 262144]
generator_A/MLP/beta/dense/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock0/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock0/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock0/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock0/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock0/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock0/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock1/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock1/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock1/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock1/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock1/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock1/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock2/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock2/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock2/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock2/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock2/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock2/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock3/res1/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock3/res1/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock3/res1/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock3/res2/conv_0/conv2d/kernel:0 (float32_ref 3x3x256x256) [589824, bytes: 2359296]
generator_A/adaptive_resblock3/res2/conv_0/conv2d/bias:0 (float32_ref 256) [256, bytes: 1024]
generator_A/adaptive_resblock3/res2/instance_layer_norm/rho:0 (float32_ref 256) [256, bytes: 1024]
generator_A/up_conv_0/conv2d/kernel:0 (float32_ref 3x3x256x128) [294912, bytes: 1179648]
generator_A/up_conv_0/conv2d/bias:0 (float32_ref 128) [128, bytes: 512]
generator_A/layer_ins_norm_0/rho:0 (float32_ref 128) [128, bytes: 512]
generator_A/layer_ins_norm_0/gamma:0 (float32_ref 128) [128, bytes: 512]
generator_A/layer_ins_norm_0/beta:0 (float32_ref 128) [128, bytes: 512]
generator_A/up_conv_1/conv2d/kernel:0 (float32_ref 3x3x128x64) [73728, bytes: 294912]
generator_A/up_conv_1/conv2d/bias:0 (float32_ref 64) [64, bytes: 256]
generator_A/layer_ins_norm_1/rho:0 (float32_ref 64) [64, bytes: 256]
generator_A/layer_ins_norm_1/gamma:0 (float32_ref 64) [64, bytes: 256]
generator_A/layer_ins_norm_1/beta:0 (float32_ref 64) [64, bytes: 256]
generator_A/G_logit/conv2d/kernel:0 (float32_ref 7x7x64x3) [9408, bytes: 37632]
generator_A/G_logit/conv2d/bias:0[2022-04-12 22:08:17.501335] e[35mWARNINGe[0m:tensorflow:From /app/photo2avatar/UGATIT.py:620: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

2022-04-12 22:08:19.471 Uncaught app exception

While if it runs smoothly, it should have following part which stores a result image into .results/ folder:

generator_A/G_logit/conv2d/kernel:0 (float32_ref 7x7x64x3) [9408, bytes: 37632] generator_A/G_logit/conv2d/bias:0 (float32_ref 3) [3, bytes: 12] 
Total size of variables: 557940616 
Total bytes of variables: 2231762464 
WARNING:tensorflow:From /content/photo2avatar/UGATIT.py:620: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead. 
2022-04-11 21:07:40.956718: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1073741824 exceeds 10% of system memory. tcmalloc: large alloc 1073741824 bytes == 0x555d4640e000 @ 0x7f7c86882b6b 0x7f7c868a2379 0x7f7c3c81fee7 0x7f7c3c60d51f 0x7f7c3c4d805b 0x7f7c3c49da36 0x7f7c3c49e8c3 0x7f7c3c49ea93 0x7f7c45452a3b 0x7f7c45458197 0x7f7c3c751e0c 0x7f7c3c744575 0x7f7c3c802021 0x7f7c3c7ff718 0x7f7c851826df 0x7f7c866556db 0x7f7c8578a61f 
2022-04-11 21:07:41.246096: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1073741824 exceeds 10% of system memory. 
WARNING:tensorflow:From /content/photo2avatar/UGATIT.py:624: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead. 
[*] Reading checkpoints... 
2022-04-11 21:08:01.888000: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1073741824 exceeds 10% of system memory. 
2022-04-11 21:08:01.889555: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1073741824 exceeds 10% of system memory. 
[*] Success to read UGATIT.model-2044000 
[*] Load SUCCESS Processing A image: ./dataset/sample/testA/0000.png 
[*] Test finished!

Thank everyone for your help!!!

2

I wonder if it is a resource issue, have you tried to run the same model in the cloud at a smaller scale? The total bytes from your successful run seems large, I would try to run at a fraction of that total size and see if that is the problem.

1 Like
4

Thanks for your answer! However, the model is well-defined, I don’t know how to use a lighter version. Or is there any way for me to increase the resource limit?
Thank you again for your advice.

5

Depending on the use of your tool you can try the Streamlit Teams which allows for high resource per app, or another cloud service (e.g. GCP, AWS…).

6

After a quick glimpse at your github repo, i would say that at first cleanup your requirements.txt file.
It has typos and includes binary packages.

7

Thanks for your answer! Could you please indicate where is the typos and problems within the requirements.txt? Thank you very much!

8

I doubt this app will ever run on streamlit cloud.
It is full of ML libraries and reloads huge ML models.
The Docker container alone is around 4.6GB, then an external model of 7.5GB is reloaded at startup.
I doubt that so many ML libraries and these massive ML models are even necessary for this application.
I would try to slim down the application considerably.

1 Like
9

I tried to get the app running on streamlit cloud, see my github fork below, apply the changes with a pull request if needed.

  • in a local docker environment it runs
  • deployment or execution of the streamlit app to streamlit cloud fails most of the time, i guess the runtime is reaching its resource limits
  • the ML model seems to not generate any output, but also no error message
  • the 8GB download from the external ML model takes forever
  • unzipping the 8GB file also takes long and is not really necessary, you could also download the three files directly
  • I doubt that you need to use two ML frameworks (Tensorflow and Pytorch) at the same time for this application
  • I doubt that you need this huge ML model
  • the app unfortunately doesn’t seem very structured and quite plugged together
10

After increasing the resource limit of GCP, I successfully ran the ML model and deployed the app! Thank you very much for the debugging process!! It is very insightful!

11

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.