13

I have a few tensors in my code and and need to get the values of those tensors. This is one them. How to print the values of tensor OA?

Input:OA
Output: <tf.Tensor 'Sum_1:0' shape=(1, 600) dtype=float32>

Input:type(OA)
Output: tensorflow.python.framework.ops.Tensor

I have tried all the available functions like tf.print(), eval(), tensor.numpy(). None of them worked for me in Tensorflow 2.0. It seems they work only for 'EagerTensor' and not for 'ops.Tensor'.

1) OA.eval(session=sess) Error: ValueError: Cannot use the given session to evaluate tensor: the tensor's graph is different from the session's graph.

2) tf.print(OA) Output:

3) print (OA.numpy()) Output: AttributeError: 'Tensor' object has no attribute 'numpy'

Is there any way to convert ops.Tensor to EagerTensor to try the above functions? Or is there any other option to print the values of ops.Tensor. Please advise.

--Adding the minimal code to reproduce the example ops.Tensor in TF2.0.

!pip install tensorflow==2.0.0
tf.__version__

import tensorflow as tf
from keras.layers import Dense, Conv1D, MaxPooling1D, Flatten, Dropout, Input, Embedding, Bidirectional, LSTM
from tensorflow.keras import regularizers

EMBEDDING_DIM = 300
max_length = 120
batch_size = 512
vocab_size = 1000
units = 300

from keras.layers import Dense, Conv1D, MaxPooling1D, Flatten, Dropout, Input, Embedding, Bidirectional, LSTM
from tensorflow.keras import regularizers

input_text = tf.keras.Input(shape= (max_length), batch_size=batch_size)

embedding_layer = tf.keras.layers.Embedding(vocab_size, EMBEDDING_DIM, input_length =max_length, name="Embedding_Layer_1")
embedding_sequence = embedding_layer(input_text)

HQ = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units,recurrent_dropout=0.5,kernel_regularizer=regularizers.l2(0.001),return_sequences=True,name='Bidirectional_1'))(embedding_sequence)
HQ = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units,recurrent_dropout=0.5,kernel_regularizer=regularizers.l2(0.001),name='Bidirectional_2'))(HQ)

print (HQ)

Output: Tensor("bidirectional_3/concat:0", shape=(512, 600), dtype=float32)

type(HQ)

Output: tensorflow.python.framework.ops.Tensor

How to check the actual values of this tensor?

Raghu
  • 387
  • 3
  • 12
  • Please provide a [minimal, reproducible example](https://stackoverflow.com/help/minimal-reproducible-example). How do you create `OA`? – jakub Feb 21 '20 at 12:54
  • 1
    @jakub - Can you please check now. I updated the question with minimal reproducible example. – Raghu Mar 06 '20 at 11:12
  • @Raghu - I had a similar need. I wanted to inspect the embedding tensor of a constant sequence of indices using: MyEmbedding = tf.keras.layers.Embedding(tf.constant([[[0], [1], [5], [500]]])). As a result, MyEmbedding was a tensorflow.python.framework.ops.Tensor object. The only way I could print the array was by following the first example here: https://stackoverflow.com/questions/52215711/tensorflow-tensor-to-numpy-array-conversion-without-running-any-session/52216282, using tf.enable_eager_execution() and tensor.numpy(). Hope it helps. – George Sep 25 '20 at 01:40
  • `tf.config.run_functions_eagerly(True)` executed once at the beginning solved the problem for me for a similar problem – Lostefra Oct 25 '21 at 10:13

2 Answers2

1

Your graph is not complete at the point you are printing HQ. You need to complete the model creation. Presumably something like

output = tf.keras.layers.xyz()(HQ)
model = tf.keras.models.Model(input_text, output)

The trick to print an intermediate layer is to just make it an output. You can make it an additional output of your existing model temporarily, or just make a new model.

inspection_model = tf.keras.models.Model(input_text, [output, HQ])

now run inference on your inspection_model to get the value of the intermediate activation HQ.

print(inspection_model(xyz))
Yaoshiang
  • 1,385
  • 4
  • 12
-2

Use .numpy() attribute like :

your_tensor.numpy()
DachuanZhao
  • 881
  • 3
  • 10
  • 24