TensorFlow Variables
TensorFlow is a form of lazy computation without actually performing it until asked. They can be manipulated using tf.Variable class. A tf.Variable represents whose value can be changed by running operations on it. Unlike tf.Tensor objects, a tf.Variable exists outside the context of a single session.run call.
A persistent tensor is stored by tf.Variable. Certain operations will allow you to manipulate the values of this tensor. These changes are visible for multiple tf.Sessions, so multiple workers can see the same values for a tf.Variable.
Creating a Variable:
tf.get_variable function create a variable which requires you to specify the Variable’s name. This name will be used by other replicas to access the same variable, as well as to name this variable’s value when check pointing and exporting models. tf.get_variable also allows you to reuse a previously created variable of the same name, making it easy to define models which reuse layers.
Variable_1 = tf.get_variable(“variable_1", [6,7,8])
This creates a variable named “variable_1″ which is a three-dimensional tensor with shape [6,7,8]. This variable will, by default, have the dtype tf.float32 and its initial value will be randomized via tf.glorot_uniform_initializer.
You may optionally specify the dtype and initializer to tf.get_variable.
For Example :
int_variable = tf.get_variable("int_variable", [1, 2, 3],
dtype=tf.int32,initializer=tf.zeros_initializer)
TensorFlow provides another convenient initializer. Alternatively, you can initialize a tf.Variable to have the value of a tf.Tensor. The shape of the initializer tensor will be used when we use tf.Tensor.
Example:
variable_2 = tf.Variable("variable_2", dtype=tf.int32, initializer=tf.constant([23, 42]))
Variable collections:
Disconnected parts of a TensorFlow program might want to create variables, There is way to access all of them. For this TensorFlow provides collections, which are named lists of tensors or other objects, such as tf.Variable instances.
By default tf.Variable gets placed in the following two collections:
tf.GraphKeys.GLOBAL_VARIABLES — multiple devices can share these variables
tf.GraphKeys.TRAINABLE_VARIABLES — variables for which TensorFlow will calculate gradients.
Initializing variables:
Before using a variable, it has to be initialized. If you are explicitly creating your own graphs and sessions (low-level TensorFlowAPI ), you must explicitly initialize the variables. Most high-level frameworks such as tf.contrib.slim, tf.estimator.Estimator and Keras automatically initialize variables before training a model.
Explicit initialization is otherwise useful because it allows you not to rerun potentially expensive initializers when reloading a model from a checkpoint as well as allowing determinism when randomly-initialized variables are shared in a distributed setting.
tf.global_variables_initializer() to initialize all trainable variables in one go, before training starts,This function returns a single operation responsible for initializing all variables in the tf.GraphKeys.GLOBAL_VARIABLES collection. Running this operation initializes all variables.
For example:
session.run(tf.global_variables_initializer())
# Now all variables are initialized.
Example:
var_init = tf.get_variable("var_init", [10, 12], dtype=tf.int32, initializer=tf.zeros_initializer)
print(var_init.shape)
Output:(10,12)
tf.global_variables
tf.global_variables(scope=None)
Returns global variables:
Global variables are shared across machines in a distributed environment. Variable() constructor or get_variable() automatically adds new variables to the graph collection GraphKeys.GLOBAL_VARIABLES. This convenience function returns the contents of that collection.
An alternative to global variables are local variables. See tf.local_variables
Args:
scope: (Optional) A string. If supplied, the resulting list is filtered to include only items whose name attribute matches scope using re.match. Items without a name attribute are never returned if a scope is supplied.
Returns:
A list of Variable objects.