Tutorial by Examples

When state_below is a 2D Tensor, U is a 2D weights matrix, b is a class_size-length vector: logits = tf.matmul(state_below, U) + b return tf.nn.softmax(logits) When state_below is a 3D tensor, U, b as before: def softmax_fn(current_input): logits = tf.matmul(current_input, U) + b ret...

Use tf.nn.sparse_softmax_cross_entropy_with_logits, but beware that it can't accept the output of tf.nn.softmax. Instead, calculate the unscaled activations, and then the cost: logits = tf.matmul(state_below, U) + b cost = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels) In this c...