tensorflow Count examples in CSV file


Example

import tensorflow as tf
filename_queue = tf.train.string_input_producer(["file.csv"], num_epochs=1)
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
col1, col2 = tf.decode_csv(value, record_defaults=[[0], [0]])

with tf.Session() as sess:
  sess.run(tf.initialize_local_variables())
  tf.train.start_queue_runners()
  num_examples = 0
  try:
    while True:
      c1, c2 = sess.run([col1, col2])
      num_examples += 1
  except tf.errors.OutOfRangeError:
    print "There are", num_examples, "examples"

num_epochs=1 makes string_input_producer queue to close after processing each file on the list once. It leads to raising OutOfRangeError which is caught in try:. By default, string_input_producer produces the filenames infinitely.

tf.initialize_local_variables() is a tensorflow Op, which, when executed, initializes num_epoch local variable inside string_input_producer.

tf.train.start_queue_runners() start extra treads that handle adding data to the queues asynchronically.