sqoop Import New Data - append mode


If you are only adding new rows in your RDBMS (not updating existing data)

You need two additional parameters:

  • --check-column : A column name that should be checked for newly appended data. Integer would be a suitable data type for this column.
  • --last-value : The last value that successfully imported into Hadoop. All the newly added data after this value will be imported.
sqoop import \
--connect jdbc:mysql://mysql.example.com/testdb \
--username sqoop \
--password sqoop \
--table employee \
--incremental append \
--check-column id \
--last-value 100