apache-flink Savepoints and externalized checkpoints Savepoints: requirements and preliminary notes

Example

A savepoint stores two things: (a) the positions of all datasources, (b) the states of operators. Savepoints are useful in many circonstances:

• slight application code updates
• Flink update
• changes in parallelism
• ...

As of version 1.3 (also valid for earlier version):

• checkpoint must be enabled for the savepoints to be possible. If you forget to explicitly enable checkpoint using:

  StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
  env.enableCheckpointing(checkpointInterval);
  

  you will get:

  java.lang.IllegalStateException: Checkpointing disabled. You can enable it via the execution environment of your job
  

• when using window operations, it is crucial to use event-time (vs ingestion or processing time) to yield proper results;

• to be able to upgrade a program and reuse savepoints, manual uid must be set. This is because, by default, Flink changes the operator's UID after any change in their code;

• Chained operators are identified by the ID of the first task. It’s not possible to manually assign an ID to an intermediate chained task, e.g. in the chain [ a -> b -> c ] only a can have its ID assigned manually, but not b or c. To work around this, you can manually define the task chains. If you rely on the automatic ID assignment, a change in the chaining behaviour will also change the IDs (see point above).

More info is available in the FAQ.