apache-flink logging Using different configuration(s) for each application


Example

In case you need different settings for your various applications, there is (as of Flink 1.2) no easy way to do that.

If you use the one-yarn-cluster-per-job mode of flink (i.e. you launch your scripts with: flink run -m yarn-cluster ...), here is a workaround :

  1. create a conf directory somewhere near your project

  2. create symlinks for all files in flink/conf:

     mkdir conf
     cd conf
     ln -s flink/conf/* .
    
  3. replace the symlink log4j.properties (or any other file you want to change) by your own configuration

  4. before launching your job, run

     export FLINK_CONF_DIR=/path/to/my/conf
    

Depending on your version of flink, you might need to edit the file flink/bin/config.sh. If your run accross this line:

FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf

change it with:

if [ -z "$FLINK_CONF_DIR" ]; then 
    FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf; 
fi