Tutorial by Examples

To complete this objective following tasks are required. Foreach Loop Container: To iterate over a user configured directory for files. Expression Task: To update a variable if file exists. Steps First goto Solution Explorer double click on Project.params and create a parameter FolderPat...

The below coordinator job will trigger coordinator action once in a day that executes a workflow. The workflow has a shell script that moves input to output. <coordinator-app name="log_process_coordinator" frequency="${coord:days(1)}" start="2017-04-29T06:00Z" ...

<workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf"> <start to="shell-node"/> <action name="shell-node"> <shell xmlns="uri:oozie:shell-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-n...

nameNode=hdfs://namenode:port start=2016-04-12T06:00Z end=2017-02-26T23:25Z jobTracker=yourjobtracker poolName=yourpool oozie.coord.application.path=${nameNode}/hdfs_path/coord_job_example/coord workflowAppUri=${oozie.coord.application.path} myscript=myscript.sh myscriptPath=${oozie.coord.ap...

inputDir=${1} outputDir=${2} hadoop fs -mkdir -p ${outputDir} hadoop fs -cp ${inputDir}/* ${outputDir}/

Copy the script, coordinator.xml and workflow.xml into HDFS. coordinator.xml must be present in the directory specified by oozie.coord.application.path in job.properties. workflow.xml should be present in the directory specified by workflowAppUri. Once everything is in place, run the below command f...

Table/Column Names Two common ways of formatting table/column names are CamelCase and snake_case: SELECT FirstName, LastName FROM Employees WHERE Salary > 500; SELECT first_name, last_name FROM employees WHERE salary > 500; Names should describe what is stored in their object. Th...

SELECT * returns all columns in the same order as they are defined in the table. When using SELECT *, the data returned by a query can change whenever the table definition changes. This increases the risk that different versions of your application or your database are incompatible with each other....

There is no widely accepted standard. What everyone agrees on is that squeezing everything into a single line is bad: SELECT d.Name, COUNT(*) AS Employees FROM Departments AS d JOIN Employees AS e ON d.ID = e.DepartmentID WHERE d.Name != 'HR' HAVING COUNT(*) > 10 ORDER BY COUNT(*) DESC; At th...

Explicit joins should always be used; implicit joins have several problems: The join condition is somewhere in the WHERE clause, mixed up with any other filter conditions. This makes it harder to see which tables are joined, and how. Due to the above, there is a higher risk of mistakes, an...

Content of file.json (one JSON object per line): {"A": 1, "B": 2} {"A": 3, "B": 4} How to read directly from a local file: pd.read_json('file.json', lines=True) # Output: # A B # 0 1 2 # 1 3 4

Creating Express Web Server Express server came handy and it deeps through many user and community. It is getting popular. Lets create a Express Server. For Package Management and Flexibility for Dependency We will use NPM(Node Package Manager). Go to the Project directory and create package....

if len(sys.argv) != 4: # The script name needs to be accounted for as well. raise RuntimeError("expected 3 command line arguments") f = open(sys.argv[1], 'rb') # Use first command line argument. start_line = int(sys.argv[2]) # All arguments come as strings, so need to ...

# The name of the executed script is at the beginning of the argv list. print('usage:', sys.argv[0], '<filename> <start> <end>') # You can use it to generate the path prefix of the executed program # (as opposed to the current module) to access files relative to that, # which...

# Error messages should not go to standard output, if possible. print('ERROR: We have no cheese at all.', file=sys.stderr) try: f = open('nonexistent-file.xyz', 'rb') except OSError as e: print(e, file=sys.stderr)

def main(): if len(sys.argv) != 4 or '--help' in sys.argv[1:]: print('usage: my_program <arg1> <arg2> <arg3>', file=sys.stderr) sys.exit(1) # use an exit code to signal the program was unsuccessful process_data()

$man <command> Displays the on-line manual pages for the command $clear Clears the terminal screen $pwd Returns the working directory name $echo <string> Writes the string to the standard output $printf <string> Format and print the string Example: print $PATH ...

A common problem with remote services is rate limiting. The remote service allows us to send only a limited number of requests or amount of data per time period. In RxJS 5 a very similar functionality is provided by the bufferTime operator and especially if we leave the second parameter unspecified...

This example contains over 1000 lines of code in total (too much to be embedded here). For that reason all code is accessible on http://blockbuilder.org/SumNeuron/956772481d4e625eec9a59fdb9fbe5b2 (alternatively hosted at https://bl.ocks.org/SumNeuron/956772481d4e625eec9a59fdb9fbe5b2). Note the bl.oc...

To show the value of making generalized functions like those in the previous example (make_title, make_axes, make_buttons, etc), consider this box and whisker chart: https://bl.ocks.org/SumNeuron/262e37e2f932cf4b693f241c52a410ff While the code for making the boxes and whiskers is more intensive tha...