当前位置: 动力学知识库 > 问答 > 编程问答 >

hadoop - Sqoop not working with Oozie (using HUE)

问题描述:

I tried to use sqoop using Oozie scheduler, however the job is killed immediately by Oozie. Sqoop command works fine through terminal.

sqoop import \

--connect "jdbc:xxxxxxxx" \

--username=xxxxxxxx \

--password=xxxxxxxx \

--driver ctree.jdbc.ctreeDriver \

--table odhfgdb \

--target-dir "/acting/user/odhfgdb-tmp" \

--fields-terminated-by '\001' \

--escaped-by \\ \

--delete-target-dir \

--split-by entry_date \

--where="entry_date < CURRENT_DATE" --m 30

Can you tell me, what do I need to do in Oozie for this command to work?

Following is the log:

2016-06-27 19:27:51,492 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

2016-06-27 19:27:51,493 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE

2016-06-27 19:27:51,494 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB!

2016-06-27 19:27:52,310 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

2016-06-27 19:27:56,418 INFO SqoopActionExecutor:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] checking action, hadoop job ID [job_1466989378023_0024] status [RUNNING]

2016-06-27 19:27:56,420 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING

2016-06-27 19:27:56,421 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB!

2016-06-27 19:28:14,293 INFO CallbackServlet:543 - SERVER[servername] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] callback for action [[email protected]]

2016-06-27 19:28:15,752 INFO SqoopActionExecutor:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] action completed, external ID [job_1466989378023_0024]

2016-06-27 19:28:15,764 WARN SqoopActionExecutor:546 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

2016-06-27 19:28:16,137 INFO ActionEndXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] ERROR is considered as FAILED for SLA

2016-06-27 19:28:16,257 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] Start action [0000010-160627000427464-oo[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

2016-06-27 19:28:16,258 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=DONE

2016-06-27 19:28:16,259 INFO ActionStartXCommand:543 - SERVER[servername] USER[admin] GROUP[-] TOKEN[] APP[My_Workflow] JOB[0000010-160627000427464-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB!

Following is the workflow generated in HUE:

<workflow-app name="My_Workflow" xmlns="uri:oozie:workflow:0.5">

<start to="sqoop-283e"/>

<kill name="Kill">

<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>

</kill>

<action name="sqoop-283e">

<sqoop xmlns="uri:oozie:sqoop-action:0.2">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<command>import --connect jdbc:xxxxxxx --username=xxxxxx --password=xxxxxxx --driver ctree.jdbc.ctreeDriver --table odhfgdb --target-dir /acting/user/odhfgdb-tmp --fields-terminated-by \001 --escaped-by \\ --delete-target-dir --split-by entry_date --where=entry_date &lt; CURRENT_DATE --m 30</command>

</sqoop>

<ok to="End"/>

<error to="Kill"/>

</action>

<end name="End"/>

</workflow-app>

分享给朋友:
您可能感兴趣的文章:
随机阅读: