topic Re: error executing hp02d02.sas in SAS Academy for Data Science

error executing hp02d02.sas

akpattnaik — Tue, 28 Nov 2017 15:31:19 GMT

Hi,

I'm in Chapter 2: accessing HDFS and Invoking Hadoop application and trying the demo hp02d02.sas

When i execute the code i got the error: Input path does not exist. (See attachment for more info.)

I went into Hue and found out /user/shared/DIACHD/data/Custord is not available. Hence, i tried to create a folder under shared. However, i got a permission issue. Again i tried to create the folder using proc hadoop mkdir statement and got the same permission issue. Attached is the screen shot trail of all my approach. Can anyone advice, if i'm missing anything please?

Re: error executing hp02d02.sas

TheresaStemler — Tue, 28 Nov 2017 16:10:49 GMT

@akpattnaik

Can you please send the entire SAS log, starting with the log for the PROC HADOOP step?

-theresa

Re: error executing hp02d02.sas

ballardw — Tue, 28 Nov 2017 16:23:13 GMT

Paste the log into a code box using the forum {I} menu icon.

Many users here do not or cannot open Docx files from unknown sources due to security issues and organization policies.

Re: error executing hp02d02.sas

akpattnaik — Wed, 29 Nov 2017 15:53:19 GMT

ERROR: 11-29 10:47:10,452 [RunPigThread][ERROR PigStatsUtil] - 1 map reduce job(s) failed!
ERROR: The pig script has encountered errors.
ERROR: The pig job, job_1511967281334_0001, has encountered errors.
ERROR: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: 
       hdfs://server2.demo.sas.com:8020/user/shared/DIACHD/data/custord
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:597)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:614)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
    at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
    at java.lang.Thread.run(Thread.java:745)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: 
       hdfs://server2.demo.sas.com:8020/user/shared/DIACHD/data/custord
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:321)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:264)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:385)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
    ... 18 more
3

Hi, Above is the error message on the SAS EG.

Re: error executing hp02d02.sas

TheresaStemler — Thu, 30 Nov 2017 00:29:36 GMT

@akpattnaik,

Please try the following steps:

Step 1: Did you successfully run cre8data.bat when you started the image?

If YES, go to Step 2. If NO, please run cre8data.bat as directed in section 1.2 of the Big Data Programming and Loading

module.

Step 2: Did you run cre8data.sas?

If YES, go to Step 3. If NO, please run cre8data.sas as directed in section 1.2 of the Big Data Programming and Loading

module.This program creates the data used in this module and a required macro variable.

Step 3: If you are returning to a saved image, submit the %LET statement in cre8data.sas to define the macro variable. You do not run

the entire program.

If this does not resolve your issue, your image maybe corrupted, I would discard this image and request a new image. Please update me on your progress.

-theresa