error executing hp02d02.sas

[akpattnaik]

Hi,

I'm in Chapter 2: accessing HDFS and Invoking Hadoop application and trying the demo hp02d02.sas

When i execute the code i got the error: Input path does not exist. (See attachment for more info.)

I went into Hue and found out /user/shared/DIACHD/data/Custord is not available. Hence, i tried to create a folder under shared. However, i got a permission issue. Again i tried to create the folder using proc hadoop mkdir statement and got the same permission issue. Attached is the screen shot trail of all my approach. Can anyone advice, if i'm missing anything please?

[TheresaStemler]

@akpattnaik

Can you please send the entire SAS log, starting with the log for the PROC HADOOP step?

-theresa

[akpattnaik]

ERROR: 11-29 10:47:10,452 [RunPigThread][ERROR PigStatsUtil] - 1 map reduce job(s) failed!
ERROR: The pig script has encountered errors.
ERROR: The pig job, job_1511967281334_0001, has encountered errors.
ERROR: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: 
       hdfs://server2.demo.sas.com:8020/user/shared/DIACHD/data/custord
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:597)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:614)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
    at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
    at java.lang.Thread.run(Thread.java:745)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: 
       hdfs://server2.demo.sas.com:8020/user/shared/DIACHD/data/custord
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:321)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:264)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:385)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
    ... 18 more
3               

Hi, Above is the error message on the SAS EG.

[ballardw]

Paste the log into a code box using the forum {I} menu icon.

Many users here do not or cannot open Docx files from unknown sources due to security issues and organization policies.

[TheresaStemler · Accepted solution]

@akpattnaik,

Please try the following steps:  

Step 1: Did you successfully run cre8data.bat when you started the image? 

            If YES, go to Step 2. If NO, please run cre8data.bat as directed in section 1.2 of the Big Data Programming and Loading

            module.

Step 2: Did you run cre8data.sas?

            If YES, go to Step 3. If NO, please run cre8data.sas as directed in section 1.2 of the Big Data Programming and Loading

            module.This program creates the data used in this module and a required macro variable.

Step 3: If you are returning to a saved image, submit the %LET statement in cre8data.sas to define the macro variable. You do not run

            the entire program.

If this does not resolve your issue, your image maybe corrupted, I would discard this image and request a new image. Please update me on your progress.

-theresa