BookmarkSubscribeRSS Feed
☑ This topic is solved. Need further help from the community? Please sign in and ask a new question.
A_SAS_Man
Pyrite | Level 9

I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.

 

filename dat ZIP '/data/file.json.gz' GZIP;

libname dat json;
1 ACCEPTED SOLUTION

Accepted Solutions
Tom
Super User Tom
Super User

@A_SAS_Man wrote:

I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.

 

filename dat ZIP '/data/file.json.gz' GZIP;

libname dat json;

That code is not going to create any ALLDATA dataset, it is just defining the LIBREF that points to the JSON file.  

If you don't want to read DAT.ALLDATA then don't read it.

 

But there is an option that should suppress it.  Try it and see if it works in your version.

https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lestmtsglobal/n1jfdetszx99ban1rl4zll6tej7j.ht...

NOALLDATA

Suppresses creation of the ALLDATA data set.

Note This option is available beginning with SAS Viya 3.4.

 

 

View solution in original post

2 REPLIES 2
Tom
Super User Tom
Super User

@A_SAS_Man wrote:

I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.

 

filename dat ZIP '/data/file.json.gz' GZIP;

libname dat json;

That code is not going to create any ALLDATA dataset, it is just defining the LIBREF that points to the JSON file.  

If you don't want to read DAT.ALLDATA then don't read it.

 

But there is an option that should suppress it.  Try it and see if it works in your version.

https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lestmtsglobal/n1jfdetszx99ban1rl4zll6tej7j.ht...

NOALLDATA

Suppresses creation of the ALLDATA data set.

Note This option is available beginning with SAS Viya 3.4.

 

 

A_SAS_Man
Pyrite | Level 9

When I navigate to the "dat" library I can see an ALLDATA file. It looks like I may have too old of a version of SAS to use that option with unfortunately, I'm just using SAS EG I'm not familiar with Viya.

 

Edit: It seems as though the NOALLDATA keyword might be working for me even though it wasn't showing up before... Testing it out now and will accept this if it works.

SAS Innovate 2025: Save the Date

 SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!

Save the date!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 2 replies
  • 613 views
  • 0 likes
  • 2 in conversation