I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.
filename dat ZIP '/data/file.json.gz' GZIP;
libname dat json;
@A_SAS_Man wrote:
I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.
filename dat ZIP '/data/file.json.gz' GZIP; libname dat json;
That code is not going to create any ALLDATA dataset, it is just defining the LIBREF that points to the JSON file.
If you don't want to read DAT.ALLDATA then don't read it.
But there is an option that should suppress it. Try it and see if it works in your version.
Suppresses creation of the ALLDATA data set.
Note | This option is available beginning with SAS Viya 3.4. |
---|
@A_SAS_Man wrote:
I am using the json engine to parse some very large datafiles. I notice in the process of parsing these files that it creates a dataset of "ALLDATA" which seems to just be some sort of appending of all the other tables/objects that isn't super useful to me. From the reading I've done my understanding is this is an artifact of how sas processes json files rather than how a json file is structured necessarily (but open to corrections there). Is there a way to have it skip doing this to help speed up the process? Code I'm using is very simple but see below.
filename dat ZIP '/data/file.json.gz' GZIP; libname dat json;
That code is not going to create any ALLDATA dataset, it is just defining the LIBREF that points to the JSON file.
If you don't want to read DAT.ALLDATA then don't read it.
But there is an option that should suppress it. Try it and see if it works in your version.
Suppresses creation of the ALLDATA data set.
Note | This option is available beginning with SAS Viya 3.4. |
---|
When I navigate to the "dat" library I can see an ALLDATA file. It looks like I may have too old of a version of SAS to use that option with unfortunately, I'm just using SAS EG I'm not familiar with Viya.
Edit: It seems as though the NOALLDATA keyword might be working for me even though it wasn't showing up before... Testing it out now and will accept this if it works.
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.
Ready to level-up your skills? Choose your own adventure.