topic Re: Does SAS have a limit to how much data it can read? in SAS Enterprise Guide

Does SAS have a limit to how much data it can read?

Trick — Thu, 21 May 2015 15:34:43 GMT

So I converted a flat file on the mainframe to sas7bdat. with a scheme as TSO_TEXT, when I tried reading this file into saseg it wasn't able to read it in. Now this converted flat file has close to 2000000 records in it. I tried using csv but when I tried opening that into sas it couldn't open it in SASEG but opened it in text pad.

Re: Does SAS have a limit to how much data it can read?

TomKari — Thu, 21 May 2015 15:41:31 GMT

Hi, Trick

I assure you, you're not even CLOSE to any SAS limitations. I think perhaps you're doing something a little offbeat.

What exactly did you do, and what are you trying to achieve?

Tom

Re: Does SAS have a limit to how much data it can read?

Trick — Thu, 21 May 2015 15:51:44 GMT

So here's my process step by step.

Run JCL.-complete
Take sorted file and FTP it over to SAS server(but not doing it now trying to test it)

So I go to the menu option on the mainframe and click on the transfer dropdown then do a receive from host.

Which then I select scheme as TSO_TXT

Destination is left the same

Host is set to the flat file

Localpath is set to the directory i'm trying to store the file to.

After the file has being saved to the directory. I open sas and try to open the .sas7bdat file in it.

Re: Does SAS have a limit to how much data it can read?

TomKari — Thu, 21 May 2015 16:22:24 GMT

It looks to me like in steps

"Host is set to the flat file" and "Localpath is set to the directory i'm trying to store the file to" you're referring to a flat file (does this just mean a file with text in it, like a windows ".txt" file?)

and in the next step you're referring to opening a .sas7bdat file.

I'm not sure if this is repetition for you; if it is, I apologize. SAS uses a proprietary data format for its data files, which it tags with a .sas7bdat extension. To create a SAS data file from a text file involves reading and converting the text file using SAS. Have you done this?

Tom

Re: Does SAS have a limit to how much data it can read?

Trick — Thu, 21 May 2015 16:26:30 GMT

so what I did was try loading a txt file into sas and got this
Yes that is correct.

Re: Does SAS have a limit to how much data it can read?

Trick — Thu, 21 May 2015 16:30:39 GMT

Another thing is that i'm using a SAS program on the mainframe to generate this flat file that i'm try to FTP over to the SAS server. If i'm reading this correctly I would have to ftp the file over as a .txt and then convert it to a .sas7bdat. Since I can't just save a file as a .sas7bdat

Re: Does SAS have a limit to how much data it can read?

ChrisHemedinger — Thu, 21 May 2015 16:42:31 GMT

To import the file in SAS Enterprise Guide, select File->Import Data, and then select the text file. That will open it into the Import Data wizard that leads you through the process.

You're seeing the message about "too large" because SAS Enterprise Guide doesn't want to open a large text file for viewing. That's a precaution that helps prevent the process from "choking" on large files in the text editor. But the Import Data task can handle very large files, no problem.

Chris

Re: Does SAS have a limit to how much data it can read?

bbear — Sat, 29 Apr 2017 19:22:03 GMT

The message I get for my file .sas7bdat is:

"The size of the uploaded files cannot be greater than 10 MB. The size of the selected files is 31.5 MB."

Any way to fix this?I can't upload it / import directly.

Re: Does SAS have a limit to how much data it can read?

SASKiwi — Sun, 30 Apr 2017 00:03:48 GMT

Welcome to the community. Please note your problem isn't the same as the original so you would get better visibility by opening a new post. BTW it is also better community etiquette .

I'm guessing you are using the SAS File Transfer task in EG - is that right? Also what version of EG are you using?

As I understand it this EG task is designed as a simple way of moving "small" SAS files between your PC and your SAS server. There are better options for large data files but they are mostly not so simple. One easy way if your SAS server runs on Windows is to have SAS server folders set up as network shares then you can do a copy and paste in Windows Explorer.

We need to know more about your SAS setup to investigate alternatives.