topic Re: Requested data exceeds maximum data size for a single request on SAS Viya Prepare Data in SAS Viya

Requested data exceeds maximum data size for a single request on SAS Viya Prepare Data

ryuda — Wed, 23 May 2018 10:26:21 GMT

Hi all,

I am quite new with SAS. I have been exploring SAS Viya. When I am using the Prepare Data UI to transpose a data sized 16MB it gives an error as follow.

Requested data exceeds maximum data size for a single request.
correlator: 40880352-edfa-4843-80c1-0baa8cbf9e29
traceId: 121ca0734d1b7684
path: /SASDataStudio/ui/asyncOperation
correlator: 40880352-edfa-4843-80c1-0baa8cbf9e29;70e77e21-d0e5-49f6-9686-f0cd42de9f12
traceId: 121ca0734d1b7684
path: /casRowSets/tables/cas~fs~cas-shared-default~fs~Public~fs~DATAEKSPORNONMIGASLOKALORIGINAL_1/rows
MLC: 0
Row count : 100
Row bytes : 1,088,860
Data limit: 1,048,576
Data bytes: 108,886,000

Does anyone has ever come across this? What is needed to do in this case?

Please help, thanks

Re: Requested data exceeds maximum data size for a single request on SAS Viya Prepare Data

RW9 — Wed, 23 May 2018 10:33:52 GMT

You could show some of the data, at least test data. From what is posted there, it is telling you that each row is roughly 1088 characters long, so when transposed the length will be 100 * 1088. I can't see any situation where you would that much data in one row, even what you have now is pretty long. Or alternatively use a word processor for dealing with novels?

Re: Requested data exceeds maximum data size for a single request on SAS Viya Prepare Data

ryuda — Mon, 28 May 2018 21:00:32 GMT

I accepted this answer as a solution, however I rather feel that limit of each record should be configurable, since we're working on a server based software. Kind of ironic that even word processor such as MS Excel can handle this. Though I can not say this is a solution since there are thousands of these files that I will be working with

Re: Requested data exceeds maximum data size for a single request on SAS Viya Prepare Data

RW9 — Tue, 29 May 2018 08:04:34 GMT

MS Excel is not a Word processor. And even that has its limitations - actually far more restrictive than a SAS dataset.

https://support.office.com/en-us/article/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3

Where would you draw the line, is 100 columns of large text data usable in anyway? Can you print it out to paper, can you do any proper analysis on that data. The answer is no. You can work with that data, as you should with almost all data, in a normalised structure, which you already have. If you need to analyse it then you need to do data cleaning and coding practices to get it into a workable format. The only application which uses "transposed thinking" is Excel, with all its numerous problems.