Learning SAS? Welcome to the exclusive online community for all SAS learners.

Upload more than 10 MB to SAS on AWS

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 17
Accepted Solution

Upload more than 10 MB to SAS on AWS

Dear all, 

 

SAS on AWS has this ridiculous upload restriction of 10MB. Is there a way to upload more data?

 

Thank you!


Accepted Solutions
Solution
‎02-11-2016 04:38 AM
SAS Super FREQ
Posts: 361

Re: Upload more than 10 MB to SAS on AWS

 

Hi, Balassa! Yes, the limit is 10MB for SAS Studio on AWS. You could break up the data into chunks if you need. Since SAS University Edition is for learning to code, the 10MB limit is usually acceptable for that purpose.

 

Alternatively, you can also use SAS University Edition as a Virtual Machine on your physical desktop machine. In that case, using Shared Folders, the amount of data you can use is limited only by your hard disk size.

 

Thanks,

Randy

View solution in original post


All Replies
Solution
‎02-11-2016 04:38 AM
SAS Super FREQ
Posts: 361

Re: Upload more than 10 MB to SAS on AWS

 

Hi, Balassa! Yes, the limit is 10MB for SAS Studio on AWS. You could break up the data into chunks if you need. Since SAS University Edition is for learning to code, the 10MB limit is usually acceptable for that purpose.

 

Alternatively, you can also use SAS University Edition as a Virtual Machine on your physical desktop machine. In that case, using Shared Folders, the amount of data you can use is limited only by your hard disk size.

 

Thanks,

Randy

Occasional Contributor
Posts: 17

Re: Upload more than 10 MB to SAS on AWS

Hi Randy!

 

Thank you very much! I should get a much faster computer then!

 

 

Occasional Contributor
Posts: 19

Re: Upload more than 10 MB to SAS on AWS

Randy,

 

I do not know what school in todays world uses data analytics where it is not necessary to load a file larger than 10MB.  In todays world of data science and peta, exa, zetta or even yotta bytes a 10 mb limit is useless.  I have the same requirment.  I am trying to load several NIH datasets that are larger than 10MB.  Breaking them apart to put them back together again is extremely a painstaking task.  I am in a PhD program in biomedical informatics at Rutgers

Frequent Contributor
Posts: 130

Re: Upload more than 10 MB to SAS on AWS

yes you can.. see

 

https://communities.sas.com/t5/SAS-Analytics-U/Is-it-possible-to-use-PROC-HTTP-in-SAS-university-edi...

 

for an example.

 

I use this method to upload large (>1GB) data files for teaching predictive analytics.

 

You need the data to be served from an unsecure http server (i.e. not a secure https:// server). I use a tame http:// server on my home network if needed when the big data is sourced originally from https:// servers.

 

Let me know if this helps.

Occasional Contributor
Posts: 19

Re: Upload more than 10 MB to SAS on AWS

I like the work around.  but it does not work in the AWS cloud environment where we do not have access to the local computer from within the SAS platform other than the built-in import functionality of SAS studio

Frequent Contributor
Posts: 130

Re: Upload more than 10 MB to SAS on AWS

Hmm. I'm not sure I nderstand what you're saying.
That seems strange to me. I run SAS UE on my local notebooks for an increasing proportion of my own university research and SAS On Demand in the SAS-administered AWS cloud for all of my SAS teaching and that proc http method works for both - the only drawback as I mentioned is that either (a) your data is already available on an unsecured HTTP server for download or (b) you have write or upload access permission to, or can set up yourself, an unsecured http web file server folder, and transfer those big external files from secure https server or other locations to your 'tame' unsecured http server.

Are you sure that when you install your SAS UE on your personal AWS server space proc http can't acess an unsecured http:// file service? You could test using my one if you had the time and storage space, I think it's still up.
Occasional Contributor
Posts: 19

Re: Upload more than 10 MB to SAS on AWS

I would love to know how than if you know its possible.  i run SAS Studio in the SAS UE on AWS.  I will run proc http and test the said functionality.  

Occasional Contributor
Posts: 5

Re: Upload more than 10 MB to SAS on AWS

Hi, I will use this topic since I think I have a similar problem.
I am using SAS University with Oracle virtual box for a project at university.
I have to do an analysis on 3 file, and one of them is 10,8MB large.
I've put the 3 files on my drive in the common folder and SAS correctly sees all of them. But if I try to open them, with the 2 smaller ones I have no problem at all, where with the larger one the loading windows starts and never complete the task (I tried to leave it there loading for even 15 minutes..).
Is this because of the size of the file? It's just that at the beginning of this topic it says that if you have the file stored on your computer and SAS correctly sees it I should have no problem working with it right?

Many thanks for any reply that you can give me.

Federico
SAS Super FREQ
Posts: 361

Re: Upload more than 10 MB to SAS on AWS

Hi, @FedeTS89. There is a limit on the size of /tmp space and the work space in SAS University Edition. The VM ships with about 5-6GB taken of 10GB total. A large dataset might not fit within the file size limits. The limits are usually sufficient for learn to code.

 

Hope this helps! Perhaps you can get the same learning experience by reducing the size of your input data?

 

Thanks,

Randy

Occasional Contributor
Posts: 5

Re: Upload more than 10 MB to SAS on AWS

Hi Randy, thanks for your reply.

I don't think that this is my case, I sent the file to a couple of other students of my class and they can open the file with their SAS University version. I tried ti install the virtual machine with SAS uni on 2 different computers and I have the same problem with both. The program works fine but when I try to open this file it freezes. I really have no idea what to do more.
I could send you the file if that could help you to find a solution?

Many thanks,
Federico
New Contributor
Posts: 3

How to break up a file that is more than 10 MB in SAS

Dear SAS Community,

 

I am new to SAS. I am using SAS University edition.

 

I purchased a SAS book called Administrative Healthcare Data by Craig Dickstein. The book provides a downloadable file set ( a series of files) that in the case 3 of them are larger than 10 mb each. How does one go about reducing a file (e.g. facilitydetail.sas7bdat) which is 12 mb in size into a smaller file ? 

 

My thought was to convert this file into a .txt file and then import into Excel and truncate it. Then save it back to a .txt file. I tried this and it did not work. I am not familiar with the SAS file types so I am reaching out to our experts to assist me.

 

I can attach one of the sample files as required.

 

Thanks for your help in advance.

 

Best,

jr

SAS Super FREQ
Posts: 361

Re: How to break up a file that is more than 10 MB in SAS

Hi, @jayroysasuser. If you simply put the files in your Shared Folder, you will see them in the Files pane on the left side; no need to upload. Does that make sense? Then you can use them in SAS University Edition.

 

Please let me know if I'm misunderstanding what you have. 

 

(I rarely upload files in SAS University Edtion; I just put them in the Shared Folder (myfiles)).

 

Thanks,

Randy

New Contributor
Posts: 3

Re: How to break up a file that is more than 10 MB in SAS

Hi Randy,

 

thank you very much for quick response. But sorry, I don't know what you mean by simply "putting" them there.

 

To clarify, I think you mean in the Server Files and Folder area correct? 

 

i tried to drag from my windows folder and drop the file there but it does not seem to work. 

 

Sorry what am I doing wrong?

 

Jay

SAS Super FREQ
Posts: 361

Re: How to break up a file that is more than 10 MB in SAS

 

Hi! You would set up a Shared Folder as described here:

 

https://www.youtube.com/watch?v=ttvffI0kx64

 

Once that Shared Folder is created, both the SAS University Edition virtual machine and your host PC (or Mac) will be able to see that same disk space. This is a Shared Folder: a folder that both the host (your PC) and the guest (the SAS University Edition VM) can both see and share.

 

So, after setting up your Shared Folder, you'll just copy the files to that location on your PC. Magically, the files will then appear in the Files tab of SAS Studio. 

 

I hope this makes sense! You should always store your files in a Shared Folder when using SAS University Edition so that you won't lose them if the VM should be updated or otherwise removed.

 

Randy

In this video you'll learn how to set up shared folders for your SAS University Edition virtual machine. SAS ANALYTICS U It's all about U! Whether you're a teacher, professor, student, academic researcher or independent learner, we offer anyone in a teaching and learning environment free and ...
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 17 replies
  • 2547 views
  • 4 likes
  • 6 in conversation