DATA Step, Macro, Functions and more

reading a large data blob from sybase

Reply
Contributor
Posts: 73

reading a large data blob from sybase

I am trying to load a table from sybase that contains a large blob of data in hex format.  The problem I'm running into is that the blob is 2147483647 characters long and SAS can only handle some 32K characters at a time.

 

I'm in the middle of trying to understand the structure of the data inside the blob which is not just numbers but first I need to be able to load the data

 

PS the table contains 44K rows that I have to read

 

any suggestions?

Super User
Posts: 3,261

Re: reading a large data blob from sybase

Posted in reply to wkossack_nspirehealth_com

If you divide your blob size by 32K, then you will find that if you were able to deconstruct your BLOB into 32K chunks then this would take 67K SAS variables to store!

 

I suggest you ask your Sybase DBA to find out if Sybase has any functionality to deconstruct BLOBs into something of more reasonable size before reading it with SAS.

 

Alternatively get Sybase to write this data out to a flat file, then use SAS DATA step processing to read the data in 32K chunks and decipher it on the fly into SAS variables.

Super User
Posts: 11,343

Re: reading a large data blob from sybase

Posted in reply to wkossack_nspirehealth_com

I would also ask some veriation on "what is that thing?" If it is an image, audio file, executable  or similar object then it may not be at all useable split in any way SAS would like it.

Super User
Posts: 5,441

Re: reading a large data blob from sybase

Posted in reply to wkossack_nspirehealth_com

Aggree with @ballardw.

What is your requirement? Chances are that you can't find any useful information there, especially if you don't know at this point what is is...

Data never sleeps
Ask a Question
Discussion stats
  • 3 replies
  • 248 views
  • 0 likes
  • 4 in conversation