BookmarkSubscribeRSS Feed
LDYL
Calcite | Level 5

I am currently struggling with a space optimisation problem. I can not compress data, because a star schema will be applied on them, so that I am trying to reduce the size "manually". I figured out that a lot of space is lost by storing small numbers on 8 bytes. 

 

However I can not reduce the length of a numeric variable stored on LASR server. For example: even when I upload a table with numeric variables lengths set to 3. all lengths are automatically set to 8. on the server. 

How can I determine this length? Is that a server configuration issue? 

 

Any help will be appreciated. 

4 REPLIES 4
alexal
SAS Employee

@LDYL,

 

Each column in the LASR server is padded to an 8-byte boundary. If the variable you are adding is not a multiple of 8 bytes, the variable will be padded to an 8-byte boundary.

 

Read this: https://en.wikipedia.org/wiki/Data_structure_alignment

LDYL
Calcite | Level 5

Thank You @alexal 

 

could You recommend me any possible solution how to reduce the size of dataset on LASR,

however with the star schema being applied ? 

Can I somehow combine compression with star schema? 

alexal
SAS Employee

@LDYL,

how to reduce the size of dataset on LASR, however with the star schema being applied ?

There is no way.

Can I somehow combine compression with star schema?

No.

SASKiwi
PROC Star

One strategy I've successfully used to reduce space is to make use of codes in character variables which are expanded to full descriptions using SAS formats (eg store USA in a character variable and have a SAS format which applies United States of America).

 

Also if you want to compress numeric variables that are just codes that aren't used in calculations, convert them to character.

 

Please note that even when you are able to compress LASR tables, in my experience the performance degradation is significant.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Tips for filtering data sources in SAS Visual Analytics

See how to use one filter for multiple data sources by mapping your data from SAS’ Alexandria McCall.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 793 views
  • 1 like
  • 3 in conversation