BookmarkSubscribeRSS Feed
MargaretC
SAS Employee

SAS has worked for several years with the Microsoft Azure Team to test Azure Managed Lustre file system (AMLFS).  We worked with them to incorporate SAS IO workload patterns and performance requirements into their testing.  This has helped ensure the initial offering of AMLFS runs as optimally as possible for our data-intensive customers. 

 

SAS is excited that AMLFS is now available for general use with both SAS 9 and Viya 4.

 

Please note, that during the in-house testing and working with pre-GA customers, MS Azure has determined the following precepts should be followed for optimal performance:

 

  1. SAS Client machines must be placed in the same Availability Zone in which the AMLFS cluster resides. This helps keep the clients and storage as physically close as possible, reducing latency.
  2. Accelerated Networking on all client VMs must be enabled.

 

The above information is documented here: https://learn.microsoft.com/en-us/azure/azure-managed-lustre/connect-clients#client-prerequisites

 

In addition to the above, during SAS workload testing, the Microsoft Azure Team discovered that running the following tuning command set on each client helped optimize I/O:

 

lctl set_param mdc.*.max_rpcs_in_flight=128 osc.*.max_pages_per_rpc=256 osc.*.max_rpcs_in_flight=16 osc.*.max_dirty_mb=1024 llite.*.max_read_ahead_mb=2048 osc.*.checksums=0  llite.*.max_read_ahead_per_file_mb=256 

                

MS Azure also found modifying striping for large files was beneficial:

 

                lfs setstripe -c 3 -S 16M /home/saslustre/lustremnt/

 

The above lctl and lfs commands are not always required to obtain the best performance, but test results seemed more consistent with the above set of tunings in place.

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

CLI in SAS Viya

Learn how to install the SAS Viya CLI and a few commands you may find useful in this video by SAS’ Darrell Barton.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 0 replies
  • 1759 views
  • 4 likes
  • 1 in conversation