On April 22, 2017, SAS learned that NO version of RHEL is supported on the AWS EC2 I3 instance. This has to do with the lack of drivers for the NVMe SSD drives and the new Elastic Network Adapters. Please note Red Hat is aggressively working on resolving this issue, but in the meantime SAS strongly encourages SAS 9.4 customers to use the EC2 I2 instance.
We understand that these are older instances, but they are still available from AWS via this previous generation instances link.
For maximum IO throughput, we strongly recommend that you use the i2.8xlarge instance for your SAS compute node(s). This instance has 16 physical cores (32 vCPUs), 244 GB of RAM, and 8 – 800GB internal SSD drives (these drives are ephemeral and all data on them will be erased during a reboot/restart of your instance) and a single 10 GB NIC card. Amazon configured several storage scenarios and gathered IO throughput statistics on them. Please see Appendix A for the details.
We suggest that you stripe the 8 – 800GB internal SSD drives to get the best IO throughput for your SAS WORK and UTILLOC file systems. Please note with striped SSD drives, you can leave UTILLOC in its default location, which is SAS WORK.
We also suggest that the RHEL administrator reads the Optimizing SAS on RHEL paper and apply the tuning recommendations in the paper.
Permanent SAS data, SAS binaries, SAS configuration files, home directories, etc. should be housed on EBS storage for your single SAS compute server instance or in Lustre for your SAS Grid infrastructures (see this paper for how to configure SAS Grid on the Amazon cloud).
As a reminder, it is advisable for all instances used by SAS (metadata tiers, mid-tier, compute tier, and external database tier) to be located in the same Amazon Region and Availability Zone.
Appendix A
To help with getting the best IO throughput, Amazon set up four different storage configuration with an i2.8xlarge instance type running RHEL-7.3_HVM_GA-20161026-x86_64-1-Hourly2-GP2 (ami-b63769a1) and got the following IO throughput numbers using the rhel_iotest.sh test from SAS:
Configuration 1:
Ephemeral SSD (local)
EBS
Configuration 2:
EBS
Configuration 3:
Ephemeral SSD (local)
Configuration 4:
Ephemeral SSD (local)
EBS
Very valuable insight @MargaretC. Thank you very much!
I have been informed by Amazon that we should not have an issue with AWS EC2 instances, so you should ignore this statement "You will need to do a one year reservation of your I2 instance so that you will be assured of having one if you need to restart/reboot your instance. Since these are previous generation instances, there are not a lot of them in the pool and because of this, you may be downsized to a smaller I2 instance after a reboot/restart." from above.
Per @MargaretC's comment, I edited the article to remove the statement about Amazon instance availability "limitation"
is it safe to assume in the Apendix that the ST1 drives were always configured in RAID0?
Very interesting that the size of the ST1 drives didn't seem to alter the per core throughput performance.
One more question. Have you considered using LVM (With striping) vs using RAID 0? We currently use LVM (with striping) for the 8 Instance Drives on our i2.8xlarge, and I was wondering if there are any advantages to RAID 0 using a tool like mdadm.
I ran rhel_iotest.sh on an i3.16xlarge.
8 Ephemeral Drives Striped using LVM as /saswork
RESULTS
-------
INVOCATION: rhel_iotest -t /saswork
TARGET DETAILS
directory: /saswork
df -k: /dev/mapper/saswork-root 14841653248 33840 14841619408 1% /saswork
mount point: /dev/mapper/saswork-root on /saswork type xfs (rw,relatime,seclabel,attr2,inode64,logbsize=256k,sunit=512,swidth=4096,noquota)
filesize: 390.08 gigabytes
STATISTICS
read throughput rate: 384.15 megabytes/second per physical core
write throughput rate: 138.46 megabytes/second per physical core
-----------------------------
********* ALL ERRORS & WARNINGS *********
<<WARNING>> insufficient free space in [/saswork] for FULL test.
<<WARNING>> - smaller stack size and # of blocks will be used.
*****************************************
**Note: Not sure why there is an Error message about space, the /saswork drive started the test with 14TB of space free. Perhaps the amount of freespace overflowed some sort of integer value in the script.
----------------------------------------------------------------------------------------------
3 X 12.5TB Drives in LVM Stripe for /users
RESULTS
-------
INVOCATION: rhel_iotest -t /users
TARGET DETAILS
directory: /users
df -k: /dev/mapper/users-root 40263219200 34032 40263185168 1% /users
mount point: /dev/mapper/users-root on /users type xfs (rw,relatime,seclabel,attr2,inode64,logbsize=256k,sunit=512,swidth=1536,noquota)
filesize: 480.09 gigabytes
STATISTICS
read throughput rate: 39.68 megabytes/second per physical core
write throughput rate: 49.73 megabytes/second per physical core
Support for AWS i3s is now available with RHEL 7.4 (https://access.redhat.com/articles/3135091)
David
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.