cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
sas_novice2
‎10-04-2023
  • 3 Posts
  • 0 Likes Given
  • 0 Solutions
  • 0 Likes Received

Data step where any of the variables has one of a list of values

by Calcite | Level 5 in SAS Programming
‎09-27-2023 11:46 PM
‎09-27-2023 11:46 PM
I'd like to write a where statement as follows:   data test; set data1-data60 where (var1 or var2 or var3 or var4) in (1,2,3,4)   essentially, what I'm trying to accomplish is to stack data1, ...., data60 and keep rows that has at least one of var1, var2, var3, var4 to have values equal to 1, 2, 3 and/or 4. Is that the correct syntax, and if not, what would the correct one be?   Thanks! ... View more

Re: Data Processing with PROC SQL for Multiple Billion Rows

by Calcite | Level 5 in SAS Programming
‎09-25-2023 03:08 PM
‎09-25-2023 03:08 PM
Thank you for the response! Wouldn't filtering the dataset one by one for all 60 of them result in at least the same time spent than filtering them altogether? Filtering all the datasets together in a single step would require the filter to only traverse once, as opposed to traverse 60 times; although the same amount of rows explored - but have to re-type the filtering code for 60 times?   Can I use that DATA step in conjunction with PROC SQL? So after your code, using PROC SQL to filter the rows?   ... View more

Data Processing with PROC SQL for Multiple Billion Rows

by Calcite | Level 5 in SAS Programming
‎09-25-2023 02:54 PM
‎09-25-2023 02:54 PM
I have 60 sas7bdat datasets, around 100 million rows each and 60-70 columns, which contains quite similar columns and would like to process them altogether to filter and create a smaller dataset (containing all the columns) containing rows with specific information with the following code:   PROC SQL;     CREATE TABLE bigtable AS     SELECT * FROM DATA1     OUTER UNION CORR     SELECT * FROM DATA2     .......     OUTER UNION CORR     SELECT * FROM DATA60;       CREATE TABLE filtered AS     SELECT * FROM bigtable     WHERE <filtering based on 6 columns in the 'bigtable' with AND & OR>;       <and then save 'filtered' in CSV>;       Two questions relating this process: DATA1, ..., DATA60 are sas7bdat datasets, but I get the following messages: "Data file is in a format that is native to another host". This may cause processing time to be much slower - datasets are in sas7bdat and in latin1 western (ISO) - any solutions, and why the data is not native formatted even when they're sas7bdat datasets? Is there a much faster way to process data via PROC SQL, to vertically combine and filter billions of rows? Thank you in advance!       ... View more
Contact Me
Online Status
Offline
Date Last Visited
‎10-04-2023 03:50 PM
Quick links