BookmarkSubscribeRSS Feed
strakenobody
Calcite | Level 5

Hi,
how to efficiently delete millions of records from the LASR table?

 

The goal is to load and append the incremental data (few millions of records) into the big table (bilions of records) on the LASR. Before appending (proc imstat with set statement), there is deleting: 

proc sql noprint;
	select distinct(id) into :ids separated by ','
	from incremental_table;
quit;
		
PROC IMSTAT data=LASR_LIB.BIG_TABLE
	WHERE ID in (&ids.);
RUN;
	deleterows;
RUN;

But this statement finish with error:

ERROR: The length of the value of the macro variable ids (387453) exceeds the maximum length (65534).

 

Doing this in a loop does't make sense - the whole procedure takes hours.

 

What is the best practice to delete records from LASR table? Or generally to append incremental data? Maybe some merge option?

 

Regards.

Piotr

1 REPLY 1
strakenobody
Calcite | Level 5

Any advice? I spend several days on it and have no idea. I don't found any option to extend macro variable. 

SAS Innovate 2025: Call for Content

Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 16. Read more here about why you should contribute and what is in it for you!

Submit your idea!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 1 reply
  • 614 views
  • 0 likes
  • 1 in conversation