I'm not 100% clear on what you're after, but this PROC MEANS run shows how to do that kind of summarization. Hopefully it will get you on track!   Tom   data have; length Material $10 Plant 8 Total 8 Posted $10 Week 8; input Material Plant Total Posted Week; cards; N384000 2001 100 1/1 1 N384000 2001 200 1/5 1 N384000 2001 300 1/3 1 N384000 2002 100 1/4 1 N384000 2003 150 1/4 1 N384000 2001 900 1/10 2 N384000 2002 50 1/11 2 N384000 2002 0 1/10 2 N384000 2003 75 1/10 2 N384000 2004 0 1/13 2 N463375 2001 100 1/1 1 N463375 2002 200 1/2 1 N463375 2003 100 1/4 1 N463375 2004 50 1/4 1 N463375 2004 75 1/6 1 N463375 2001 0 1/11 2 N463375 2002 150 1/11 2 N463375 2003 200 1/10 2 N463375 2003 200 1/10 2 N463375 2004 200 1/13 2 run; proc means data=have noprint nway; class Material Plant Week; var Total; output out=want sum()=TotalSum; run; ... View more

Here's a tutorial (about 15 minutes) that shows how to start with the Import Data task, create a code template, modify it using the INFILE with a wildcard spec to pull multiple files, and keep track of which records come from which files.   ... View more

There should be an option in the Sort Task to remove duicates as well as a DISTINCT option in query builder to return only unique rows.  It’s not exactly clear what your rules are for removing records so I suspect these may not do what you want. They will cover the case shown. So, if you can’t figure it out it would help if you expand your example to be closer to your actual data.  @maliksmom2000 wrote: I have about 500 columns and 150M records.  These are inspections and about half are duplicate rows for the same "unit" that has been inspected.  There may be up to five or six records for some units and they have different data that needs to be kept.  However there are many others that have 20 copies of the same inspections and only two of the records are different.  How do I do a query builder to eliminate all but unique records? Unit ID will always be the same Inspection date, record number, etc. may be different between two records for the same unit.  I need to keep both but I can't have 200M records for 6.4M units.  I would assume I'd have upwards of 30M with the different inspection records for each unit.  Example: Unit ID            Inspection Date             Finding Code 1                            2/14                                  A 1                            2/14                                  A 1                            2/15                                  B  1                            2/15                                  A   I would need to remove the second record from 2/14.  I want distinct rows only but I need to compare all of the columns not just one for each record.  If that makes sense     Help!   ... View more

When code goes wrong it often helps to show 1) the actual result from the code and 2) the expected result when you have one.   If the result is as @Astounding suggests it would likely show up pretty easily especially when compared to an expected value. In fact it would likely show if only 3 or 4 variables were concatenated. ... View more

 How did you do the update, in the file, replacing the file in EG, is it an import transformation or an user written ? 🙂   //Fredrik ... View more