BookmarkSubscribeRSS Feed
vkgaurav_org
Calcite | Level 5

Hi

I loading a data from csv to SAS using SAS DI. Initially table contains the 16 variables, but now suddenly source contains 17 variables so I have added the same variable to source and target properties and updated the same. It is perfectly showing in DI but in EG table it is showing only 16 variables, it is not updated in EG. Now I have to add the new variable at 8th column position. Anybody have any idea How can we do that. Below is the example:

Old table Structure:

ID Name Order_NO unit_price

 

New Table structure:

ID Name Order_ID(New _column) Order_No unit_price

 

 

Any advise and help is appericiated....

 

Thanks

Vishwajeet

3 REPLIES 3
vkgaurav_org
Calcite | Level 5

I would like to add one morething, the data already contains approx five billion record.

RW9
Diamond | Level 26 RW9
Diamond | Level 26

Do you have an import agreement in place?  If not then that is your problem.  The ideal process is to first, before any data is sent, agree with the vendor what the file format will be, what it will contain.  Write this down in a document and have both parties sign this.  This forms the basis of what they should send you, and what you should receive.  If the data received is not per that agreement, you go back to vendor and ask them to provide another file.  As with anything programming -> Define, document, test, proceed to production.

 

As for your problem, its likely that on the first import your dataset was setup, and there is data there.  Now it doesn't want to change that structure.  You would need to (and I don't use DI, so don't know) change the underlying dataset, either through code, or in your table manager so that column is present, then reimport the data.  Again, this is where that agreement is vital as changes to the data require changes to the database.

LinusH
Tourmaline | Level 20

Even if @RW9 is not a DI Studio user, he's on to it.

DI Studio is a metadata specification tool, so it lacks a bit of the data management parts, like automatically create alter table statements.

So, you add columns in metadata, create mappings from the source.

If your target table have's a some kind of update strategy in the table loader, chances are that you will receive an error when redploying your job code.

So you need to add new/changed columns to the physical table in this scenario. If you are working with SAS tables/data sets - I'm sorry, but a rewrite of all data will occur.

 

If you are/or want to become a mature data warehouse shop, you need to implement these kind of actions in your production/maintenance process.

Data never sleeps

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 2128 views
  • 0 likes
  • 3 in conversation