I have this EG project where if I try to save my final outcome dataset in a permanent SAS library, the results disappear. If I have the process flow create the dataset in the work library the results are correct and the data appears fine. Does anyone know what could be the cause of this?
Check your SAS log. It will tell you if there is a technical error.
I've never had a SAS dataset 'disappear', but I've certainly had student's 'misplace' them, usually because the libname points to a place they didn't realize they had set it for. Or they have set the permanent libname to point to the same physical directory as WORK (so that when SAS exits, it is erases it). Or they run out of disk space and ignore the error message.
The dataset doesn't disappear, just the data itself. For example, when i go to output the dataset to the work library there are 50 rows and 20 columns. When I change the library for it to save into to a permanent library, it ouputs 50 rows and 0 columns (just row numbers).
Now I have gone on to our metadata server and went to open up the permanent dataset that i attempted to create. I opened it via base SAS, and the dataset opened fine and appeared as it should. So even though there is no output in EG, the dataset is being created fine. However, when I go to use this dataset in Information Map Studio (or view it in Mgmt Console) it does not recognize any fields. Herein lies the problem.
Have you tried registering the metadata via the Source Designer in DI Studio. I am no expert but was having the same problem with permanent data sets created in EG that could not be viewed elsewhere. This resolved the problem for me.
I assume that there is some confusion between the metadata definitions and the physical dataset/library path.
...metadata server and went to open up the permanent dataset that i attempted to create. I opened it via base SAS, and the dataset opened fine..."
Does that mean you've run a base sas session on the physical box where the sas metadata server runs as well?
Because: This way you accessed the physical sas file directly without using the metadata definitions. It doesn't matter that the metadata server is also running on this box.
I assume that in some runs you use a libname statement (without meta) to access the data and in other cases you try to access the data via metadata definitions.
Is the metadata definition for the library pre-assigned or not? Or how do you assign the library?
I would first check the metadata definitions:
Is the table you want to write to defined correctly (all the fields mapped)?
Is the table part of the library you use?
Does the library point to the expected physical location?
It could be that you write your data to one location (using a libname statement without meta keyword) but that your looking somewhere else for this file when you're using the metadata definitions.
I don't yet understand how the EG log can tell you that it wrote 0 columns and there are still all the data written - are you sure you're not looking at old data?