Your SAS programs, embedded in web apps and elsewhere

How to deal with missing data (" .")

Reply
N/A
Posts: 0

How to deal with missing data (" .")

In Proc SQL, I need to exclude data rows that have a certain field showing as " ." In other words, I want to exclde data with missing a missing value in a certain field. In the WHERE clause of my sql, what do I code to exclude these rows with missing data?

I tried using <> . - no good. I tried using NOT IS MISSING - no good.

If someone would be willing to help me out, I'd be grateful. Thanks in advance!
N/A
Posts: 0

Re: How to deal with missing data (" .")

NOT is NULL
Frequent Contributor
Posts: 91

Re: How to deal with missing data (" .")

Is that actually a character variable with a blank and a period or a character variable with just a period in it? Really it doesn't make any difference the period is used to indicate missing data for numeric data. A character variable with a period is not a missing value, a character variable that is blank can be (is) considered missing.

So, if it is a character variable, you would exclude it as

where var ne " ."
or
where var ^= " ."
N/A
Posts: 0

Re: How to deal with missing data (" .")

Peter and advoss, thanks for your replies. The data is numeric (actually, money amounts, and so I will try the NOT IS NULL angle.

Again, thanks!
SAS Super FREQ
Posts: 8,743

Re: How to deal with missing data (" .")

Hi:
This document "Potential Result Set Differences between Relational DBMSs and the SAS System" may help you understand exactly what you're dealing with. Particularly the section entitled: "RDBMS Null Values Versus SAS Missing Values". Here's the link:
http://support.sas.com/resources/papers/resultsets.pdf

The syntax, IS MISSING, IS NOT MISSING, IS NULL, IS NOT NULL, will work with SAS data sets and will return the same results. The nice thing about this syntax is that you don't have to worry about the type of the variable in order to code the expression. The other way to code for missing or not missing is:
[pre]
where numvar = .;
where numvar ne .;
where charvar = ' ';
where charvar ne ' ';
[/pre]

It would be an error to try this:
[pre]
where numvar = '.';
[/pre]
because SQL wants NUMVAR to be of the same type as what it is being compared to. If you try to code as shown above, you will get this error message (or one like it):
[pre]
ERROR: Expression using equals (=) has components that are of different data types.
[/pre]

This is how things work with SAS data sets. As the above paper explains, if you are accessing data in an RDBMS or another type of data source (not SAS), then NULL in the RDBMS might not be the same as MISSING or NULL to a SAS data set.

cynthia
N/A
Posts: 0

Re: How to deal with missing data (" .")

And I tip my hat to you! Thank you so much for your insight!
Ask a Question
Discussion stats
  • 5 replies
  • 345 views
  • 0 likes
  • 3 in conversation