In this topic we will deal with removing special characters
1- Tranwrd
tranwrd(column, "?", " ")
input: "Hello?World"
output: "Hello World"
Note: the result is always 200 byte, so keep using length=<length>
good for: replacing or removing single character
document: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000215027.htm
2- Translate
Translate(column, "`~!@#$%^&*()-_=+\|[]{};:',.<>?/")
input: "Hel)lo?Wo.r*ld"
output: "HelloWorld"
Note: try to replace ? which used as "space" using tranwrd then use translate
good for:
1- removing special characters if spaces are exist
2- It also have some character option for lower, upper, and letters,...documentation: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000215153.htm
3- Compress and Kcompress
compress(column, "c3b9"x)
compress(column,"`~!@#$%^&*()-_=+\|[]{};:',.<>?/")
c3b9 is the hexadecimal of ù
x is used for hexadecimal representation
input: "Hello World�"
output: "Hello World"
Note: use x if you need hex repr cause chars like � has different hex representation
take the string which has this special character to text to hex converted and use it's hex to remove it
good for:
1- removing hard special characters
2- It also have some modifiers in documentation which help a lot
Documentation: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000212246.htm
4- kpropdata(COLUMNA ,'hex', 'utf-8') followed by Tranwrd #1
kpropdata(COLUMN ,'hex', 'utf-8')
if you have this char "�" and followed compress and kcompress hex and normal text and not works
you should consider kpropdata which Removes or converts unprintable characters
also if you find hex "efbfbd" which refers to corrupted string
the only way which works with me was kpropdata followed by tranwrd
