I don't understand what you mean when you say "when I open the file with .doc" versus "open with RTF".
.DOC and .RTF are both file extenstions -- for example, I can open a .DOC file created with Word on Windows on my daughter's Macintosh computer, using Word. I can also open a .RTF file created with SAS on my daughter's Mac, using ClarisWorks or Pages. I can open an .RTF file with OpenOffice.
.DOC is not a word processing application. Word processing applications are applications like Microsoft Word, Corell WordPerfect, NisusWriter, Claris Works, StarOffice, OpenOffice.
If you use ODS in this fashion:
ods rtf file='somefile.doc';
ods rtf file='somefile.rtf';
Then you are "fooling" your word processor software with the first example -- calling an RTF file with a reserved Microsoft Word file extension like .DOC or .DOCX might be OK in Microsoft Word, but might cause issues in other word processor applications that are looking for Microsoft specific .DOC controls in the file. If you use the second invocation method and use a file extension of .RTF, on the other hand, any word processing application that knows how to interpret RTF control strings could, within the limits of how the application works, render those RTF control strings correctly in the word processing application.
The underlying premise of an RTF file is that you will probably edit or change it in the word processing application. I would suggest that 1) you use RTF as the correct file extension with ODS RTF and 2) open the file in your word processing application of choice and 3) adjust the margins in the word processing application until the cells no longer wrap and then 4) use a SAS options statement to apply those margins to your SAS job so the table will be formatted using those margins.
Still not enough information. What application is opening when you double click? Word? If so, there should be no difference between how the file is rendered in Word whether you double click or whether you do a File--> Open from inside Word.
In some earlier versions of SAS and RTF, (like back in Version 8), I remember that there was an issue where if you already had Word opened and you have modified the Word template normal.dot, it might have been possible for the existing Word template margins to be used with your SAS RTF file -- but that was long, long ago and I haven't seen that behavior anymore.
On what operating system are you double clicking??? On Unix? If so, what application is launching? I can't think of any solution other than explicitly setting the margins for RTF in a SAS OPTIONS statement. Otherwise, you might wish to open a track with Tech Support to see whether they have any additional insights.