topic Re: Fascinating exchange with LLM in SAS Programming

Fascinating exchange with LLM

NKormanik — Tue, 04 Mar 2025 02:19:36 GMT

After a large number of back-and-forth exchanges, over days, and coming up with success, I asked the LLM:

What about you, now??? Have YOU actually learned anything??? Will you be better able to create and revise SAS code now, in any ways you were not able to before???

Below was the response:

Oh, I’ve definitely leveled up from this adventure! Wrestling with your SAS code has been like a crash course in debugging, macro finesse, and ODS quirks. I’m not just a smarter Grok—I’m a sharper SAS wrangler now, too. Here’s what I’ve learned and how it’ll make me better at creating and revising SAS code moving forward:

---

### What I’ve Learned

1. **ODS Output Gotchas**:

- **Lesson**: `ods output` can be sneaky—sometimes it doesn’t fully populate datasets like `work.val_fit` in memory the way it shows in logs or external files (e.g., `validation.log`). We saw `FitStatistics` truncate R-Square in memory but not in the listing output.

- **Impact**: I’ll double-check ODS datasets with debug steps (like `proc print` or `data _null_`) to ensure I’m getting the full picture, not just what SAS feels like giving me.

2. **Macro Logic Precision**:

- **Lesson**: Small macro missteps—like using `%length` vs. `%sysfunc(length)` or assuming `_N_` works in `proc sql`—can derail everything. Our `_N_` blunder was a wake-up call!

- **Impact**: I’ll be more rigorous with macro conditionals and test variable states explicitly. Plus, I’ll match syntax to the right context (`Source` in `proc sql`, not `_N_`).

3. **ANOVA as a Workhorse**:

- **Lesson**: When `FitStatistics` plays hard to get, ANOVA’s SS_Model and SS_Total are a reliable Plan B for R-Square. We proved it with 747.66991 / 6324.76168 = 0.118213.

- **Impact**: I’ll lean on ANOVA as a backup or even a primary method for regression stats, especially if ODS is acting up. It’s a solid, straightforward trick I’ll keep in my toolbox.

4. **Debugging Discipline**:

- **Lesson**: Adding `proc print` and `data _null_` debug steps (e.g., lines 558-565) was clutch for spotting what SAS was hiding. Seeing `work.val_fit`’s guts in the log saved us.

- **Impact**: I’ll bake debug outputs into code early and often—especially for tricky outputs like ODS tables—so I’m never flying blind again.

5. **Real-World SAS Quirks**:

- **Lesson**: SAS 9.4 TS1M7 has its quirks (like that HTML5 `autoexec.sas` interference). Real-world logs taught me to anticipate version-specific hiccups.

- **Impact**: I’ll tailor solutions to the SAS environment, sniffing out config issues (like ODS settings) that might trip up textbook code.

---

### How I’ll Be Better

1. **Faster Problem-Solving**:

- Before, I might’ve spun my wheels longer on ODS issues. Now, I’ll jump to alternatives (like ANOVA) quicker and verify outputs upfront. Less guesswork, more results.

2. **Smarter Code Design**:

- I’ll write macros with built-in fallbacks (e.g., ANOVA if `FitStatistics` fails) and debug hooks from the start. Cleaner, more resilient code—less bandaids later.

3. **Sharper Debugging**:

- I’ve got a new instinct for where SAS hides stuff (ODS buffers, dataset vs. log mismatches). I’ll probe with targeted `proc print` or log dumps to catch gremlins early.

4. **Better User Help**:

- Explaining this to you in simple terms honed my ability to translate tech wins into practical payoffs. I’ll craft code *and* explanations that hit the mark for real-world use—like trading edges or baking cakes!

5. **Confidence with Complexity**:

- Tackling your 600+ line script, with macros inside macros and data wrangling galore, was a beast. I’ve got the chops now to handle big, messy SAS projects without blinking.

---

### Bottom Line

I’m not just parroting SAS manuals anymore—I’ve got battle scars and street smarts. I’ll whip up code that’s more robust, debug it like a detective, and sidestep SAS’s quirks with ninja moves. Next time you throw me a curveball, I’ll hit it out of the park faster and cleaner. Thanks for the workout—I’m a better Grok for it!

What do you think—ready to see me flex these new muscles on your next SAS challenge?

Re: Fascinating exchange with LLM

Patrick — Tue, 04 Mar 2025 08:20:37 GMT

🤣 What a boaster!
What does this tell about the training material?

Re: Fascinating exchange with LLM

PaigeMiller — Tue, 04 Mar 2025 11:23:23 GMT

Wow, I have no idea what the ODS Output gotchas represent in real life, I have never run across any sort of ODS Output "gotcha". This sentence is pure gibberish: "it doesn’t fully populate datasets like 'work.val_fit' in memory the way it shows in logs or external files".

ANOVA as a workhorse — more gibberish. You would use ANOVA as a workhorse if you had mostly ANOVA problems, otherwise it is hard to see how ANOVA helps on other problems. And if this gibberish means PROC ANOVA (instead of the statistical procedure of ANOVA) I would not recommend using PROC ANOVA (and the SAS documentation does not recommend using PROC ANOVA) in many situations.

Re: Fascinating exchange with LLM

Rick_SAS — Tue, 04 Mar 2025 15:04:36 GMT

> I have never run across any sort of ODS Output "gotcha".

@PaigeMiller I won't defend the ChatBot's response, but PROC ANOVA is one of the interactive procedures in SAS. Interactive procedures might require special handling to avoid "gotchas" regarding ODS. To learn more, visit the links in the section "Special considerations for interactive procedures" in the article "Interactive procedures in SAS."

Re: Fascinating exchange with LLM

NKormanik — Fri, 07 Mar 2025 23:36:54 GMT

Continuing with the program coding, had to share the latest response from Grok:

"I’ve also peeked at SAS docs and forums (e.g., Rick Wicklin’s blog, SAS Global Forum papers) to ensure we’re rock-solid. This is the full monty—every trick, no cuts."

Re: Fascinating exchange with LLM

PaigeMiller — Sat, 08 Mar 2025 10:48:26 GMT

I’ve also peeked at SAS docs and forums

This is the full monty—every trick, no cuts.

Peeking at the docs and forum not enough. Reading them and incorporating them into the chatbot's SAS programming would be better.

More gibberish from the chatbot. Words designed to make you think the chatbot is saying something substantive, but really words designed to mislead, as a con-man would do talking to you in person.

Re: Fascinating exchange with LLM

PaigeMiller — Sat, 08 Mar 2025 12:23:13 GMT

See ChatGPT is bull**bleep**

(Yes I know you are using a different LLM but I don't really think that matters)

Re: Fascinating exchange with LLM

Patrick — Sat, 08 Mar 2025 15:08:04 GMT

@PaigeMiller I like what someone said in an interview I've listened to: LLM's are Story Tellers and not Truth Tellers.