Piers Tracker

Undirected Network Models - The Gaussian graphical model (partial correlation networks)

Piers — Tue, 26 Nov 2024 19:38:27 GMT

Dear all

I know that the above mentioned network models can be computed in R, with which I am not familiar.

Does anyone know is the same kinds of network models can be computed in SAS, please?

If so, could you kindly point me in the right direction

Best

Piers

Combined within and between subjects multivariate multiple regression - can a single model suffice?

Piers — Thu, 18 Apr 2024 11:09:55 GMT

Hi all

I have two datasets addressing essentially the same question about ideal body shape and body composition.

In the first, between subjects dataset (BETWEEN), participants are assigned to one of three experimental groups: personal ideal body shape (PERS), cultural ideal body shape (CULT), and most attractive body shape (ATT). In each group, participants judge images of body composition that fit the requirements of that condition. Then, the average muscle and adipose are calculated from their decisions. We also gather information about 2 covariates: the age and actual body mass index (BMI) of the participant.

In the second, within subjects dataset (WITHIN), each participant makes judgements about all three conditions: personal ideal body shape (PERS), cultural ideal body shape (CULT), and most attractive body shape (ATT). The average body composition for the bodies in each of the three conditions is computed for each participant, and the same covariates, age and BMI obtained.

It is relatively straight forward to carry out appropriately structured MANCOVA style analyses using PROC GLM, separately for the BETWEEN and WITHIN datasets. But this means that I cant directly assess the potential effect of sample type - i.e. WITHIN versus BETWEEN. There are reasons to do with psychological biases that I would like to estimate this.

So, does anyone know if I can build a single model for the multivariate output (each decision about body composition renders both an adipose a skeletal muscle value), which incorporates sample type as a fixed effect, along with condition (PERS, CULT, and ATT), and the two covariates. I imagine that at least one difficult bit is structuring the random effects correctly.

Bottom line, I would be extremely grateful to anyone who can provide expert advice on this problem.

Many thanks in advance

Piers

Re: Converting large simulation into a loop, to avoid large datasets and running out of memory

Piers — Fri, 30 Apr 2021 07:22:53 GMT

Very many thanks.

I found that once I could access a stable disc space area, this simulation works fine using data steps and by statements only - its also quite efficient, even though generating datasets with 3,255,000,000 rows to be fed into the regression model. This only takes ~30mins

Re: Converting large simulation into a loop, to avoid large datasets and running out of memory

Piers — Tue, 27 Apr 2021 13:46:10 GMT

I have been reading Rick Wicklin's advice, and he is all for avoiding macros, preferring if possible data steps with BY statements. So I have tried to revert to this approach.

Re: Need more disc space for a simulation

Piers — Tue, 27 Apr 2021 12:32:16 GMT

I could try drive G: which also has more space than C:

Drive F: has been assigned to be my google drive and I has had odd problems with read/write access for SAS using this drve before, for reasons that are very opaque

Re: Converting large simulation into a loop, to avoid large datasets and running out of memory

Piers — Tue, 27 Apr 2021 05:44:17 GMT

Hi Reeza

Very many thanks for this. But as a complete newbie to these kinds of approaches, I am struggling to find what I assume is a bug.

I have attached a modified version of my original script with only 10 replications per matrix, and only 10 datapoints per iteration of proc simnorm. It includes your code as well, but runs much more quickly as the dataset is so much smaller.

I succeed apparently in creating both the Macro_param_list and the Run_macros steps if I keep the call execute(str); commented out. I show a short screen shot of the latter:

However, when I include call execute(str); by uncommenting it, I get the following sets of errors. First:

So, somehow the rep part is problematic - or the reference to it, although it does get to the end of the list, where 6510 observations are read, which is correct.

Then I get:

It is very clear that the regressions are being run, but the datasets created for each regression are empty. Also, the filenames for each data set only contain REP_1, REP_2 etc. The Filenames do not contain information about the MTX.

If you have time to take a second look I would be very grateful. After about 3 hours or so, I just cant spot what the error is.

Best

Piers

Converting large simulation into a loop, to avoid large datasets and running out of memory

Piers — Mon, 26 Apr 2021 16:04:20 GMT

Hello - again

Attached is a script where I:

1) generate all conceivable covariance matrices for a regression model A = B + C + B*C

2) strip out any matrix that does not have a positive determinant

3) re-configure the data into Type = COV form to submit to PROC SIMNORM

4) run the regressions on the dataset created by PROC SIMNORM

The reason for this is that I would like to know if there are parts of the covariance space which might give rise to significant (p < .05) interaction terms, even though the data are derived from normal distributions

Experienced programmers will be giggling by now because they will have predicted that I am generating huge datasets, and that PROC REG runs out of memory in the attempt to sore the output as a file.

Therefore, I am seeking help as to how to recode this script so that the very large data sets can be avoided (ideally I would like to have 10k duplications of each of the 651 covariance matrices, and to run each rgression on 10k data points), and proc reg does not run out of memory.

Any help would be very gratefully received.

Piers C

Re: Need more disc space for a simulation

Piers — Mon, 26 Apr 2021 13:13:36 GMT

Goodness me - so straightforward

I just tested the first step in my script, and the first data set was stored in the new location just fine (I renamed the library to be called PLCTEMP).

However, in the explorer view, I notice that if I try to renew or delete the datafile it wont let me.

It gives the following:

ERROR: A lock is not available for PLCTEMP.RAW.DATA

Presumably this has to do with write/read permission.

Is this solvable?

Best, and very many thanks so far

Piers

Need more disc space for a simulation

Piers — Mon, 26 Apr 2021 12:08:01 GMT

Hello,

I am running a numerical simulation with PROC SIMNORM which is trying to create a large data set that has 6 columns as output. The input is a dataset with 26,040,000 rows.

PROC SIMNORM appears to be handling it well until it runs out of work space. So, I would like to know how to redirect SAS where it writes the temporary work files.

Basically, I have 1.4Tb available on another disc (F: drive), and would like to try to use this instead of the default which is on the C: drive.

I do have admin rights.

Help very gratefully received

Piers C

Re: Trouble creating Type = COV dataset

Piers — Sun, 25 Apr 2021 10:05:21 GMT

Wow - very many thanks. The reason for doing this is to try and discover under what covariance conditions the regression model A = B + C + B*C gives a potentially significant interaction terms even though A, B, and C are derived from multivariate normal distributions. You have really been astonishingly helpful

Best

Piers

Re: Trouble creating Type = COV dataset

Piers — Sat, 24 Apr 2021 16:59:59 GMT

Many thanks for this. Its enormously helpful.

I am not a mathematician, and therefore need to understand more about matrix determinants. So, if you have time:

1) Could you point me in the direction of a sensible reference which lays out what the important issues are

2) Do you have SAS code which allowed you to calculate the determinant from the data structure I have, please?

Again, many thanks

Piers

Trouble creating Type = COV dataset

Piers — Sat, 24 Apr 2021 12:50:11 GMT

Hi.

I would really appreciate some help with the SAS code I attach. What I want to do is to create a dataset consistent with the Type = COV structure, to be read into PROC SIMNORM. I attach the code used to create the dataset. However, I get the following error message when I ask PROC SIMNORM to read the dataset scov :

ERROR: Invalid covariance or conditional covariance matrix; matrix is not positive definite.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.SSIM may be incomplete. When this step was stopped there were 1900
observations and 5 variables.
WARNING: Data set WORK.SSIM was not replaced because this step was stopped.

And I do not understand what I have done wrong.

Any help would be deeply appreciated

Best

Piers C

PROC Mixed for EEG time series comparison

Piers — Sat, 01 Feb 2020 11:38:04 GMT

Hello,

I have individual time series data from 32 EEG channels. In the data, time runs from -200ms to 800ms. Zero is where a stimulus is presented. Two groups of subjects (10 patients v 20 controls) carried out four experimental conditions. My data have been averaged across trials, separately for each group, participant and condition. In other words, for each participant I have a separate time series for each condition, which has been averaged across trials.

For individual EEG channels I want to overplot the average EEG signal for both groups (on the y-axis) as a function of time (from -200 to 800ms, in 10ms increments), and indicate where in time the two time series are statistically significantly different from each other - i.e. effectively a set of multiple pairwise comparisons, one for each time point.

So far, to deal with the fact that we have a repeated measures design, I have used PROC MIXED as follows:

ods output diffs = diff;
PROC MIXED DATA=mlm2 COVTEST METHOD=ML;
CLASS sub group time;
MODEL EEGdata = group time group*time / SOLUTION
OUTP=pred;
RANDOM INTERCEPT / SUBJECT=sub TYPE= un;
LSMEANS group*time / diff;
run;

data subset; set diff;
if group = _group then delete;
if time ^= _time then delete;
timepoint_id = group||_group||time1;
raw_p = Probt;
keep timepoint_id raw_p;
run;

proc multtest inpvalues=subset holm hoc fdr;
run;

The general idea was to use LSmeans to get a set of pairwise comparisons which would give me a complete set of uncorrected t-tests. Then select the ones I want and run those p-values through proc multtest to do the statistical corrections. I would have liked to use the slice option, but could not find a way to obtain adjustments for the p-values, directly, so I settled on this rather clumsy route. I know you can use the adjust option in the lsmeans statement, but the only way I could think of using this to prevent computing all pairwise comparisons (thereby obtaining a very conservative adjustment as a result), was to use the lsmestimate route - but coding that looked horrific with 100 time points.

In summary:

1) Does anyone see an objection to the use of proc mixed to run the basic model in this way?

2) I am aware that taking account of autocorrelation in the data would be ideal, but including a random effect like this:

random intercept time / subject=sub type=ar(1)

caused the models to run forever, even when I resampled the timeseries to a much coarser scale. This also made me think that I might be doing something wrong.

3) Are there better / smarter / more elegant ways of obtaining the pairwise comparisons I am seeking.

Bottom line, help needed please ....

Thanks

Piers C

Re: Bidimensional regression

Piers — Fri, 20 Sep 2019 14:29:08 GMT

So - no SAS specific luck with lexjansen.com or sas.com. Generic internet searched do return hits, but this is with the R package, as I said before. I wonder if there is another term for it?
P

Re: Bidimensional regression

Piers — Fri, 20 Sep 2019 14:22:05 GMT

I will have a go ... and thanks for your help, P

Re: Bidimensional regression

Piers — Fri, 20 Sep 2019 12:25:38 GMT

Dear Paige

I have seen these papers, and they are a) very mathematical, and b) do not clearly outline how SAS has been used to implement their routines.

Hence I was wondering whether there might be SAS technical reports, or, dare I hope, worked examples using bidimensional regression in SAS

Best wishes

Piers

Didimensional regression

Piers — Fri, 20 Sep 2019 10:44:46 GMT

Hello,

Does anyone know if its possible to use the SAS procedures to run bidimensional regression, as originally described by WR Tobler (1994).

At a push I can imagine there might be a SAS macro for this, but so far I have not managed to find anything.

I am aware of an R package 'BiDimRegression', but would prefer to use SAS if possible.

Many thanks for any help with this.

Piers C