
Monday
ChrisNZ
Tourmaline | Level 20
Member since
06-23-2011
- 6,994 Posts
- 1,877 Likes Given
- 650 Solutions
- 2,759 Likes Received
-
Latest posts by ChrisNZ
Subject Views Posted 264 a week ago 1732 2 weeks ago 209 2 weeks ago 322 2 weeks ago 178 3 weeks ago 615 3 weeks ago 475 3 weeks ago 212 3 weeks ago 304 3 weeks ago 522 3 weeks ago -
Activity Feed for ChrisNZ
- Liked Re: printout of Hash Object variable in the SAS log for yabwon. Saturday
- Liked Re: printout of Hash Object variable in the SAS log for data_null__. Saturday
- Liked Re: Run multiple macros---write via code and not manually for PaigeMiller. a week ago
- Posted Re: Add index to a CAS table which is already indexed on SAS Viya. a week ago
- Posted Re: SAS batch run on SAS Programming. 2 weeks ago
- Posted Re: Why style not applied to SAS Job HTML output? on SAS Programming. 2 weeks ago
- Posted Re: Graph DPI forcing large dimensions on Graphics Programming. 2 weeks ago
- Got a Like for Re: Add SAS Packages Framework to the SAS Base/Viya. 2 weeks ago
- Got a Like for Re: How do i transform data into these? 2 different scenarios here. 2 weeks ago
- Liked Re: ODS RTF Side to Side Graph/Tables for Ksharp. 3 weeks ago
- Got a Like for Re: How do i transform data into these? 2 different scenarios here. 3 weeks ago
- Posted Re: How do i transform data into these? 2 different scenarios here on SAS Programming. 3 weeks ago
- Got a Like for Re: Efficiently Adding a Shifted Date Column to a Large SAS Dataset. 3 weeks ago
- Got a Like for Re: Why is the SAS documentation website always so slow to load?. 3 weeks ago
- Got a Like for Re: Why is the SAS documentation website always so slow to load?. 3 weeks ago
- Posted Re: Why is the SAS documentation website always so slow to load? on SAS Programming. 3 weeks ago
- Liked Why is the SAS documentation website always so slow to load? for Stratocaster. 3 weeks ago
- Posted Re: Efficiently Adding a Shifted Date Column to a Large SAS Dataset on SAS Programming. 3 weeks ago
- Liked Re: Efficiently Adding a Shifted Date Column to a Large SAS Dataset for mkeintz. 3 weeks ago
- Posted Re: How to Write in Bucket s3 Using Bulkload (SAS 9.4M8) on SAS Programming. 3 weeks ago
-
My Liked Posts
Subject Likes Posted 1 06-04-2022 02:52 AM 2 3 weeks ago 1 3 weeks ago 2 3 weeks ago 1 3 weeks ago -
My Library Contributions
Subject Likes Author Latest Post 2 1 3 14 3
2 weeks ago
1. Just a note
If you have SAS/Connect, you have this functionality by using MP connect
2. Please check
file names are like 'Test.sas' for log file or 'Test.sas' for lst type file.
It looks like you have file extensions hidden in Windows. Disable that nonsense.
... View more
2 weeks ago
You need to provide an example we can reproduce, and explain where your expectations are not met.
... View more
2 weeks ago
From the documentation: about Universal Printing:
SAS supports TIFF 6.0 for RGBA and CMYK colors.
When SAS creates a TIFF image, the image is compressed.
If the NOUPRINTCOMPRESSION is set, the size of a TIFF image file that SAS creates is extremely large.
Have you tried that syntax and that option?
... View more
3 weeks ago
2 Likes
It looks like people have been overwhelmed by the wall of text, and have no time to spend deciphering your complex explanations.
Try simplifying what you want to achieve to increase uptake.
... View more
3 weeks ago
2 Likes
I never have to wait 5 minutes, but 5 to 15 seconds is common. That's much too slow to load a page
The SAS documentation website is the slowest website I use by a wide margin.
Also, the two screens (user id then password) to login are annoying.
... View more
3 weeks ago
1 Like
Performance is fickle and depends on the data and how you use it.
ALTER creates a new data set, and isn't usually faster than CREATE, but it depends..
SQL does things in a different way, and can also exhibit slowness compared to data steps. SAS has really tuned these over the years.
Consider this:
data TEST1 TEST2 TEST3;
do I=1 to 5e7;
output;
end;
run;
proc sql; %*** 4.1 seconds ***;
alter table TEST1 add J int;
quit;
proc sql; %*** 4.7 seconds ***;
create table TEST2 as select *, . as J from TEST2 ;
quit;
data TEST3; %*** 2.7 seconds ***;
set TEST3;
retain J .;
run;
A data step is fastest here, CREATE takes 70% more time. ALTER is in the middle.
I ran that code on SAS ODA btw.
Bottom line: You'll build a feel for what methods work best for different tasks, but when performance becomes important, nothing replaces bench-marking your processes with your data on your hardware.
... View more
3 weeks ago
Well, if you are using 9.4, then you need another application to create the parquet files for you.
You can use Hive as you seem to be doing now, or use a Python package such as Panda that you can call from SAS, for example.
Then you transfer the files to S3.
There might be other methods (such as AWS Glue), but I'm not familiar with them.
Note that option BULKLOAD allows loading multiple rows of data as one unit, to insert or append them to a DBMS table.
A S3 bucket is not a DBMS. You are not shifting rows, you are shifting files, so this option has nothing to do with S3.
Caveat: I never used S3, so my understanding of its capabilities might be limited.
... View more
3 weeks ago
1 Like
You need to provide more details.
Are you using 9.4 or Viya? Do the parquet files already exist?
SAS 9.4 cannot create parquet files.
Viya can, see here, and see here for an S3 upload example.
... View more
3 weeks ago
1 Like
I don't see where you use a macro variable.
In any case, the message
ERROR: CLI describe error: [IBM][CLI Driver][DB2/AIX64] SQL0101N The statement was not processed because a limit such as a memory limit, an SQL limit, or a database limit was reached. SQLSTATE=54001
indicates that the statement is too long or too complex.
This is a DB2 limitation: Your SQL clause is too long or complex and DB2 rejects it. See https://www.ibm.com/docs/en/db2-for-zos/12?topic=codes-101
Talk to the DB2 DBA for recommendations.
As stated, the simplest work-around is to join to an uploaded table.
Another solution is to send a query smaller than the maximum allowed, and apply the rest of the tests in a subsequent phase.
... View more
a month ago
If someone knows why...
I answer my own question:
Testing c.ORIGIN and c.TYPE assesses observation-level values while testing calculated M and calculated N assesses summarised values.
Adding a presence flag to see whether a record is prsent computes the counts properly:
data HAVE;
retain FLG 1;
set SASHELP.CARS;
if mod(_N_,5) = 0 then LENGTH=.;
if ORIGIN='Asia' and TYPE='SUV' then LENGTH=.;
run;
proc sql;
create table COUNTS as
select a.ORIGIN
, b.TYPE as CARTYPE
, sum(LENGTH is missing) as M
, count(*) as N
, 'miss' as FMTNAME
, 'N' as TYPE
, monotonic() as OBS
, case when sum(FLG) < 1
then 'No records'
else catx(' ', calculated M, 'missing out of' , calculated N, '=', put(calculated M /calculated N, percent.) )
end as LABEL
from (select unique ORIGIN from HAVE) a
full outer join
(select unique TYPE from HAVE) b
on 1
left join
HAVE c
on a.ORIGIN = c.ORIGIN
and b.TYPE = c.TYPE
group by 1, 2;
quit;
proc format cntlin=COUNTS(rename=(OBS=START));
run;
proc report data=COUNTS;
columns CARTYPE ORIGIN, OBS;
define CARTYPE / group ;
define ORIGIN / across;
define OBS / sum format=miss. ' ';
run;
... View more
a month ago
You can fully control the contents of a cell if you create a format.
For example:
data CARS;
set SASHELP.CARS;
if mod(_N_,5) = 0 then LENGTH=.;
keep ORIGIN TYPE LENGTH;
run;
proc sql;
create table CNT as
select a.ORIGIN
, b.TYPE as CARTYPE
, sum(LENGTH is missing) as M
, count(*) as N
, 'miss' as FMTNAME
, 'N' as TYPE
, monotonic() as OBS
, case when calculated M = calculated N
then 'No records'
else catx(' ', calculated M, 'missing out of' , calculated N, '=', put(calculated M /calculated N, percent.) )
end as LABEL
from (select unique ORIGIN from CARS) a
full outer join
(select unique TYPE from CARS) b
on 1
left join
CARS c
on a.ORIGIN = c.ORIGIN
and b.TYPE = c.TYPE
group by 1, 2;
quit;
proc format cntlin=CNT(rename=(OBS=START));
run;
proc report data=CNT;
columns CARTYPE ORIGIN, OBS ;
define CARTYPE / group ;
define ORIGIN / across;
define OBS / sum format=miss. ' ';
run;
Note:
case when calculated M = calculated N
should be
case when missing(c.ORIGIN) | missing(c.TYPE)
but this ruins the group by operation for some reason.
If someone knows why...
... View more
03-17-2025
03:50 AM
Before trying to upload you should check that you can connect to https://*****.com.au from Python.
Can you?
... View more