SAS Support Communities

Hi, It might be that your organisation doesn't permit connections out to s3, as if its timing out, this sounds like it might be a connection issue. You could try using curl from a command line prompt to your s3 bucket, this would at least check if your getting a connection. Have a look at this website and check it meets your requirements https://www.googlinux.com/retrieving-s3-objects-using-curl/   regards,   Colin   ... View more

Hi, Not sure there is a debug option, though the following will list files to the log,    proc s3 config="c:/temp/aws/tks3.conf"; list "XXXsasdata"; /* will list files to the log */ run;   I've also used with Viya a libname statement that points directly to the folder and you can then read directly in the usual way. I'm not sure how AWS transfer charges apply, so i always disconnect afterwards !   caslib mys3 datasource=(srctype="s3" accesskeyid="AAAAAAAAAAAA" secretaccesskey="BBBBBBBBBBBBBBBB" region="EU_Ireland" bucket="XXXXsasdata" /* objectpath="/asr/projB" */ usessl=true); ... View more

I guess a use case might be in the insurance sector where fraud is often hidden, though there are probably cases where an investigation has taken place and a result has been found of fraud or not-fraud.   Anyone used this technique in other applications?   Colin ... View more

Hi,   It sounds like you need PROC S3, this reads / lists / creates files in S3 buckets. It has a variety of statements so check in support.sas.com for more details, but you will need something like:   /* read data in and import excel file*/ filename _s3file temp; %let _locals3file = %sysfunc(pathname(_s3file)); /* connect to S3 */   proc s3 config="/home/sas/Documents/tks3.conf"; get "/XXXXsasdata/fakenames100.csv" "&_locals3file"; /* XXXXsasdata  is your bucket name */ run;   /* this bit hopefully works - i was using xlsx spreadsheet so haven't tested with csv */ proc import datafile =_s3file replace dbms=csv out=work.fake1; getnames=yes; run;   you will also need a file called tks3.conf (or you can put the details in code, but this contains something like the following) ssl=yes keyID=XXXXXXXX secret=YYYYYYYYYYYYYYYYYYYYYYYY region=euireland     Let me know how you get on - good luck.   Colin ... View more

Hi, Great article Celia. I'd also add in another feature of networks and that of repeated events. Probably best to explain by an example. Let's say we have 5 linked motor insurance claims on a network and all 5 occur at night. From our past records only 25% of claims occur at night. If we assume independence of events then the probability of all 5 claims being at night = (0.25)^5 = .001 (or  around 1 in 1000).  You can use the binomial distribution for 0,1,2,3 &4 claims at night.   The independence assumption is quite strong and probably OK in this case [though you may be able to think of exceptions e.g. maybe all night time shift staff etc.], in other cases may not work as well.   Good luck with your analysis.   Colin ... View more

Hi, I also like to use t-sne (or t stochastic neighbour embedding), which can highlight anomalies (and other patterns). In the example below using credit card fraud data from Kaggle, I've reduced the dimensionality from 27 down to 2 and then overlaid the fraud flags (the fraud flags played no part in building the model - only for display). Many of the frauds seem quite anomalous from the rest of the group, so might help to identify new fraud typologies. If you get a chance try it out on some data and see what you get. Good luck. Colin   ... View more