The below might work for you.  It's built on a few assumptions - specifically that ID, Name and Logline uniquely identify a row, and are always in order in your input file, and that Logline is sequential, starts from 1 each time ID or Name changes, and has no gaps in the sequence. Basically, uses retain to keep track of the previous end date (similar to Reeza's suggestion with lag), and then uses that to identify the second row in each pairing.  Given the above assumption, this can never happen when Logline = 1, so it outputs two lines, with Loglines - 1, and then with Logline as is.  This output is then used as a filter, via a merge. I'd be interested to see if there's a way of doing this in one step (without resorting to SQL). data have; infile datalines dsd dlm=' '; length ID 8.; format name $1.; length logline 8.; format stdtc ddmmyys10.; format endtc ddmmyys10.; input ID name $ logline stdtc:date. endtc:date.; datalines; 101 a 1 01-jan-2013 03-jan-2013 101 a 2 05-jan-2013 06-jan-2013 101 b 1 01-jan-2013 03-jan-2013 101 b 2 05-jan-2013 06-jan-2013 101 c 1 01-jan-2013 03-jan-2013 101 c 2 05-jan-2013 06-jan-2013 102 a 1 01-jan-2013 03-jan-2013 102 a 2 04-jan-2013 06-jan-2013 102 b 1 01-jan-2013 03-jan-2013 102 b 2 04-jan-2013 06-jan-2013 102 b 3 07-jan-2013 10-jan-2013 ; run; data have2/view=have2; set have; by ID name logline; /*the input has to be sorted by this*/ format previous_endtc ddmmyys10.; retain previous_endtc; keep ID name logline; if first.name then call missing(previous_endtc); else if previous_endtc + 1 ~= stdtc then do;     logline = logline - 1;     output;     logline = logline + 1;     output; end; previous_endtc = endtc; run; data want; merge have (in=h1) have2 (in=h2) ; by ID name logline; if h2 then output; run; ... View more

A very good point Art.  Depending on the analysis being done, it could be as simple as adding a grouping column (randomly or whatever suits) and then just including that as a classification variabale.  That way SAS will just split the results out by the groups anyway, and not have to bother with complex coding. ... View more

I could be misunderstanding your requirements, but you can use the hash object to output to datasets arbitrarily, without having to use macros.  The main downside is that the dataset must be sorted by the group number, but you should be able to to this while maintaining the relative order within each group if that's important... The below example generates some random data at the start, and appends a group number (filenum) field as it goes.  This field is included in the ultimate output, but if required we should be able to remove it. data Test; format key 8. x1-x4 y1-y4 21.3 filenum 8.; array n x1-x4 y1-y4; filenum = 0; do key = 1 to 100000;     do i = 1 to dim(n);         n = round(ranuni(-1)*50000,0.001);     end;     filenum + 1;     if filenum > 5 then filenum = 1;     output; end; drop i; run; proc sort data=Test; by filenum; run; proc sql; select name into :fieldlist separated by "','" from sashelp.vcolumn where libname='WORK' /*change to your library*/ and memname='TEST' /*change to dataset*/ ; quit; data _null_; call symput("fieldlist","'&fieldlist.'"); run; %put &fieldlist.; data _null_; if 0 then set Test; /*set up PDV with formats, but load no data*/ declare hash h (ordered:'a'); h.defineKey(&fieldlist.); h.defineData(&fieldlist.); h.defineDone(); do until (last.filenum);     set Test;     by filenum;     h.add(); end; h.output(dataset: 'group' || strip(put(filenum,8.))); run; If you want more groups split out, then just increase the value that filenum resets at.  As long as the "output" line in the final data step constructs a valid sas dataset name, then you can use this method to output any number of datasets you like, without specifying them in code. Of course your analysis will then require a macro to run your proc means (etc.) against all of those datasets, but hopefully this will be tidier for creating the datasets in the first place. ... View more

No worries.  I'm no expert on this - I largely put it together from the SAS examples (and the regex from US and Canada zip code validation RegEx), but: data ZipCodes;     infile datalines dsd dlm=',';    input name: $16. zip:$40.;    datalines; Johnathan,A2A 2B2 a39js 93jaaj39 Seth,AAA B2B 3C3apples Kim,39204B2B3C3B2B3C3 Samuel,93849-3843 ; data ZipCodes2; set ZipCodes; regex = prxparse('/[ABCEGHJKLMNPRSTVXY]{1}\d{1}[A-Z]{1} *\d{1}[A-Z]{1}\d{1}/'); if prxmatch(regex, zip) then do;     postcode = prxposn(regex,0,zip); end; drop regex; run; For the examples given, this will pick up the first text string that matches the regex string.  This is pretty much as you described, although it excludes some characters in the first block (i.e. DFIOQUZ) - if this isn't correct you could just add them back in, or change it to A-Z. If your data really has junk in it, then you'll need to be mindful of any false positives too. ... View more

It might help to have a sample of what your data looks like (with real people's information obscured of course!). Generally though, post codes in your country will likely follow a specific pattern.  As such locating them is probably done best using Regular Experssions.  The SAS online help example 3 for the function PRXMATCH (http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a002296115.htm) attempts to do just that. I imagine the SAS example is for US zip codes, so if you're not in the US then you'll need to find the pattern suitable for your own country. Wikipedia provides some examples for UK post codes, although I'd imagine the regex strings would need modified slightly to work with SAS.  Postcodes in the United Kingdom - Wikipedia, the free encyclopedia Hope that helps! DF. ... View more

One possibility - you may not have accounted for how SAS deals with blank characters. For example: data _null_;     format a $10.;     format b $3.;     a = 'abc';     b = 'abc';     y = md5(a);     z = md5(b);     put y= hex32.;     put z= hex32.; run; Yeilds the output: y=804E99ADC4A63980A12CB81D3D62861E z=900150983CD24FB0D6963F7D28E17F72 Were A and B equivalent, their MD5 results would be identical. I don't have a quick way of running php at work, so I used this site: http://www.functions-online.com/md5.html The results were: "abc       " (7 spaces) -> 804e99adc4a63980a12cb81d3d62861e "abc" -> 900150983cd24fb0d6963f7d28e17f72 which seems to confirm, at least for me example. Hope that helps! ... View more

I don't think there's a simple solution for this - it's a problem we've had since migrating to EG too.  We use 4.1, so if there's a better way of doing this in the later versions, someone please let me know and I can add it to my list of reasons to upgrade(!). Your local PC doesn't do any of the processing, as there is no SAS server running there any more.  Everything has to be done by the remote server, which doesn't have access to files on your local PC.  Enterprise Guide has its own built-in import/export routines, but these cannot be named dynamically based on your SAS code. The solution we tend to use, is to have SAS output files to a folder somewhere on the SAS server itself.  This allows you to use the same code as before (obviously converting local windows paths to "local" unix paths etc.), and then use something like FTP to retrieve the files. Your server admin would have to create this location for you, and assign credentials for FTP. Files can be retrieved manually using FTP, or automatically alongside scheduling your report (for example).  Most of our processes are scheduled using code based on the Schedule option in Enterprise Guide's interface. The below code is from EG 4.1, so you'll need to double check for any changes for 4.3.  (There may be a simpler way of writing this, but its in VBScript so it was never going to be pretty!) It will copy a file called "daily_output_20120123.csv" from /var/opt/users/DF/ to c:\outputs\.  A file containing the relevant FTP commands is created localled - I normally rename this with each script to match the report name (to avoid overlaps). The FormatDate() function formats today's date in the way that I need it - you may require customization. All this code goes in the VBS file that EG creates for you automatically.  Obviously the path to the EGP file would need updated as required. Option Explicit Dim app Call dowork 'shut down the app If not (app Is Nothing) Then     app.Quit     Set app = Nothing End If call FtpGet Sub dowork()     On Error Resume Next     '----     ' Start up Enterprise Guide using the project name     '----     Dim prjName     Dim prjObject     prjName = "\\home\projects\my report.egp"    'Project Name     Set app = CreateObject("SASEGObjectModel.Application.4")     If Checkerror("CreateObject") = True Then         Exit Sub     End If     '-----     ' open the project     '-----     Set prjObject = app.Open(prjName,"")     If Checkerror("app.Open") = True Then         Exit Sub     End If     '-----     ' run the project     '-----     prjObject.run     If Checkerror("Project.run") = True Then         Exit Sub     End If     '-----     ' Save the new project     '-----     prjObject.Save     If Checkerror("Project.Save") = True Then         Exit Sub     End If     '-----     ' Close the project     '-----     prjObject.Close     If Checkerror("Project.Close") = True Then         Exit Sub     End If End Sub Function Checkerror(fnName)     Checkerror = False     Dim strmsg     Dim errNum     If Err.Number <> 0 Then         strmsg = "Error #" & Hex(Err.Number) & vbCrLf & "In Function " & fnName & vbCrLf & Err.Description         'MsgBox strmsg  'Uncomment this line if you want to be notified via MessageBox of Errors in the script.         Checkerror = True     End If End Function Public Sub FtpGet()     Dim remotepath, localpath, get1     Dim vFTPServ     Dim fNum     remotepath = "/var/opt/users/DF/"     localpath = "C:\Outputs\"     localpath = Trim(localpath)     If Right(localpath, 1) <> "\" Then localpath = localpath & "\"     remotepath = Trim(remotepath)     If Right(remotepath, 1) <> "/" Then remotepath = remotepath & "/"     get1 = "daily_output_" & Trim(FormatDate(Date-1)) & ".csv"     vFTPServ = "192.168.79.221"     Dim FSO, File1     Set FSO = CreateObject("Scripting.FileSystemObject")     Set File1 = FSO.CreateTextFile("c:\ftpcommands_daily_report.txt", True)     File1.WriteLine("user username password")     File1.WriteLine("cd " & remotepath)     File1.WriteLine("bin")     File1.WriteLine("get """ & get1 & """ """ & localpath & get1 & """")     File1.WriteLine("close")     File1.WriteLine("bye")     File1.Close     SET File1 = NOTHING     SET FSO = NOTHING     dim objShell     set objShell = wscript.createObject("wscript.shell")     objShell.Run "cmd /c ftp -n -i -g -s:c:\ftpcommands_daily_report.txt" & " " & vFTPServ     Set objShell = Nothing End Sub Function FormatDate(d)     FormatDate = Year(d) & pd(Month(d), 2) & pd(Day(d), 2) End Function Function pd(n, totalDigits)     If totalDigits > Len(n) Then         pd = String(totalDigits - Len(n), "0") & n     Else         pd = n     End If End Function ... View more

A quick google search suggests JDML files are just XML files, which SAS can read.  If this is indeed the case for your files then you might find it helpful to start with http://www2.sas.com/proceedings/sugi29/119-29.pdf .  This is something I've only started to explore myself too. Otherwise, perhaps you could post a sample file and some details of the file structure?  There may be other options. ... View more

I've done as Patrick says myself, as a one-off. 7-zip allows you to extract multiple EGP files at once into folders named after each project, and maintains the structure within the .egp zip format files.  This allows you to extract all of the code.sas files, as they're actually split into seperate files.  The normal windows search function then allows you to search through all the files for the keywords you're looking for. It's not ideal for anything but a one-off exercise mind you.  Also I couldn't see a way to tie the code.sas file with its name in the EG project - I guess it's stored in project.xml somewhere though... ... View more