ERROR: ERROR: Bad request. Use the debug option for more info.

[Ihsan-Mahdi]

Hello,

I used the macro below to calculate shortest driving distances between a set of addresses. I got the error "ERROR: Bad request. Use the debug option for more info."

I used this code to check "filename gmaps url
'http://support.sas.com/techsup/service_intro.html'
debug
;"  which gave no output at all!

What is the "Bad request" error? And Can I fix it?

Thanks.

The macro I used:

*Create empty data set to append all distances;
data drdist;
run;
%macro ShortDrDist(j=);
*Create loop to access each observation;
%do i=1 %to &j;
*Assign macro variables needed;
data _null_;
set AddressY;
if _n_=&i then do;
call symput('addr1',trim(strip(Address1)));
call symput('addr2',trim(strip(Address2)));
call symput('referhid',referhid);
call symput('hospid',hospid);
end;
run;
*Find Road distances;
filename gmaps url
"http://maps.google.com/maps?daddr=&addr1.%nrstr(&saddr)=&addr2";
data drdistX&i (drop=sitecode idrivdist rdstart);
infile gmaps recfm=f lrecl=1000 end=eof;
input sitecode $1000.;
referhid="&referhid";
hospid="&hospid";
AddStart=tranwrd(tranwrd("&addr1","+"," "),",",", ");
AddEnd=tranwrd(tranwrd("&addr2","+"," "),",",", ");
if find(sitecode,'miles');
idrivdist=find(sitecode,'miles');
rdstart=idrivdist-1;
drivdist=input(compress(scan(substr(sitecode,1,rdstart),1,
' "','bc'),','),best12.);
run;
proc sort data=drdistX&i; by drivdist; run;
data drdist&i;
set drdistX&i;
by drivdist;
if _n_=1;
run;
data drdist;
set drdist
drdist&i;
if drivdist^=.;
label AddStart='Starting Address'
AddEnd='Ending Address'
drivdist='Shortest Driving Distance (miles)'
referhid='Referring Hospital'
hospid='Destination Hospital';
run;
filename gmaps clear;
%end;
%mend;
%ShortDrDist(j=&obsnum);

[ballardw]

Since you are running this in a number of macros set OPTIONS MPRINT;

then run the code and show us the LOG with the entire error message in the context of the code generated by the macro.

[Ihsan-Mahdi]

This is what I got:

MPRINT(SHORTDRDIST): *Find Road distances;
MPRINT(SHORTDRDIST): filename gmaps url "http://maps.google.com/maps?daddr=747 Broadway+Seattle+WA+98122&saddr=325 9th
Ave+Seattle+WA+98104-2420";
MPRINT(SHORTDRDIST): data drdistX1 (drop=sitecode idrivdist rdstart);
MPRINT(SHORTDRDIST): infile gmaps recfm=f lrecl=1000 end=eof;
MPRINT(SHORTDRDIST): input sitecode $1000.;
MPRINT(SHORTDRDIST): referhid=" 1";
MPRINT(SHORTDRDIST): hospid=" 29";
MPRINT(SHORTDRDIST): AddStart=tranwrd(tranwrd("747 Broadway+Seattle+WA+98122","+"," "),",",", ");
MPRINT(SHORTDRDIST): AddEnd=tranwrd(tranwrd("325 9th Ave+Seattle+WA+98104-2420","+"," "),",",", ");
MPRINT(SHORTDRDIST): if find(sitecode,'miles');
MPRINT(SHORTDRDIST): idrivdist=find(sitecode,'miles');
MPRINT(SHORTDRDIST): rdstart=idrivdist-1;
MPRINT(SHORTDRDIST): drivdist=input(compress(scan(substr(sitecode,1,rdstart),1, ' "','bc'),','),best12.);
MPRINT(SHORTDRDIST): run;

ERROR: Bad request. Use the debug option for more info.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.DRDISTX1 may be incomplete. When this step was stopped there were 0 observations and 5 variables.
WARNING: Data set WORK.DRDISTX1 was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.59 seconds
cpu time 0.54 seconds

[Quentin · Accepted solution]

Thanks for sharing the MPRINT log.  Often a next good step is to copy the SAS code shown in the log, and replicate the problem with a simple step.

This step replicates the problem:

filename gmaps url 'http://maps.google.com/maps?daddr=747 Broadway+Seattle+WA+98122&saddr=325 9th+Ave+Seattle+WA+98104-2420';

data drdistX1 ;
infile gmaps recfm=f lrecl=1000 end=eof debug;
input sitecode $1000.;
run ;

Log is:

52   filename gmaps url 'http://maps.google.com/maps?daddr=747 Broadway+Seattle+WA+98122&saddr=325 9th+Ave+Seattle+WA+98104-2420';
53
54   data drdistX1 ;
55   infile gmaps recfm=f lrecl=1000 end=eof debug;
56   input sitecode $1000.;
57   run ;

NOTE: >>> GET /maps?daddr=747 Broadway+Seattle+WA+98122&saddr=325 9th+Ave+Seattle+WA+98104-2420 HTTP/1.0
NOTE: >>> Host: maps.google.com
NOTE: >>> Accept: */*
NOTE: >>> Accept-Language: en
NOTE: >>> Accept-Charset: iso-8859-1,*,utf-8
NOTE: >>> User-Agent: SAS/URL
NOTE: >>>
NOTE: <<< HTTP/1.0 400 Bad Request
NOTE: <<< Content-Type: text/html; charset=UTF-8
NOTE: <<< Referrer-Policy: no-referrer
NOTE: <<< Content-Length: 1555
NOTE: <<< Date: Wed, 28 Feb 2024 21:56:49 GMT
NOTE: <<<
ERROR: Bad request. Use the debug option for more info.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.DRDISTX1 may be incomplete.  When this step was stopped there were 0 observations and 1 variables.
WARNING: Data set WORK.DRDISTX1 was not replaced because this step was stopped.

It looks like google doesn't olike the space characters in the URL, e.g. the space between 747 and Broadway. If you replace them with +, it works:

filename gmaps url 'http://maps.google.com/maps?daddr=747+Broadway+Seattle+WA+98122&saddr=325+9th+Ave+Seattle+WA+98104-2420';

data drdistX1 ;
infile gmaps recfm=f lrecl=1000 end=eof debug;
input sitecode $1000.;
run ;

Log is: 

64   filename gmaps url 'http://maps.google.com/maps?daddr=747+Broadway+Seattle+WA+98122&saddr=325+9th+Ave+Seattle+WA+98104-2420';
65
66   data drdistX1 ;
67   infile gmaps recfm=f lrecl=1000 end=eof ;
68   input sitecode $1000.;
69   run ;

NOTE: The infile GMAPS is:
      Filename=http://maps.google.com/maps?daddr=747+Broadway+Seattle+WA+98122&saddr=325+9th+Ave+Seattle+WA+98104-2420,
      Local Host Name=
      Local Host IP addr=
      Service Hostname Name=
      Service IP addr=
      Service Name=N/A,Service Portno=443,Lrecl=1000,
      Recfm=Fixed

NOTE: 171 records were read from the infile GMAPS.
NOTE: SAS went to a new line when INPUT statement reached past the end of a line.
NOTE: The data set WORK.DRDISTX1 has 170 observations and 1 variables.

I didn't test the whole macro, or even the whole data step.  But fixing the URL at least allows the infile statement to work in scraping the page.  

BTW, Mike Raithel wrote a paper on scraping data from google maps, might be a helpful resource for you:
https://www.basug.org/_files/ugd/6fce8c_964ee9ecca2d4def95d523ccef5b9465.pdf

He presented it as a webinar for BASUG, the recording is:
https://www.basug.org/videos?wix-vod-video-id=4ffd50aeee284b1ca2a7c3ef5a91790e&wix-vod-comp-id=comp-...

[Ihsan-Mahdi]

Thank you so much. I figured the issue had to do with address formatting. much appreciate your kind support

[Reeza]

Run the macro with the macro debugging options and check your log.

options mprint mlogic;

Bad request sounds like a response from the Google site, ie a bad URL request was passed to the filename function.