I don't know what you are seeing or what your data are, but here is a program that correctly computes the FFT and IFFT of a signal with 4,000,000 elements. As far as I can tell, the FFT is computing correctly on this large vector:
proc iml;
NN = 4000000;
t = T(do(-10,10, 20/(NN-1)));
y = sin(t) - 2*sin(2*t) + 3*sin(3*t) + 0.1*rannor(t);
z = fft(y);
f= ifft(z/NN);
maxdiff = max(abs(y-f));
print maxdiff;
It's possible that your data are causing instabilities of some sort. Check to see if they are blowing up (spanning many orders of magnitude) or are widely oscilating or are otherwise degenerate.