The technical upper limit is much higher than the practical upper limit.
Check the documentation for computational problems and computational resources. A couple of things are likely to be going on.
1) you have run out of real memory and the system is swapping
2) by adding additional random effects, the model is no longer as well specified and the algorithm is having a hard time converging.
1) invites a hardware solution.
2) requires skull time and data investigation.