Software Deployment

I tried publishing an application MATLAB. The source msi total size is around 1.2 GB. The source I got had 5 separate msi which can be installed each independently but Matlab to work all 5 msi needs to be installed in the system. What I am doing is creating separate ZSERVICE for each msi and keeping them in single group service so that I can deploy those using the single group service.
But one of them is not getting published to RCS. The error says "unable to alloc 65536 bytes" and then publisher hangs.
The msi is question is having 15,679 files and 276 MB in size.
What I noticed more the following folder under the
\toolbox\compiler\mcr\matlab which have 31 folders. This folders have subfolder names with @.Some files are getting published but it is not able to publish all.
I want to know is there any settings need to be done in Publisher so that it can accept the special character in folder names or allocate more than 65536 bytes.Please guide me to solve the problem. I tried all the publishing methods. Thanks and regards,