CXF Sending large arrays with SOAP request

Question

We are trying to send a large SOAP request to an endpoint using apache CXF.

The SOAP request contains an array of fields, which can hold 1..n number of records.

we reached a threshold of 12000 Records which is equivalent to the size of ~8.5 MB, Then we got an exception that says:

Error writing to server

Environment:

Windows server build 17763
Tomcat 9.0.46
Apache CXF 3.2.4
SpringBoot 2.5.0

What I tried to do to solve it:

Increasing the maxPostSize in tomcat server.xml config file.
Turning off/on Chunking in CXF HTTPClientPolicy
Turning on/off AutoRedirect at CXF HTTPClientPolicy
Turning on/off keep-alive at CXF HTTPClientPolicy

All this didn't work, So if there's any suggestions on why this is happening would be great.

Exception

org.apache.cxf.interceptor.Fault: Could not send Message.
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:67) ~[cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:533) [cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:442) [cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:343) [cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:296) [cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96) [cxf-rt-frontend-simple-3.2.4.jar:3.2.4]
        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:139) [cxf-rt-frontend-jaxws-3.2.4.jar:3.2.4]
        at com.sun.proxy.$Proxy168.bankPaymentsLogTransfer(Unknown Source) [na:na]
        at ws.blackbox.pay.sep.util.SepClient.paymentLog(SepClient.java:582) [classes/:0.0.1-SNAPSHOT]
        at ws.blackbox.pay.sep.util.SepClient$$FastClassBySpringCGLIB$$c9f05903.invoke(<generated>) [classes/:0.0.1-SNAPSHOT]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) [spring-core-5.3.7.jar:5.3.7]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) [spring-aop-5.3.7.jar:5.3.7]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.3.7.jar:5.3.7]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) [spring-aop-5.3.7.jar:5.3.7]
        at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:166) [spring-retry-1.3.1.jar:na]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.3.7.jar:5.3.7]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) [spring-aop-5.3.7.jar:5.3.7]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) [spring-aop-5.3.7.jar:5.3.7]
        at ws.blackbox.pay.sep.util.SepClient$$EnhancerBySpringCGLIB$$a6cc8c2d.paymentLog(<generated>) [classes/:0.0.1-SNAPSHOT]
        at ws.blackbox.pay.sep.util.CutOffTimeService.run(CutOffTimeService.java:18) [classes/:0.0.1-SNAPSHOT]
Caused by: java.io.IOException: IOException invoking https://domainOfEndpoint/bank_payments_log_transfer/bank_payments_log_transferPort: Error writing to server
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_251]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_251]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_251]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_251]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1402) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1386) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.io.CacheAndWriteOutputStream.postClose(CacheAndWriteOutputStream.java:56) ~[cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:216) ~[cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) ~[cxf-core-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:673) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63) ~[cxf-core-3.2.4.jar:3.2.4]
        ... 20 common frames omitted
Caused by: java.io.IOException: Error writing to server
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_251]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_251]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_251]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945) ~[na:1.8.0_251]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1944) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1514) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[na:1.8.0_251]
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[na:1.8.0_251]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352) ~[na:1.8.0_251]
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:377) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:373) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_251]
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:373) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.processRetransmit(HTTPConduit.java:1450) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1437) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1567) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1373) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        ... 25 common frames omitted
Caused by: java.io.IOException: Error writing to server
        at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:700) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:712) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1591) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[na:1.8.0_251]
        at sun.net.www.protocol.http.HttpURLConnection.getHeaderFields(HttpURLConnection.java:3084) ~[na:1.8.0_251]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderFields(HttpsURLConnectionImpl.java:297) ~[na:1.8.0_251]
        at org.apache.cxf.transport.http.Headers.readFromConnection(Headers.java:292) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.updateCookiesBeforeRetransmit(URLConnectionHTTPConduit.java:337) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1435) ~[cxf-rt-transports-http-3.2.4.jar:3.2.4]
        ... 27 common frames omitted

Exception in thread "Thread-11" javax.xml.ws.WebServiceException: Could not send Message.
        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:150)
        at com.sun.proxy.$Proxy168.bankPaymentsLogTransfer(Unknown Source)
        at ws.blackbox.pay.sep.util.SepClient.paymentLog(SepClient.java:582)
        at ws.blackbox.pay.sep.util.SepClient$$FastClassBySpringCGLIB$$c9f05903.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
        at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:166)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
        at ws.blackbox.pay.sep.util.SepClient$$EnhancerBySpringCGLIB$$a6cc8c2d.paymentLog(<generated>)
        at ws.blackbox.pay.sep.util.CutOffTimeService.run(CutOffTimeService.java:18)
Caused by: java.io.IOException: IOException invoking https://domainOfEndpoint/bank_payments_log_transfer/bank_payments_log_transferPort: Error writing to server
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1402)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1386)
        at org.apache.cxf.io.CacheAndWriteOutputStream.postClose(CacheAndWriteOutputStream.java:56)
        at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:216)
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
        at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:673)
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
        at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:533)
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:442)
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:343)
        at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:296)
        at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:139)
        ... 13 more
Caused by: java.io.IOException: Error writing to server
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1944)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1514)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:377)
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:373)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:373)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.processRetransmit(HTTPConduit.java:1450)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1437)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1567)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1373)
        ... 25 more
Caused by: java.io.IOException: Error writing to server
        at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:700)
        at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:712)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1591)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
        at sun.net.www.protocol.http.HttpURLConnection.getHeaderFields(HttpURLConnection.java:3084)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderFields(HttpsURLConnectionImpl.java:297)
        at org.apache.cxf.transport.http.Headers.readFromConnection(Headers.java:292)
        at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.updateCookiesBeforeRetransmit(URLConnectionHTTPConduit.java:337)
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1435)
        ... 27 more

SOAP structure

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:bank="http://service/Bank_payments_log_transfer.wsdl" xmlns:typ="http://service/Bank_payments_log_transfer.wsdl/types/"> 
   <soapenv:Header/>
   <soapenv:Body>
      <bank:bankPaymentsLogTransfer>
         <wsRequest>
            <typ:paramIn>
               <typ:bankCode>01</typ:bankCode>               
<typ:tokenKey>26E5DC20733939F2CEA787397F8FD67EA3C1027F7B91858674D0DD4B9616</typ:tokenKey>
               <typ:stmtDate>20190424</typ:stmtDate>
               <typ:transRec>
                  <!--Zero or more repetitions:-->
                  <typ:array>
                     <typ:pmtStatus>PmtNew</typ:pmtStatus>
                     <typ:dueAmt>1000.25</typ:dueAmt>
                     <typ:paidAmt>1005.25</typ:paidAmt>
                     <typ:processDate>20190312130417</typ:processDate>
                     <typ:accessChannel>ATM</typ:accessChannel>
                  </typ:array>
               </typ:transRec>
            </typ:paramIn>
            <typ:signature>C6c338Fkl1x8voMplROM9jc8LTeu3r02gIvT0Q==</typ:signature>
         </wsRequest>
      </bank:bankPaymentsLogTransfer>
   </soapenv:Body>
</soapenv:Envelope>

Web Service:

BankPaymentsLogTransfer_Service bankPaymentsLogTransferService = new BankPaymentsLogTransfer_Service();
BankPaymentsLogTransfer port = bankPaymentsLogTransferService.getBankPaymentsLogTransferPort();
HTTPConduit http = (HTTPConduit)ClientProxy.getClient(port).getConduit();
http.setClient(httpClientPolicyService.httpClientPolicy());

BkPaymentLogTrsfResponseUser response = port.bankPaymentsLogTransfer(bkPaymentLogTrsfRequestUser);

HttpClientPolicy:

 HTTPClientPolicy httpClientPolicy = new HTTPClientPolicy();
 httpClientPolicy.setConnectionRequestTimeout(Long.parseLong(HTTP_CONNECTION_REQUEST_TIMEOUT));
 httpClientPolicy.setConnectionTimeout(Long.parseLong(HTTP_CONNECTION_TIMEOUT));
 httpClientPolicy.setConnection(ConnectionType.CLOSE);
 httpClientPolicy.setAllowChunking(ALLOW_CHUNKING);
 httpClientPolicy.setAutoRedirect(SET_AUTO_REDIRECT);

Does this answer your question? [Error writing to server](https://stackoverflow.com/questions/31921426/error-writing-to-server) TL;DR; it looks like the remote server shuts the connexion down at some point, probably because it has a size limit. Now the clean way for the server would usually be to send HTTP/413 Entity too large, but sometimes, it's a firewall / load balancer / reverse-proxy configured for security that just shuts the connexion down without a warning. — GPI, Aug 03 '21 at 20:31

CXF Sending large arrays with SOAP request

0 Answers0