Friday, March 02, 2007

Apache Axis and Commons HTTPClient

Someone asked me how they could turn on NT Authentication for web service using Apache Axis (the web service used by ColdFusion). By default Axis uses its own HTTP client code, org.apache.axis.transport.http.HTTPSender, to send the XML/SOAP POST requests to a web service. This uses HTTP 1.0 and generally works file.

Axis also supports the Jakarta Commons HTTPClient library, and has since 1.0. To configure Axis to use this instead of its own library you must edit the client-config.wsdd file used by Axis. It gets found on the classpath and generally you don't actually have one and the one built in to axis.jar gets used.
The interesting line is the http transport. To switch Axis to use the HTTPClient jar, you would change this:

<transport name="http" pivot="java:org.apache.axis.transport.http.HTTPSender"></transport>

To this:

<transport name="http" pivot="java:org.apache.axis.transport.http.CommonsHTTPSender"></transport>

Once you have this code configured, Axis will use the HTTPClient library for it HTTP needs. Since the HTTPClient library supports NT Authentication, you just set the username/password on the Stub object as you would normally do for (say) Basic Authentication and it will just work. If you are talking to a .NET web service, you are done.

BUT switching the line in the client-config.wsdd file alone doesn't do the trick if you are talking to a ColdFusion web service. Talking to a ColdFusion web service via Apache, you will get an "411 - length required" error back that looks like this:

The Apache JRun connector doesn't allow chunked encoding without a content length (generally true for all general pupose connectors, including mod_python) and the CommonsHTTPSender class in Axis does not provide a content-length. Go figure.
Using the built-in JRun web server you get a a "content not allowed in prolog" error because it appears the JRun web service doesn't understand chunked transfer encoding. Go figure again.
I didn't try it with IIS. My guess it that it might work.
To solution? Turn off the chunked encoding, which you can do by setting a property on the web service object in CFML like this:

ws = CreateObject("webservice", "http://localhost/ws/service.cfc?wsdl");
httpheaders = CreateObject("java", "java.util.Hashtable").init();
httpheaders.put("chunked", "false");
ws._setProperty("HTTP-Request-Headers", httpheaders);
result= ws.myOperation();

The Apache code wants a Hashtable, where it should just look for a Map, so you can't just use a CFML structure for the value of the HTTP-Request-Headers.


54 comments:

AlanW said...

This looks very promising as I have spent much time trying to get NTLM Authentication to work from a Java client for Axis.

Do you know if this would also support the transport "https"?

Tom said...

Yes, it should support SSL as well.

Posti said...

Hi Tom

Wondering if you could shed any light on getting a web service to work in CFMX 7.0.2 multi instance
it seems whatever we do we run into package conflicts. Ive notice some technotes on log4j and webservices.jar but they have been no help.

see stack trace below:


04/11 09:24:20 error
[1]org.apache.commons.logging.LogConfigurationException: java.lang.ClassCastException
at org.apache.commons.logging.LogFactory.newFactory(LogFactory.java:558)
at org.apache.commons.logging.LogFactory.getFactory(LogFactory.java:345)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:409)
at org.apache.axis.components.logger.LogFactory.getLog(LogFactory.java:76)
at org.apache.axis.client.AxisClient.clinit(AxisClient.java:84)
at org.apache.axis.client.Service.getAxisClient(Service.java:143)
at org.apache.axis.client.Service.init(Service.java:152)
at com.amadeus.xml.AmadeusWebServicesLocator.init(AmadeusWebServicesLocator.java:12)



using the createObject("webservices", method we acutally run into issues compiling since the path to the stub files being created is longer than 256 characters!

so attempting to compile manually at a different location and call via createObject("java", ... causes the stack trace mentioned above.

Id love to see a post on how you would recommend you manually compile a web service?

it appears jrun has axis 1.1 installed and CF is running axis 1.2

im able to compile the stubs using the axis 1.2 libs but it appears when running the code its using the 1.1 libraries ?

cmwolak said...

This information was very helpful. Worked like a charm. My next question is, we're using WSDL2Java to generate our BindingStub file, where this code snippet belongs. Is there some way to tell WSDL2Java to generate this code (for example, to tell it that we'll be using CommonsHTTPSender), or do we have to add this code manually each time we do WSDL2Java? Thanks for a great post with a very targeted solution!

John Jarrard said...

Tom, Thanks for the great tip. I was wondering if you could clue me in about requesting the AXIS use HTTP 1.1

I'm trying to connect (over https) to a SQL Server 2005 EndPoint. It requires that each call be auth'd, and it requires that http1.1 be used since it returns the data in "chunks" - god love that word.

Anyway, I've found this tip on Axis' mail list, but I don't know how to get this to work in CF8.
ctx.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,HTTPConstants.HEADER_PROTOCOL_V11);

Any help you might give would be wonderful! I appreciate your blog very much. Good stuff.

Tom said...

John,

You should be able to call setProperty() on the web service object returned from CreateObject():

ws = CreateObject("webservice", "http://...");
ws.setProperty("axis.transport.version", "HTTP/1.1");

Tom said...

I meant to use an underscore ("_setProperty") on the web service stub.

John Jarrard said...

Hey Tom,

Thanks for that... I guess that is not my issue. Shoot. I'm getting a weird error when I use CFINVOKE/CreateObject

(505)HTTP Version not supported

I added the 1.1 code you suggested... The strange thing is that I have tested the service with SoapUI and I've created my own soap *message packet* and posted it via CFHTTP and both seem to get a response that looks right.

But when I use the CreateObject method or CFINVOKE tag, I get that bizarre response.

I used Charles (great little proxy) to observe the traffic, and what I see is that CF gets the WSDL happily then either doesn't auth correctly again or potentially CF doesn't understand the WSDL...

I tried switching to CommonsHTTPSender, but when I do a CFDump I get a java factory looking thing...

Would you mind explaining the 505 error to me a little more? I'm obviously forcing HTTP/1.1 with your suggested header...

I really appreciate anything you can do to help.

John Jarrard said...

Hey Tom, So after a little messing around, I found "1.1" to the value of ...version. Not "HTTP/1.1"

Now the weird-ish thing is the result I'm getting when I CFDUMP.

You can take a look at a screen shot of it here: Take a look at the result

If you want the passwords to test it, email me jbj{at}mac{dot}com - I'll gladly send it to you.

Any clue why I'm getting the Java object rather than a result set?

John Jarrard said...

Hey Tom -

So I figured out how to get a result set finally. I revamped the rather kludgy convertDotNetDataset function from Joe Rinehart.

Your blog is dope. (that's a good thing). I do have one question though. Does AXIS or CF ever plan to build in dataset conversion to accomodate M$ ever-so-idiotic format? The method I'm using is a dog. Slow slow slow.

Thanks again for everything - JJ

Tom said...

Good thing I just sent off an email message before checking for all the comments. :-)

Anyway, yes we have talked about automatically converting the MS proprietary DataSet to our Query object. Its enhancement 64065 and its being evaluated for the next major CF release. This is no guarantee that it will get done. We are looking at enhancing web service support (updating to a newer Axis engine maybe)so I hope that we can do this also.

Marc Esher said...

Tom, thanks for the post. It's getting me "really" close to what I need to accomplish and I'm wondering if you can shed a bit more light on something.

I'm a contributor to the mxunit unit testing framework for cf. I'm using axis 1.4 to make ws calls to CF. Everything works fine except when NTLM is in the picture.

putting together everything I learned, I modified the generated stubs to accept a username/password (in the Locator class and then to use the commonsHttpSender in the BindingStub.

here are the snippets:

Locator....
public RemoteFacadeServiceLocator(String url, String username, String password){
RemoteFacadeCfc_address = url;
if(username != null && username.trim().length()>0){
System.out.println("Setting username and password.");
this.username = username;
this.password = password;
}
}


Binding Stub....


org.apache.axis.transport.http.CommonsHTTPSender requestConnectionHandler = new org.apache.axis.transport.http.CommonsHTTPSender();
org.apache.axis.transport.http.CommonsHTTPSender responseConnectionHandler = new org.apache.axis.transport.http.CommonsHTTPSender();
_call.setClientHandlers(requestConnectionHandler, responseConnectionHandler);

//without this, it can throw a "content not allowed in prolog" message when using the CommonsHTTPSender;
//I had this happen to me when running against the built-in CF webserver; it did not behave this way with IIS.
//found the fix here: http://tjordahl.blogspot.com/2007/03/apache-axis-and-commons-httpclient.html
Hashtable headers = new Hashtable();
headers.put("chunked", "false");
headers.put("Connection","keep-alive");
_call.setProperty("HTTP-Request-Headers", headers);


Still, though, no matter what I try I get a 401 Not authenticated error when using NTLM. Basic works just fine. But noone I know can get the plugin to work using NTLM.

Do you see anything in the code that I might be missing?

Thanks a lot for taking a few minutes to look!

Also, I don't know if you'll remember this or not, but remember at Max this past year, during the "Meet the CF team" session, when Tim Buntel started fielding questions and a dude piped up and said "Thanks Tom Jordahl for all your work on Axis!".... that dude is me!

Thanks again.

Marc

Tom said...

@Marc - I am not sure why you would need to hack the WSDL2Java generated code in the manner you describe.

Simple replacing the transport in the Axis client-config.wsdd should take care of most of that for you, including the NTLM negotiation with IIS. Once you use the Commons HTTP library, just setting the username and password should be enough. Perhaps you are not include the NT Domain in the username (i.e. ADOBE\tjordahl)?

I can tell you that the built=in web service in JRun/CF is not going to support this.

Anonymous said...

How exactly do you set the username/password on the Stub object?

It kept on failing at the WSDL call.

Cheers
Glen

Tom said...

@anonymous - you simply call the setUsername() and setPassword() functions on the web service object.
CFScript code:

ws = CreateObject("webservice", "wsdl-url");
ws.setUsername("Tom");
ws.setPassword("mysecret");

result = ws.doTheThing(arg1, arg2);

Steve said...

IN CF 7.1 I am trying to connect to sharepoint webservices. I have edited the client-config.wsdd file and restarted. I cannot create the webservice (error:cannot generate stub objects), so I cannot set ws.username and ws.password. I cannot pass wsargs (not an option in 7.1) so I am uncertain if it is even possible to accomplish this in CF 7.1. I feel that authentication is the issue here.

Code:
ws=createobject("webservice","http://site/subsite/_vti_bin/lists.asmx?wsdl");

Generates error: Cannot generate stub objects for webservice invocation.

Any advice appreciated.

Tom said...

@Steve - My guess is that there is something about the WSDL that the sharepoint web service publishes that the Apache Axis engine in ColdFusion MX 7 (which is identical to CF8) can't generate Java code for.

I would try running the wsdl2java.exe on this WSDL by going to the runtime/bin directory of you CF install and running the command:

wsdl2java.exe -v http://url/of/wsdl

This will create a bunch of Java files, or it will generate an error. If it generates an error, then I am uncertain what I can do for you. You could try upgrading the Axis in CF (which is version 1.2.1) to the latest 1.4 release. We haven't tested this at all but it might be worth a try. You will have to update the jar files in the ColdFusion lib directory with the latest versions.

Steve said...

I gave that a try and I did get an error:

java.net.ProtocolException: Server redirected too many times (20) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

I am researching this.

Steve said...

I upgraded AXIS to 1.4 and applied your fix to cfmx_bootstrapper.jar. There are no changes. I have CF8, but I haven't upgraded yet. I think I will bit the bullet and do that.

Sri said...

I have used Axis 1.4 as Web service Client for consuming some external web
services. I have to use a proxy host(in our network) to connect to external
services which is currently handled by setting it in the System Properties.

Things were working fine with the defualt HttpSender which handles stuff
between Client and the Soap Server.

I came across CommonHttpSender which can be used to to get control of some
settings like setting connections pools and Connection timeout(which i am
interested in) etc.,

When I replaced the HttpSender to CommonsHttpSender in the
clilent-config.wsdd, I am facing problems to get it working..
*
Problem 1:*
When a proxy is enabled(System Properties) and session enabled(
_locator.setMaintainSession(true);) since I need to maintain session

I get NullPointerException on Line 190 in CommonsHttpSender ( boolean
secure =* hostConfiguration.getProtocol()*.isSecure();) .. I don't why..:-(

I fixed(temp work around) it by extending this Sender and overrided the
getHostConfiguration() method by adding the below line explicitly
config.setHost(targetURL.getHost(), port, targetURL.getProtocol()); in
what ever the case may be.

*Problem 2:*
The above work around fixes the NullPointerException but now get
org.apache.commons.httpclient.NoHttpResponseException

When I run the same program with out proxy setting (out of our network) I
get
(404)Object Not Found
at org.apache.axis.transport.http.CommonsHTTPSender.invoke(
CommonsHTTPSender.java:218)

I am not sure why the same Service works with defualt HttpSender but not
with the CommonsHttpSender.

Is there any thing like configuration etc., which I missed out thats needs
to be taken care to plug in the CommonsHttpSender.

*Environment: Apache Axis 1.4, commons-httpclient-3.0.1.jar, Java
1.5(150_06) and all supporting jars.

Steve said...

I'm back after two months. I finally upgrade to CF 8. I am trying desperately to connect to the simple lists.asmx webservice on a sharepoint site. When I access the URL from my browser I retrieve the WSDL fine. I do not have to authenticate (I think it is using windows integrated authentication).

I used:

cfobject name="ws" webservice="http://site/lists.asmx?wsdl" password="xxxx" username="DOMAIN\xxx">

I get "Unable to read WSDL from URL: site/lists.asmx. Error: 401 Unauthorized.

I ran wsdl2java.exe -v http://site/lists.asmx and I got:
ERROR: transport error 202: bind failed: Address already in use
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../.
./../src/share/back/debugInit.c:690]
FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_E
RROR_TRANSPORT_INIT(197)

I found a post about the CF command line debugger causing this, so I disabled it in CF Admin.

Tried wsdl2java.exe -v http://site/lists.asmx and I got:
java.net.ProtocolException: Server redirected too many times (20)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLCon
nection.java:1253)
at java.net.URL.openStream(URL.java:1009)
at org.apache.crimson.parser.InputEntity.init(InputEntity.java:209)
at org.apache.crimson.parser.Parser2.parseInternal(Parser2.java:471)
at org.apache.crimson.parser.Parser2.parse(Parser2.java:305)
at org.apache.crimson.parser.XMLReaderImpl.parse(XMLReaderImpl.java:442)

at org.apache.crimson.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl
.java:185)
at org.apache.axis.utils.XMLUtils.newDocument(XMLUtils.java:322)
at org.apache.axis.utils.XMLUtils.newDocument(XMLUtils.java:367)
at org.apache.axis.wsdl.symbolTable.SymbolTable.populate(SymbolTable.jav

This is where I stopped with CF 7.1.

Some research on this leads me to believe I am still having authentication issues. One post I read said that WSDL tries once without authentication and then tries again with credentials and this causes it to bounce between an error screen and login box or something.

Just out of curiosity I did a cfhttp get to the same url with the same credentials. It returns the HTML for a HTTP Error 401.2 - Unauthorized: Access is denied due to server configuration.
Any advice would be appreciated.

Steve said...

I can download the WSDL manually and save it to the Bin folder. I run WSDL2java manually and it generates the stubs! Any to tell CF to just use those stubs, no need to regen?

Steve said...

A break through of sorts. This works from the command line:

wsdl2java.exe -v -U [domain]\[username] -P [password] http://site/lists.asmx?wsdl

But I can't get it to work from CF:
cfobject name="ws" webservice="http://site/lists.asmx?wsdl" wsdl2javaArgs="-v -U [domain]\[username] -P [password]">

I am at a total loss as to why it is not working.

Tom said...

@Steve - First this is probably something that you want to work with Adobe support on. Failing that, you can use the Adobe forums to get help as well. Posting stack traces to my blog comments isn't the greatest. :-)

The simplest way to check that ColdFusion can get the WSDL is to use the CFHTTP tag to retrieve it. The CFHTTP code is the same code that cfobject uses to get the WSDL.

If the web service in fact *does* require NT Authentication to get the WSDL (check with a non-MS browser), this is something that ColdFusion doesn't support. I suggest saving a copy of the WSDL for the service in a local directory that is web accessible (i.e. localhost) and use that to consume the service.

Travis said...

I am having an issue with the way AXIS is creating the XML-SOAP request that it sends to a 3rd party webservice.

PROBLEM: AXIS is adding xsi:type="xsd:int" to the request.

<ns1:month xsi:type="xsd:int">12</ns1:month>

The 3rd party webservice rejects the request, stating “You submitted an invalid XML request. Please verify your request and retry the transaction.”

If I manually remove the xsi:type="xsd:int" string and perform a manual CFHTTP post of the raw SOAP message (that I captured using getSoapRequest) , it works fine. However, this requires use of CFHTTP and not the native use of the wsdl object, which then requires me to parse the XML response myself, which is another time-consuming and error-prone task.

Things I have tried:
See: http://ws.apache.org/axis/java/reference.html#WSDL2JavaReference

Modify server-config.wsdd, add the following to the <globalConfiguration>
<parameter name="sendXsiTypes" value="false"/>
This setting seems to make no difference. The SOAP request generated by AXIS is identical regardless if the above is set to true or false. Yes, I was sure to restart the CFApp server each time I changed the above.


Adding wsdl2JavaArgs:
argStruct.wsdl2JavaArgs="--property PROP_SEND_XSI=false";  ERRORS OUT, tried several versions of this.


Things I haven’t figured out yet:
Perform a _setProperty on the object before calling the method, like:
optimalPayments.creditCard._setProperty("java?????.axis.????.sendXsiTypes", false);
Although, I’m not sure the above will even work, see
http://osdir.com/ml/text.xml.axis.user/2002-12/msg00513.html


CALLING CODE:

<cfscript>
argStruct = structNew();
argStruct.refreshWSDL="yes";
argStruct.wsdl2JavaArgs="";
//argStruct.wsdl2JavaArgs="-W -T 1.2";
//argStruct.wsdl2JavaArgs="--property PROP_SEND_XSI=0";
optimalPayments.creditCard = createObject("webservice", "https://webservices.optimalpayments.com/creditcardWS/CreditCardService/v1?wsdl", argStruct);

requestStruct = structNew();
requestStruct.merchantAccount = structNew();
requestStruct.merchantAccount.accountNum = "89999340";
requestStruct.merchantAccount.storeID = "test";
requestStruct.merchantAccount.storePwd = "test";
requestStruct.merchantRefNum = randRange(99,99999999);
requestStruct.merchantRefNum = "987564";
requestStruct.amount = "10.00";
requestStruct.card = structNew();
requestStruct.card.cardNum = "5442981111111023";
requestStruct.card.cardExpiry = structNew();
requestStruct.card.cardExpiry.month = 12;
requestStruct.card.cardExpiry.year = 2010;
requestStruct.billingDetails = structNew();
requestStruct.billingDetails.zip = "49444";

response = optimalPayments.creditCard.ccPurchase(requestStruct);
request = getSoapRequest(optimalPayments.creditCard);
</cfscript>

<cfdump var="#optimalPayments.creditCard#">
<cfdump var="#request#">
<cfdump var="#response#">



GENERATED SOAP REQUEST:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<ccPurchase xmlns="http://www.optimalpayments.com/creditcard/v1">
<ns1:ccAuthRequestV1 xmlns:ns1="http://www.optimalpayments.com/creditcard/xmlschema/v1">
<ns1:merchantAccount>
<ns1:accountNum>89999340</ns1:accountNum>
<ns1:storeID>test</ns1:storeID>
<ns1:storePwd>test</ns1:storePwd>
</ns1:merchantAccount>
<ns1:merchantRefNum>987564</ns1:merchantRefNum>
<ns1:amount>10.00</ns1:amount>
<ns1:card>
<ns1:cardNum>5442981111111023</ns1:cardNum>
<ns1:cardExpiry>
<ns1:month xsi:type="xsd:int">12</ns1:month>
<ns1:year xsi:type="xsd:int">2010</ns1:year>
</ns1:cardExpiry>
</ns1:card>
<ns1:billingDetails>
<ns1:zip>49444</ns1:zip>
</ns1:billingDetails>
</ns1:ccAuthRequestV1>
</ccPurchase>
</soapenv:Body>
</soapenv:Envelope>



Thanks in advance for any pointers you can give.

Travis May
TollFreeForwarding.com

Tom said...

Travis,

You need to edit the client-config.wsdd to have the sendXsiTypes property set to false.

The server config will only affect server side stuff.

Hope that helps, if not you should contact Adobe support and they should be able to help you.

Martijn said...

Hi there!

Interesting blog :)
I've tried what you have written there.
But I'm still having some trouble with authorisation (Error:401) trying to get the webservices. (WSDL)
It has something to do with SP giving authorization to the CF-server. But how?
I can't understand what the problem exactly is.

I will try to describe our situation:
First Windows Server 2008 running:
Coldfusion8 on IIS
MySQL

Second Windows Server 2008 running:
WSS 3.0 (Windows Sharepoint Services) on IIS

On the first server there's an intranet made in CF. We would like to invoke Sharepoint webservices within the intranet webpage.

I assume you have some (a lot of) experience in letting CF communicate with SP-webservices.
Could you lead me in the right direction?
Verry much appreciated :)

Tom said...

@Martijm - Sharepoint integration is certainly possible with CF. It sounds like other posters may have gotten it to work. I personally have not used Sharepoint web services.

If you are having problems, I would suggest getting the folks in Adobe customer support to give you a hand.

Maddmike said...

FYI, this works for digest authentication as well.

I had to do an implementation of Microsoft Virtual Earth which only allows Digest Authenticaion this is the only way I could get it to work.

Charlie Arehart said...

Hey Tom, I realize this blog entry is quite old, and you've moved on to the Flex team, but if you can spare a minute, you said two things in the content/comments that seem contradictory. I wonder if you can clarify. Or I could be misconstruing things.

In the blog content, you had said, "Once you have this code configured, Axis will use the HTTPClient library for it HTTP needs. Since the HTTPClient library supports NT Authentication, you just set the username/password on the Stub object as you would normally do for (say) Basic Authentication and it will just work."

But then in the comments you say, "If the web service in fact *does* require NT Authentication to get the WSDL (check with a non-MS browser), this is something that ColdFusion doesn't support."

Thanks for any thoughts.

Charlie Arehart said...

Oh, I think I may now realize the distinction. More in a moment, but I do have another question on all this.

You say we can "just set the username/password on the Stub object as you would normally do for (say) Basic Authentication and it will just work"

But what if we don't want to prompt the user for the password? Is there any way we can get the credentials from CF (via IIS, perhaps) to pass along on their behalf to the web service?

As for the previous comment, was your latter comment just about accessing the WSDL, not the rest of the web service? If so, I see suggestions (http://cfsilence.com/blog/client/index.cfm/2008/3/17/ColdFusionSharepoint-Integration--Part-1--Authenticating) that the solution there is to save the WSDL locally and point to that.

But even if that's solved, the key challenge is how to proceed with the windows authentication without prompting the user again for the password. It would be awesome if that's solved, whether in CF 7,8, or 9. Thanks for any thoughts.

Tom said...

@Charlie - getting the WSDL and invoking and operation on the web service are two distinct things.

One the core ColdFusion code does, using the HTTP code used by cfhttp.

The other, the Axis engine that powers the CF web service client does, which can be configured to use the Apache HTTPClient library to do its stuff.

Hope that helps.

Tom said...

@Charlie - You always have to have a username and password, no way around it if you are authenticating. Windows NT authentication just uses (virtually) the username and password typed in when the user logged in to Windows.

Since CF is a server, you have to provide the username and password no matter what the auth method. The only solution is to hard code them in the CFML template or prompt someone for the info.

Charlie Arehart said...

Thanks for the reply, Tom. In short, I'll say bummer. :-)

I’d argue, though, that it would seem we wouldn't quite want to say it’s because "CF is a server", per se, since (from what I understand) an ASP.NET app can indeed use the user’s authentication (as provided to get past IIS. I realize a .NET app runs inside the IIS process, which likely explains why it can get to it. Sadly, it just seems CF has no way to get that info. I was just hoping maybe you knew something, as others have asked.

Prompting the user when they have already logged in (assuming the CF App is also behind its own IIS setup requiring windows auth/NTLM) is just something they're wanting to avoid.

People have wanted this for other things, like database queries, etc. Do you think there's ever a chance that CF would be able to get/pass along the authentication? Just seems like it shouldn't be impossible, and that JSP, PHP, Ruby, and other developers would be in the same boat so as to drive/identify a solution.

Or am I missing something obvious that would forever preclude it? :-)

vaalpens said...

Hi Tom,

I have been searching for help on resolving this dreaded 401 error when trying to invoke a webservice. This blog so far seems to be the closest to finding a solution.

Following are some facts regarding this issue:
-The webservice use basic authentication
-I can use cfhttp with id and password to access the wsdl
-I can use wsdl2java.exe -v -U [domain]\[username] -P [password] http://company.net:8080/app/ws/SeviceWS?wsdl to create the java files
-I know I can send a request and receive a response from webservice because the target site was also set up with a non-secure webservice and we were able to communicate (it is on our intranet)

Following are what is not working:
-I can't get to register WS using the CF8 admin. I get error "Error creating web service. Please ensure that you have entered a correct Web Service name or URL"
-When running it from within CF using a cfinvoke and passing the username and password, even passing the username and password in the wsdl2javaarguments parameter, I still get an error. The error I get is:

Cannot generate stub objects for web service invocation.
Name: http://company.net:8080/app/ws/SeviceWS?wsdl. WSDL: http://company.net:8080/app/ws/SeviceWS?wsdl. java.io.IOException: Server returned HTTP response code: 401 for URL: http://company.net:8080/app/ws/SeviceWS?xsd=1

I have been trying different things, even doing a cfhttp request to get the wsdl and then saving it. This then resulted in the "Parsing error: Fatal Error: URI=null Line=3: White spaces are required between publicId and systemId.". I can probably fix this by hand, but that defeats the object of automation.

After reading a lot of blogs and looking at my errors, it seems that the issue is not the http authentication to the original http://company.net:8080/app/ws/SeviceWS?wsdl, but the second request to get the schema http://company.net:8080/app/ws/SeviceWS?xsd=1 is the one that fails. Based on the error it shows that the wsdl was returned but the schema could not be located due to the same basic authentication. This sure sounds to me like ColdFusion is passing the username/password on the wsdl request but not the xsd request.

I would appreciate it if anybody can add some insight to this problem.

Any help will be greatly appreciated.

Thanks
Charl D. Naude

Tom said...

@vaalpens - it sounds like there is a problem retrieving the Schema that is part of the WSDL but is NOT inlined in the WSDL.

Is the WSDL using an include for this? If you can invoke WSDL2Java from the command line and it is able to read the Schema, the ColdFusion code is using the Axis API to do exactly the same thing. There could be a problem with the way CF passes the username/password down in to the stub generation code library that it uses to parse the WSDL - I believe that XML parser will have to execute an HTTP call to get the included Schema. This may be where the credentials are not reaching. Hard to say. But as I said, if you can use the Axis command line to get the WSDL, this should match what CF is doing too.

My recommendation is to get all parts of the WSDL and bring them local to the CF server. You can then register this web service in the CF administrator and use it from your applications. If and when the WSDL changes, you can refresh the service in the admin (after downloading the new WSDL/Schema) and your apps will see the new stuff.

This will remove the "getting WSDL" step from the whole process, which simplifies the whole thing.

Good luck!

vaalpens said...

Hi Tom,

Thanks for the quick response. I really appreciate the insight.

Yes, the WSDL is using an xsd:import to pull in the schema.

Based on your suggestion I have decided to pull down the wsdl and xsd files and put it on my webserver. Following are the steps that I followed:
-Use cfhttp to get wsdl and xsd files and cffile to save it on the webserver
-Manually updated the wsdl file to xsd:import the local xsd file. (I'll probably do the replace during the import/save process at some time)
-I then tried to register the web service through admin or use it with cfinvoke. Both these failed again.
-This is when I realized that my webserver did not know how to handle .wsdl and .xsd files
-On IIS I added the .wsdl and .xsd extensions with MIME type "text/xml". This did the trick.
-I was then able to send my request and receive the response from the web service

Sorry for the above detail but hopefully it will help some other developer searching for solutions.

I probably need to contact Adobe support to find out if they know about this issue.

Thanks again for all the help.

Charl D. Naude

João Fernandes said...

Hey Tom, is there any understandable reason why a webservice trough HTTPSender works and not when switching to CommonsHTTPSender?
With CommonsHTTPSender I get {http://xml.apache.org/axis/}stackTrace:java.net.ConnectException: Connection refused: connect. This happens just by switching the transport value in the client-config.wsdd. I wanted to switch to commons since I'm getting 'faultString: java.net.ConnectException: Connection timed out: connect' when I need to invoke multiple times the same webservice.

Tom said...

João,

I don't know if any reason for the connection to be refused. You could try looking at the network traffic to see what is going on between the two different libraries.

adrian ysk said...

Hi Tom,

I am trying to set the TCP_NODELAY option to ON at the socket level. I found out that only the HTTPClient library in CommonHTTPSender is supporting it. The default HTTPSender does not have the option.

I am setting it with the following command. However, I am not sure whether it is working or not and how do I verify it?

((Stub)test)._setProperty(org.apache.commons.httpclient.params.HttpConnectionParams.TCP_NODELAY, Boolean.TRUE);

Tom said...

@arian - the great thing about Axis and commons-http is that they are open source and you can use a Java debugger to verify that your setting is being recognized.

The other suggestion is to use a network sniffer to monitor the behavior of the socket to see if it is doing what you want.

Graham said...

I tried the solution to resolve issues with the NTLM handshake using AXIS 1.4 and it worked perfectly for me. I edited the client-config.wsdd in the axis.jar file (rather than creating another one somewhere in the classpath) and added a couple of jar files that were required to the classpath (commons-httpclient-3.0.jar and commons-codec-1.3.jar). Then I set the username and password using stub.setUsername and stub.setPassword. Remembering to add the domain at the start of the username separated with a slash. The result - Success :o) Thank You!

Graham said...

BTW

My client was generated using AXIS wsdl2java and I did not have to edit the generated stubs at all.

subhash said...

Hi Tom,

I am trying to edit client-config.wsdd but I am unable to find it in Axis2. Can you tell me where the file can be located?

Thanks in Advance
Regards,
Subhash
subhash.lingampally@gmail.com

Tom said...

@Subhash No, Axis and Axis2 are completely different.

subhash said...

Thanks for your quick reply Tom.

Regards,
Subhash

subhash said...

BTW, I am trying to connect to MSCRM 4.0 using the code above with axis 2, I am getting 401 unauthorized errors. Do you have any clue on this? Quick response is appreciable.

Thanks in Advance.

Regards,
Subhash

Patricio Otamendi said...

Hi Tom, I have to call a web service, but the program will be behind a proxy. I know that whith HttClient I can call any website outside the lan. But with axis I can't make it. The solution that you proppose I can't use it because now commons-httpclient doesn't exists anymore. How can I resolve this problema? Thank a lot, Patricio Otamendi

Tom said...

Patricio - Check out http://hc.apache.org/httpclient-3.x/index.html for where commons-httpclient went and http://hc.apache.org/httpcomponents-client-ga/ for the latest stuff.

You are on your own for the details, but I imagine the class names haven't changed so the instructions should still be helpful.

Feel free to post details here if you have to make any significant changes and I can update the post if needed.

Or you can just get the older version of the jar file, Apache archives all old releases. Should be too hard to find in the archives.

Patricio Otamendi said...

Well, I found the link to commons-httpclient, but it doesn't work as I spected (proxies issues). So, what I have done was to write my own org.apache.axis.handlers.BasicHandler
I extended this class, and implemented a solution using the HttClient (version 4.1.1, wich is the version that I need). Now I want to rebuild my class that extends BasicHandler to do it better. I need to call to two diferent web services, with different URLs. But my solution was to use a static HttpClient, since the class must have a constructor without parameters. Thanks for your advises, Patricio Otamendi

Anonymous said...

The username must contain a \ to get Axis to use 'NTCredentials' instead of 'UsernamePasswordCredentials'.

Using

ws.setUsername("Tom");
ws.setPassword("mysecret");

thows

10/05 15:20:13 [web-4] INFO ntlm authentication scheme selected
10/05 15:20:13 [web-4] ERROR Credentials cannot be used for NTLM authentication: org.apache.commons.httpclient.UsernamePasswordCredentials
org.apache.commons.httpclient.auth.InvalidCredentialsException: Credentials cannot be used for NTLM authentication: org.apache.commons.httpclient.UsernamePasswordCredentials
at org.apache.commons.httpclient.auth.NTLMScheme.authenticate(NTLMScheme.java:332)


Using

ws.setUsername("x\Tom");
ws.setPassword("mysecret");

results in

10/05 15:27:19 [web-11] INFO Failure authenticating with NTLM @www.zzz.no:80

:::SOAP Request:::
. . .
:::SOAP Response:::

html
head
title Access is denied.
title

Iris said...

Hi,
I have the same issue with CommonsHTTPSender as João Fernandes had:
Hey Tom, is there any understandable reason why a webservice trough HTTPSender works and not when switching to CommonsHTTPSender?
With CommonsHTTPSender I get {http://xml.apache.org/axis/}stackTrace:java.net.ConnectException: Connection refused: connect.
I'm interested to know how he solved his issue.
Thanks in advance,
Iris.

Anonymous said...

Hi,

Is there a way to set axis to use CommonsHTTPSender instead of HTTPSender programmatically? (for ex. by changing some propery?)

Thanks..

Iris said...

// get a modifiable configuration and instantiate it with a basic client configuration
config = new SimpleProvider(new BasicClientConfig());
// create a new simple targeted chain
SimpleTargetedChain chain = new SimpleTargetedChain(new CommonsHTTPSender());
// and add it to the configuration
config.deployTransport("http", chain);
// Obtain a service object from service locators
MetadataService mdService = new MetadataServiceLocator(config);
DataIntegrationService diService = new DataIntegrationServiceLocator(config);