Thursday, February 24, 2011

Download web content from java program



Downloading the content from java program might be easy. Here is one more example with proxy settings.

I am using htmlUnit.jar and other jars to get through.

import java.io.IOException;

import org.xml.sax.SAXException;

import com.meterware.httpunit.GetMethodWebRequest;
import com.meterware.httpunit.WebConversation;
import com.meterware.httpunit.WebForm;
import com.meterware.httpunit.WebRequest;
import com.meterware.httpunit.WebResponse;

public class start {

public static void main(String[] args) throws IOException, SAXException {
WebConversation conversation = new WebConversation();
conversation.setProxyServer("myproxy.proxy.com", 9087,"domain\\username" ,"password");


WebRequest request = new GetMethodWebRequest("http://www.google.com");

WebResponse response = conversation.getResponse(request);

System.out.println(response.getText());

}
}

output prints the google's page with all text as of page source.

Thanks for reading. Bye !!

No comments:

Post a Comment