Search:  
NamespacesDiscussions.NET v1.1Feedback
Screen scraping POST operation
Messages   Related Types
This message was discovered on ASPFriends.com 'aspngescalate' list.
Responses highlighted in red are from those people who are likely to be able to contribute good, authoratitive information to this discussion. They include Microsoft employees, MVP's and others who IMHO contribute well to these kinds of discussions.

Peter Brunone (VIP)

I've tried asking this on aspngfreeforall and aspngvb *twice* with no
response. I'm following the example at
http://www.aspalliance.com/stevesmith/articles/netscrape.asp and it works
just fine, but I'm trying to convert it to POST... aside from changing the
action, I'm not sure what to do and nobody seems to know. Could it be that
I need to add some parameters to get it going? (it just times out right
now).

<%@ Page debug="true" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script language="VB" runat="server">
Sub Page_Load(Src As Object, E As EventArgs)
Dim strURL As String
strURL = "http://localhost/scraper/sampleform.asp";
myPage.Text = readHtmlPage(strURL)
pageURL.Text = strURL
End Sub

Function readHtmlPage(url As String) As String
Dim objResponse As WebResponse
Dim objRequest As WebRequest
Dim result As String

objRequest = System.Net.HttpWebRequest.Create(url)
objRequest.Method = "POST" ' Added this
objResponse = objRequest.GetResponse()
Dim sr As New StreamReader(objResponse.GetResponseStream())
result = sr.ReadToEnd()
myHeaders.Text = objRequest.Headers.ToString
myMethod.Text = objRequest.Method
'clean up StreamReader
sr.Close()

return result
End Function
</script>
<html>
<body>
URL: <asp:literal id="pageURL" runat="server"/><BR>

<asp:literal id="myPage" runat="server"/><BR>

Method: <asp:literal id="myMethod" runat="server"/><BR>
Headers: <asp:literal id="myHeaders" runat="server"/><BR>
</body>
</html>
Reply to this message...
 
    
Michel Comeau (VIP)
Hi Peter,

I am not the expert but I'll try to explain what I believe is happening,
but I might be completely wrong.

In your code, you change the Method to "POST" but you don't include any
posted data. When you call WebRequest's GetResponse(), an analysis of
the content is done, .NET realizes you didn't post anything and
considers that "GET" is better when nothing is posted.

I think you should try with posting some content. If you need help into
posting content, I will gladly provide an example (not at hand right
now)

Regards,
Michel C.

-----Original Message-----
From: Peter Brunone [mailto:Click here to reveal e-mail address]
Sent: Thursday, August 02, 2001 17:48
To: aspngescalate
Subject: [aspngescalate] Screen scraping POST operation

I've tried asking this on aspngfreeforall and aspngvb *twice* with
no response. I'm following the example at
http://www.aspalliance.com/stevesmith/articles/netscrape.asp and it
works just fine, but I'm trying to convert it to POST... aside from
changing the action, I'm not sure what to do and nobody seems to know.
Could it be that I need to add some parameters to get it going? (it just
times out right now).

<%@ Page debug="true" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script language="VB" runat="server">
Sub Page_Load(Src As Object, E As EventArgs)
Dim strURL As String
strURL = "http://localhost/scraper/sampleform.asp";
myPage.Text = readHtmlPage(strURL)
pageURL.Text = strURL
End Sub

Function readHtmlPage(url As String) As String
Dim objResponse As WebResponse
Dim objRequest As WebRequest
Dim result As String

objRequest = System.Net.HttpWebRequest.Create(url)
objRequest.Method = "POST" ' Added this
objResponse = objRequest.GetResponse()
Dim sr As New StreamReader(objResponse.GetResponseStream())
result = sr.ReadToEnd()
myHeaders.Text = objRequest.Headers.ToString
myMethod.Text = objRequest.Method
'clean up StreamReader
sr.Close()

return result
End Function
</script>
<html>
<body>
URL: <asp:literal id="pageURL" runat="server"/><BR>

<asp:literal id="myPage" runat="server"/><BR>

Method: <asp:literal id="myMethod" runat="server"/><BR>
Headers: <asp:literal id="myHeaders" runat="server"/><BR> </body>
</html>

| [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
Reply to this message...
 
    
Mitch Denny (VIP)
Michel/Peter,

Was just working on a quick and dirty one just then:

    // Declare local instances.
    HttpWebRequest myRequest = null;
    StreamWriter myWriter = null;
    HttpWebResponse myResponse = null;
    StreamReader myReader = null;

    // Construct the request. In my testing I discovered that
    // the Method property was case sensitive. Wierd.
    myRequest =
(HttpWebRequest)HttpWebRequest.Create(@"http://localhost/test.asp";);
    myRequest.Method = "POST";

    // Write some parameters in, that is what is required with
    // any of the methods that push data up to the server.
    myWriter = new StreamWriter(myRequest.GetRequestStream());
    myWriter.Write("parameter1=value1¶meter2=value2");
    myWriter.Close();    

    // HTTP needs this so that it knows when to hang up the
    // connection. The ContentLength header is defined as the
    // number of bytes that come after the headers (I am sure
    // that is not the exact wording in the RFC, but it must
    // be pretty close.
    myRequest.ContentLength = 35;

    // Do the dirty deed.
    myResponse = (HttpWebResponse)myRequest.GetResponse();

    // Attach a reader to it.
    myReader = new StreamReader(myResponse.GetResponseStream());

    // Spew it out to the screen.
    this.Response.Write(myReader.ReadToEnd());

Michel, maybe you can help me. I managed to get this working OK,
but its not picking up my parameters that I have passed. I can't
think why not. I've done this stuff manually in Perl before and it
has worked ... I can't spot what I am doing wrong.

----------------------------------------
- Mitch Denny
- http://www.warbyte.com
- Click here to reveal e-mail address
- +61 (414) 610-141
-

[Original message clipped]
Reply to this message...
 
    
Mitch Denny (VIP)
OK, this is a trace of what it is doing:

    POST /Default.aspx HTTP/1.1
    Content-Length: 35
    Expect: 100-continue
    Connection: Keep-Alive
    Host: localhost

    parameter1=value1¶meter2=value2

Seems OK to me.

----------------------------------------
- Mitch Denny
- http://www.warbyte.com
- Click here to reveal e-mail address
- +61 (414) 610-141
-

[Original message clipped]
Reply to this message...
 
    
Alex Lowe
Mitch,

First, this code will only work when browsing a local page. Not sure why!?

Second, there are few problems with the code you sent (*when you try to
access a remote web server*):
1st compile error: myRequest.ContentLength = 35; generates "This must be set
before content has been written." (or something like that)
2nd compile error: myResponse = (HttpWebResponse)myRequest.GetResponse();
generates "The remote server returned an error: (405) Method Not Allowed. "

The second error is where I have been at for a while.

Alex - Asplists.com Moderation Team
http://www.aspalliance.com/aldotnet/
http://www.asp-grandrapids.net

----- Original Message -----
From: "Mitch Denny" <Click here to reveal e-mail address>
To: "aspngescalate" <Click here to reveal e-mail address>
Sent: Friday, August 03, 2001 10:40 AM
Subject: [aspngescalate] RE: Screen scraping POST operation

| Michel/Peter,
|
| Was just working on a quick and dirty one just then:
|
| // Declare local instances.
| HttpWebRequest myRequest = null;
| StreamWriter myWriter = null;
| HttpWebResponse myResponse = null;
| StreamReader myReader = null;
|
| // Construct the request. In my testing I discovered that
| // the Method property was case sensitive. Wierd.
| myRequest =
| (HttpWebRequest)HttpWebRequest.Create(@"http://localhost/test.asp";);
| myRequest.Method = "POST";
|
| // Write some parameters in, that is what is required with
| // any of the methods that push data up to the server.
| myWriter = new StreamWriter(myRequest.GetRequestStream());
| myWriter.Write("parameter1=value1¶meter2=value2");
| myWriter.Close();
|
| // HTTP needs this so that it knows when to hang up the
| // connection. The ContentLength header is defined as the
| // number of bytes that come after the headers (I am sure
| // that is not the exact wording in the RFC, but it must
| // be pretty close.
| myRequest.ContentLength = 35;
|
| // Do the dirty deed.
| myResponse = (HttpWebResponse)myRequest.GetResponse();
|
| // Attach a reader to it.
| myReader = new StreamReader(myResponse.GetResponseStream());
|
| // Spew it out to the screen.
| this.Response.Write(myReader.ReadToEnd());
|
| Michel, maybe you can help me. I managed to get this working OK,
| but its not picking up my parameters that I have passed. I can't
| think why not. I've done this stuff manually in Perl before and it
| has worked ... I can't spot what I am doing wrong.
|
| ----------------------------------------
| - Mitch Denny
| - http://www.warbyte.com
| - Click here to reveal e-mail address
| - +61 (414) 610-141
| -
|
| > -----Original Message-----
| > From: Michel Comeau [mailto:Click here to reveal e-mail address]
| > Sent: Friday, 3 August 2001 6:11 PM
| > To: aspngescalate
| > Subject: [aspngescalate] RE: Screen scraping POST operation
| >
| >
| > Hi Peter,
| >
| > I am not the expert but I'll try to explain what I believe is
| > happening,
| > but I might be completely wrong.
| >
| > In your code, you change the Method to "POST" but you don't
| > include any
| > posted data. When you call WebRequest's GetResponse(), an analysis of
| > the content is done, .NET realizes you didn't post anything and
| > considers that "GET" is better when nothing is posted.
| >
| > I think you should try with posting some content. If you need
| > help into
| > posting content, I will gladly provide an example (not at hand right
| > now)
| >
| > Regards,
| > Michel C.
| >
| >
| >
| > -----Original Message-----
| > From: Peter Brunone [mailto:Click here to reveal e-mail address]
| > Sent: Thursday, August 02, 2001 17:48
| > To: aspngescalate
| > Subject: [aspngescalate] Screen scraping POST operation
| >
| >
| >
| > I've tried asking this on aspngfreeforall and aspngvb *twice* with
| > no response. I'm following the example at
| > http://www.aspalliance.com/stevesmith/articles/netscrape.asp and it
| > works just fine, but I'm trying to convert it to POST... aside from
| > changing the action, I'm not sure what to do and nobody seems to know.
| > Could it be that I need to add some parameters to get it
| > going? (it just
| > times out right now).
| >
| > <%@ Page debug="true" %>
| > <%@ Import Namespace="System.Net" %>
| > <%@ Import Namespace="System.IO" %>
| > <script language="VB" runat="server">
| > Sub Page_Load(Src As Object, E As EventArgs)
| > Dim strURL As String
| > strURL = "http://localhost/scraper/sampleform.asp";
| > myPage.Text = readHtmlPage(strURL)
| > pageURL.Text = strURL
| > End Sub
| >
| > Function readHtmlPage(url As String) As String
| > Dim objResponse As WebResponse
| > Dim objRequest As WebRequest
| > Dim result As String
| >
| > objRequest = System.Net.HttpWebRequest.Create(url)
| > objRequest.Method = "POST" ' Added this
| > objResponse = objRequest.GetResponse()
| > Dim sr As New StreamReader(objResponse.GetResponseStream())
| > result = sr.ReadToEnd()
| > myHeaders.Text = objRequest.Headers.ToString
| > myMethod.Text = objRequest.Method
| > 'clean up StreamReader
| > sr.Close()
| >
| > return result
| > End Function
| > </script>
| > <html>
| > <body>
| > URL: <asp:literal id="pageURL" runat="server"/><BR>
| >
| > <asp:literal id="myPage" runat="server"/><BR>
| >
| > Method: <asp:literal id="myMethod" runat="server"/><BR>
| > Headers: <asp:literal id="myHeaders" runat="server"/><BR> </body>
| > </html>
| >
| >
| >
| > | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| > | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
| >
| >
| > | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| > | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
| >
|
|
| | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
|
Reply to this message...
 
    
Peter Brunone (VIP)

This is part of my problem; I'm not sure how to get form parameters for
a POST into a request like this. GET parameters are easy...

BTW, since my original posting on ngfreeforall, I have gotten it to
actually use the POST method... but then it just cranks forever and times
out. More evidence, it would seem, that I need parameters... but how to get
them in there?

A Guthrie-style lightning bolt would be appropriate right about now (or
Warren, Howard, etc... you name it, I'll listen)...

----- Original Message -----
From: "Michel Comeau" <Click here to reveal e-mail address>
To: "aspngescalate" <Click here to reveal e-mail address>
Sent: Friday, August 03, 2001 3:10 AM
Subject: [aspngescalate] RE: Screen scraping POST operation

| Hi Peter,
|
| I am not the expert but I'll try to explain what I believe is happening,
| but I might be completely wrong.
|
| In your code, you change the Method to "POST" but you don't include any
| posted data. When you call WebRequest's GetResponse(), an analysis of
| the content is done, .NET realizes you didn't post anything and
| considers that "GET" is better when nothing is posted.
|
| I think you should try with posting some content. If you need help into
| posting content, I will gladly provide an example (not at hand right
| now)
|
| Regards,
| Michel C.
|
|
|
| -----Original Message-----
| From: Peter Brunone [mailto:Click here to reveal e-mail address]
| Sent: Thursday, August 02, 2001 17:48
| To: aspngescalate
| Subject: [aspngescalate] Screen scraping POST operation
|
|
|
| I've tried asking this on aspngfreeforall and aspngvb *twice* with
| no response. I'm following the example at
| http://www.aspalliance.com/stevesmith/articles/netscrape.asp and it
| works just fine, but I'm trying to convert it to POST... aside from
| changing the action, I'm not sure what to do and nobody seems to know.
| Could it be that I need to add some parameters to get it going? (it just
| times out right now).
|
| <%@ Page debug="true" %>
| <%@ Import Namespace="System.Net" %>
| <%@ Import Namespace="System.IO" %>
| <script language="VB" runat="server">
| Sub Page_Load(Src As Object, E As EventArgs)
| Dim strURL As String
| strURL = "http://localhost/scraper/sampleform.asp";
| myPage.Text = readHtmlPage(strURL)
| pageURL.Text = strURL
| End Sub
|
| Function readHtmlPage(url As String) As String
| Dim objResponse As WebResponse
| Dim objRequest As WebRequest
| Dim result As String
|
| objRequest = System.Net.HttpWebRequest.Create(url)
| objRequest.Method = "POST" ' Added this
| objResponse = objRequest.GetResponse()
| Dim sr As New StreamReader(objResponse.GetResponseStream())
| result = sr.ReadToEnd()
| myHeaders.Text = objRequest.Headers.ToString
| myMethod.Text = objRequest.Method
| 'clean up StreamReader
| sr.Close()
|
| return result
| End Function
| </script>
| <html>
| <body>
| URL: <asp:literal id="pageURL" runat="server"/><BR>
|
| <asp:literal id="myPage" runat="server"/><BR>
|
| Method: <asp:literal id="myMethod" runat="server"/><BR>
| Headers: <asp:literal id="myHeaders" runat="server"/><BR> </body>
| </html>
Reply to this message...
 
    
Alex Lowe
Peter,

This is kind of working for me (and Mitch posted something similar a bit a
go):

<%@ Page debug="True" trace="true" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script language="C#" runat="server">
void Page_Load(Object Src, EventArgs E) {
myPage.Text = readHtmlPage("http://www.asp.net/default.aspx";);
}

private String readHtmlPage(string url)
{
String result = "";
String stufftopost = "test1=You&test2=Posted&test3=Successfully";
StreamWriter myWriter = null;

HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
objRequest.Method = "POST";
objRequest.ContentLength = stufftopost.Length;

myWriter = new StreamWriter(objRequest.GetRequestStream());
myWriter.Write(stufftopost);
myWriter.Close();

HttpWebResponse objResponse =
(HttpWebResponse)objRequest.GetResponse();

StreamReader sr = new StreamReader(objResponse.GetResponseStream());
result = sr.ReadToEnd();
sr.Close();

return result;
}
</script>
<html>
<body>
<asp:literal id="myPage" runat="server"/>
</body>
</html>

The only problem is that for some reason the posted values are not available
on the remote server (ie
http://www.aspalliance.com/aldotnet/examples/posttestasp returns a blank
page when it is For looping through any Form variables passed in). I am on a
mission right now to figure out what the POST specs are.

Alex - Asplists.com Moderation Team
http://www.aspalliance.com/aldotnet/
http://www.asp-grandrapids.net

----- Original Message -----
From: "Peter Brunone" <Click here to reveal e-mail address>
To: "aspngescalate" <Click here to reveal e-mail address>
Sent: Friday, August 03, 2001 11:33 AM
Subject: [aspngescalate] RE: Screen scraping POST operation

|
| This is part of my problem; I'm not sure how to get form parameters
for
| a POST into a request like this. GET parameters are easy...
|
| BTW, since my original posting on ngfreeforall, I have gotten it to
| actually use the POST method... but then it just cranks forever and times
| out. More evidence, it would seem, that I need parameters... but how to
get
| them in there?
|
| A Guthrie-style lightning bolt would be appropriate right about now
(or
| Warren, Howard, etc... you name it, I'll listen)...
|

| | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
|
Reply to this message...
 
    
Alex Lowe
Peter,

Mission complete. Here is a WORKING POST SCRAPER!

<%@ Page Language="C#" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script language="C#" runat="server">
void Page_Load(Object Src, EventArgs E) {
myPage.Text = readHtmlPage("http://www.asp.net";);
}

private String readHtmlPage(string url)
{
String result = "";
String urlstring =
@"http://www.aspalliance.com/aldotnet/examples/posttest.asp"" target="_blank">http://www.aspalliance.com/aldotnet/examples/posttest.asp";;
String stufftopost = @"test1=You&test2=Posted&test3=Successfully";
StreamWriter objWriter = null;

HttpWebRequest objRequest =
(HttpWebRequest)WebRequest.Create(urlstring);
objRequest.Method = "POST";
objRequest.ContentLength = stufftopost.Length;
objRequest.ContentType = "application/x-www-form-urlencoded";

objWriter = new StreamWriter(objRequest.GetRequestStream());
objWriter.Write(stufftopost);
objWriter.Close();

HttpWebResponse objResponse =
(HttpWebResponse)objRequest.GetResponse();
StreamReader sr = new StreamReader(objResponse.GetResponseStream());
result = sr.ReadToEnd();
sr.Close();

return result;
}
</script>
<html>
<body>
<asp:literal id="myPage" runat="server"/>
</body>
</html>

Alex - Asplists.com Moderation Team
http://www.aspalliance.com/aldotnet/
http://www.asp-grandrapids.net

| ----- Original Message -----
| From: "Peter Brunone" <Click here to reveal e-mail address>
| To: "aspngescalate" <Click here to reveal e-mail address>
| Sent: Friday, August 03, 2001 11:33 AM
| Subject: [aspngescalate] RE: Screen scraping POST operation
|
|
| |
| | This is part of my problem; I'm not sure how to get form parameters
| for
| | a POST into a request like this. GET parameters are easy...
| |
| | BTW, since my original posting on ngfreeforall, I have gotten it to
| | actually use the POST method... but then it just cranks forever and
times
| | out. More evidence, it would seem, that I need parameters... but how to
| get
| | them in there?
| |
| | A Guthrie-style lightning bolt would be appropriate right about now
| (or
| | Warren, Howard, etc... you name it, I'll listen)...
| |
|
| | | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| | | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
| |
|
|
| | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
|
Reply to this message...
 
    
Alex Lowe
Sorry, I posted an older test page a minute ago. Here is latest and greatest
code:

<%@ Page Language="C#" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script language="C#" runat="server">
void Page_Load(Object Src, EventArgs E) {
myPage.Text =
readHtmlPage("http://www.aspalliance.com/aldotnet/examples/posttest.asp";);
}

private String readHtmlPage(string url)
{
String result = "";
String stufftopost = @"test1=You&test2=Posted&test3=Successfully";
StreamWriter myWriter = null;

HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
objRequest.Method = "POST";
objRequest.ContentLength = stufftopost.Length;
objRequest.ContentType = "application/x-www-form-urlencoded";

myWriter = new StreamWriter(objRequest.GetRequestStream());
myWriter.Write(stufftopost);
myWriter.Close();

HttpWebResponse objResponse =
(HttpWebResponse)objRequest.GetResponse();
StreamReader sr = new StreamReader(objResponse.GetResponseStream());
result = sr.ReadToEnd();
sr.Close();

return result;
}
</script>
<html>
<body>
<asp:literal id="myPage" runat="server"/>
</body>
</html>

Alex

| | [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| | http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
|
Reply to this message...
 
    
Peter Brunone (VIP)
Awesome!

Now help me understand this :)

Lines like

HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);

confuse me. Is this a C# thing? I don't understand the syntax on the right
side, particularly the parens at the beginning of the statement. ???

I know it's been said before, but you guys are the best...

|-----Original Message-----
|From: Alex Lowe [mailto:Click here to reveal e-mail address]
|Sent: Friday, August 03, 2001 12:16 PM
|To: aspngescalate
|Subject: [aspngescalate] RE: Screen scraping POST operation
|
|
|Sorry, I posted an older test page a minute ago. Here is latest
|and greatest
|code:
|
|<%@ Page Language="C#" %>
|<%@ Import Namespace="System.Net" %>
|<%@ Import Namespace="System.IO" %>
|<script language="C#" runat="server">
| void Page_Load(Object Src, EventArgs E) {
| myPage.Text =
|readHtmlPage("http://www.aspalliance.com/aldotnet/examples/posttest.asp";);
| }
|
| private String readHtmlPage(string url)
| {
| String result = "";
| String stufftopost = @"test1=You&test2=Posted&test3=Successfully";
| StreamWriter myWriter = null;
|
| HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
| objRequest.Method = "POST";
| objRequest.ContentLength = stufftopost.Length;
| objRequest.ContentType = "application/x-www-form-urlencoded";
|
| myWriter = new StreamWriter(objRequest.GetRequestStream());
| myWriter.Write(stufftopost);
| myWriter.Close();
|
| HttpWebResponse objResponse =
|(HttpWebResponse)objRequest.GetResponse();
| StreamReader sr = new StreamReader(objResponse.GetResponseStream());
| result = sr.ReadToEnd();
| sr.Close();
|
| return result;
| }
|</script>
|<html>
|<body>
|<asp:literal id="myPage" runat="server"/>
|</body>
|</html>
Reply to this message...
 
    
Andy Smith (VIP)
parens are the C# casting operator.

Andy Smith
Staff Programmer
Harding ESE

    -----Original Message-----
        From:    Click here to reveal e-mail address=20
Sent:    Friday, August 03, 2001 2:52 PM
To:    "aspngescalate" <Click here to reveal e-mail address>
Subject:    [aspngescalate] RE: Screen scraping POST operation
=09
Awesome!

Now help me understand this :)

Lines like

HttpWebRequest objRequest =3D (HttpWebRequest)WebRequest.Create(url);

confuse me. Is this a C# thing? I don't understand the syntax on the =
right
side, particularly the parens at the beginning of the statement. ???

I know it's been said before, but you guys are the best...

|-----Original Message-----
|From: Alex Lowe [mailto:Click here to reveal e-mail address]
|Sent: Friday, August 03, 2001 12:16 PM
|To: aspngescalate
|Subject: [aspngescalate] RE: Screen scraping POST operation
|
|
|Sorry, I posted an older test page a minute ago. Here is latest
|and greatest
|code:
|
|<%@ Page Language=3D"C#" %>
|<%@ Import Namespace=3D"System.Net" %>
|<%@ Import Namespace=3D"System.IO" %>
|<script language=3D"C#" runat=3D"server">
| void Page_Load(Object Src, EventArgs E) {
| myPage.Text =3D
|readHtmlPage("http://www.aspalliance.com/aldotnet/examples/posttest.asp";);=

| }
|
| private String readHtmlPage(string url)
| {
| String result =3D "";
| String stufftopost =3D @"test1=3DYou&test2=3DPosted&test3=3DSuccessf=
ully";
| StreamWriter myWriter =3D null;
|
| HttpWebRequest objRequest =3D (HttpWebRequest)WebRequest.Create(url)=
;
| objRequest.Method =3D "POST";
| objRequest.ContentLength =3D stufftopost.Length;
| objRequest.ContentType =3D "application/x-www-form-urlencoded";
|
| myWriter =3D new StreamWriter(objRequest.GetRequestStream());
| myWriter.Write(stufftopost);
| myWriter.Close();
|
| HttpWebResponse objResponse =3D
|(HttpWebResponse)objRequest.GetResponse();
| StreamReader sr =3D new StreamReader(objResponse.GetResponseStream()=
);
| result =3D sr.ReadToEnd();
| sr.Close();
|
| return result;
| }
|</script>
|<html>
|<body>
|<asp:literal id=3D"myPage" runat=3D"server"/>
|</body>
|</html>

| [aspngescalate] member Click here to reveal e-mail address =3D YOUR ID
| http://www.asplists.com/asplists/aspngescalate.asp =3D JOIN/QUIT
Reply to this message...
 
    
G. Andrew Duthie (VIP)
The parens are used to explicitly case the result of
WebRequest.Create(url) to the HttpWebRequest type. So here's how
it breaks down:

HttpWebRequest [defined class (type) of the variable]
[space]
objRequest [name of the object instance]
[space]=[space]
(HttpWebRequest) [cast to HttpWebRequest]
WebRequest.Create(url); [create a WebRequest from the specified
URL]

hth,
--
G. Andrew Duthie
Graymad Enterprises
Click here to reveal e-mail address

[Original message clipped]
Reply to this message...
 
    
Michel Comeau
I believe that in VB they have to use CType()

Michel C.

-----Original Message-----
From: Andy Smith [mailto:Click here to reveal e-mail address]
Sent: Friday, August 03, 2001 5:15 PM
To: aspngescalate
Subject: [aspngescalate] RE: Screen scraping POST operation

parens are the C# casting operator.

Andy Smith
Staff Programmer
Harding ESE

    -----Original Message-----
        From:    Click here to reveal e-mail address
Sent:    Friday, August 03, 2001 2:52 PM
To:    "aspngescalate" <Click here to reveal e-mail address>
Subject:    [aspngescalate] RE: Screen scraping POST operation

Awesome!

Now help me understand this :)

Lines like

HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);

confuse me. Is this a C# thing? I don't understand the syntax on the right
side, particularly the parens at the beginning of the statement. ???

I know it's been said before, but you guys are the best...

|-----Original Message-----
|From: Alex Lowe [mailto:Click here to reveal e-mail address]
|Sent: Friday, August 03, 2001 12:16 PM
|To: aspngescalate
|Subject: [aspngescalate] RE: Screen scraping POST operation
|
|
|Sorry, I posted an older test page a minute ago. Here is latest
|and greatest
|code:
|
|<%@ Page Language="C#" %>
|<%@ Import Namespace="System.Net" %>
|<%@ Import Namespace="System.IO" %>
|<script language="C#" runat="server">
| void Page_Load(Object Src, EventArgs E) {
| myPage.Text =
|readHtmlPage("http://www.aspalliance.com/aldotnet/examples/posttest.asp";);
| }
|
| private String readHtmlPage(string url)
| {
| String result = "";
| String stufftopost = @"test1=You&test2=Posted&test3=Successfully";
| StreamWriter myWriter = null;
|
| HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
| objRequest.Method = "POST";
| objRequest.ContentLength = stufftopost.Length;
| objRequest.ContentType = "application/x-www-form-urlencoded";
|
| myWriter = new StreamWriter(objRequest.GetRequestStream());
| myWriter.Write(stufftopost);
| myWriter.Close();
|
| HttpWebResponse objResponse =
|(HttpWebResponse)objRequest.GetResponse();
| StreamReader sr = new StreamReader(objResponse.GetResponseStream());
| result = sr.ReadToEnd();
| sr.Close();
|
| return result;
| }
|</script>
|<html>
|<body>
|<asp:literal id="myPage" runat="server"/>
|</body>
|</html>

| [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT

| [aspngescalate] member Click here to reveal e-mail address = YOUR ID
| http://www.asplists.com/asplists/aspngescalate.asp = JOIN/QUIT
Reply to this message...
 
    
Mitch Denny (VIP)
Alex,

Ding Ding Ding! We have a winner! Well done, I knew it must have been
something simple like that (Content-type). I know why I couldn't find
it in the Perl code.

Another developer had taken it and worked the code and put all the
different things in sub-routines, so that the parameter encoding wasn't
in the same place as the setting of the content-type.

----------------------------------------
- Mitch Denny
- http://www.warbyte.com
- Click here to reveal e-mail address
- +61 (414) 610-141
-

[Original message clipped]
Reply to this message...
 
    
Mitch Denny (VIP)
Actually, that is something I've never liked, with VB I was
hoping that they would do something about that. But I guess
VB is more "function rich" than C#, so if thats their model
they should stick with hit.

----------------------------------------
- Mitch Denny
- http://www.warbyte.com
- Click here to reveal e-mail address
- +61 (414) 610-141
-

[Original message clipped]
Reply to this message...
 
 
System.EventArgs
System.IO.StreamReader
System.IO.StreamWriter
System.Net.HttpWebRequest
System.Net.HttpWebResponse
System.Net.WebRequest
System.Net.WebResponse
Ad
MBR BootFX
Best-of-breed application framework for .NET projects, developed by Matthew Baxter-Reynolds and MBR IT
 
 Copyright © Matthew Baxter-Reynolds 2001-2008. '.NET 247 Software Development Services' is a trading style of MBR IT Solutions Ltd.
Contact Us - Terms of Use - Privacy Policy - www.dotnet247.com