[ale] Perl https download

Robert L. Harris Robert.L.Harris at rdlg.net
Thu Jul 26 08:17:19 EDT 2007



I tried changing the initial "LoginURI" to "IndexURI" which is the first real
peice of data I need.  No go, I get this twice (didn't comment out the second
"get":

$content :<h4>Not logged in</h4><p>You must be logged in to view that page.</p>:

  I tried changing "$request = HTTP::Request->new(GET => $IndexURI);" for some other
options but none of them are working.  Looking at cpan 
(http://search.cpan.org/~gaas/libwww-perl-5.806/lib/LWP/UserAgent.pm) I tried :

$response=$ua->get($IndexURI);

to:

# Second page
$response=$ua->get($IndexURI);
$content=$response->content;
print $response->content;  # or whatever


The first page still succeeds, the second still says I need to log in.





Thus spake Tim Watts (timtw at earthlink.net):

> Hi again,
> 
> I'm not an LWP/HTTP perl expert but I think what /may/ be going on is that the 
> script is wiping out the Authorization header that gets set on the $request 
> object when the second call to HTTP::Request->new() is made. 
> 
> Try this instead: rather than creating a new Request object, just change the 
> target URI and then submit the request. Alternatively, get the Authorization 
> header from the original Request object, create a new one and set the header 
> on the new Request object.
> 
> 
> On Wednesday 25 July 2007 9:35 pm, Robert L. Harris wrote:
> > That's basically what I've come up with so far:
> >
> > #!/usr/bin/perl -w
> >
> > use LWP::UserAgent;
> > use HTTP::Cookies;
> > use LWP::Simple;
> > use HTTP::Request;
> > use strict;
> >
> > #
> > my ($LoginURI)="https://www.datafeed.org";
> > my ($IndexURI)="https://www.datafeed.org/internal/querry?range=3day";
> > my ($User)="myuser";
> > my ($Passwd)="mypasswd";
> >
> >
> > # Load LWP class for "cookie jar" objects
> > my $ua = LWP::UserAgent->new( );
> > my $cookie_jar = HTTP::Cookies->new(file => "lwp.cookie", autosave => 1);
> > $ua->cookie_jar( $cookie_jar );
> > $cookie_jar->save;
> >
> > # define user agent
> > $ua = LWP::UserAgent->new();
> > $ua->agent("USER/AGENT/IDENTIFICATION");
> >
> >
> > # make request
> > my $request = HTTP::Request->new(GET => $LoginURI);
> >
> > # authenticate
> > $request->authorization_basic($User, $Passwd);
> >
> > # except response
> > my $response = $ua->request($request);
> > my $content=$response->content;
> >
> > print "\$content :$content:\n";
> >
> >
> >
> >
> > # Get Second Page, Index to the data
> > $request = HTTP::Request->new(GET => $IndexURI);
> > # except response
> > $response = $ua->request($request);
> > $content=$response->content;
> >
> > print "\$content :$content:\n";
> >
> >
> >
> >
> > The first (LoginURI) works and I get logged in and the view of an internal
> > page.  The second part (IndexURI) responds:
> >
> > $content :<h4>Not logged in</h4><p>You must be logged in to view that
> > page.</p>:
> >
> > So it seems to not be passing auth or a cookie.  I tried putting the
> > authorization_basic right after the second request command but no change.
> >
> > Thus spake Christopher Fowler (cfowler at outpostsentinel.com):
> > > This is an example.  Use it to get a cookie that is valid for the
> > > website then go grab whatever you need
> > >
> > > my $ua = LWP::UserAgent->new();
> > > $ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
> > > my $response = $ua->post("$proto://$ARGV[0]/cgi-bin/login",
> > >     { 'user' => "root", 'pass' => "password" });
> > >
> > >
> > > _______________________________________________
> > > Ale mailing list
> > > Ale at ale.org
> > > http://www.ale.org/mailman/listinfo/ale
> > >
> > :wq!
> >
> > ---------------------------------------------------------------------------
> > Robert L. Harris                     | GPG Key ID: E344DA3B
> >                                          @ x-hkp://pgp.mit.edu
> > DISCLAIMER:
> >       These are MY OPINIONS             With Dreams To Be A King,
> >        ALONE.  I speak for              First One Should Be A Man
> >        no-one else.                       - Manowar
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://www.ale.org/mailman/listinfo/ale

:wq!
---------------------------------------------------------------------------
Robert L. Harris                     | GPG Key ID: E344DA3B
                                         @ x-hkp://pgp.mit.edu
DISCLAIMER:
      These are MY OPINIONS             With Dreams To Be A King,
       ALONE.  I speak for              First One Should Be A Man
       no-one else.                       - Manowar

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature




More information about the Ale mailing list