[ale] Perl https download

[Tim Watts]

Hi,

Um, no. You still need to perform the login request. AFTER the login request, 
here's what you need to do:

# ...after the login, the $request object should contain an 
# Authorization header... (if my theory is correct :-)

# re-use the same request object, just change the URI
# thus preserving the Authorization header.
$request->setURI($IndexURI); # WARNING: PSEUDO-CODE!!!

# Now get the page you want:
$response = $ua->request($request)
...

If this approach doesn't work, try doing what I suggested in my 1st response 
to your problem. Good Luck!

On Thursday 26 July 2007 8:17 am, Robert L. Harris wrote:
> I tried changing the initial "LoginURI" to "IndexURI" which is the first
> real peice of data I need.  No go, I get this twice (didn't comment out the
> second "get":
>
> $content :<h4>Not logged in</h4><p>You must be logged in to view that
> page.</p>:
>
>   I tried changing "$request = HTTP::Request->new(GET => $IndexURI);" for
> some other options but none of them are working.  Looking at cpan
> (http://search.cpan.org/~gaas/libwww-perl-5.806/lib/LWP/UserAgent.pm) I
> tried :
>
> $response=$ua->get($IndexURI);
>
> to:
>
> # Second page
> $response=$ua->get($IndexURI);
> $content=$response->content;
> print $response->content;  # or whatever
>
>
> The first page still succeeds, the second still says I need to log in.
>
> Thus spake Tim Watts (timtw at earthlink.net):
> > Hi again,
> >
> > I'm not an LWP/HTTP perl expert but I think what /may/ be going on is
> > that the script is wiping out the Authorization header that gets set on
> > the $request object when the second call to HTTP::Request->new() is made.
> >
> > Try this instead: rather than creating a new Request object, just change
> > the target URI and then submit the request. Alternatively, get the
> > Authorization header from the original Request object, create a new one
> > and set the header on the new Request object.
> >
> > On Wednesday 25 July 2007 9:35 pm, Robert L. Harris wrote:
> > > That's basically what I've come up with so far:
> > >
> > > #!/usr/bin/perl -w
> > >
> > > use LWP::UserAgent;
> > > use HTTP::Cookies;
> > > use LWP::Simple;
> > > use HTTP::Request;
> > > use strict;
> > >
> > > #
> > > my ($LoginURI)="https://www.datafeed.org";
> > > my ($IndexURI)="https://www.datafeed.org/internal/querry?range=3day";
> > > my ($User)="myuser";
> > > my ($Passwd)="mypasswd";
> > >
> > >
> > > # Load LWP class for "cookie jar" objects
> > > my $ua = LWP::UserAgent->new( );
> > > my $cookie_jar = HTTP::Cookies->new(file => "lwp.cookie", autosave =>
> > > 1); $ua->cookie_jar( $cookie_jar );
> > > $cookie_jar->save;
> > >
> > > # define user agent
> > > $ua = LWP::UserAgent->new();
> > > $ua->agent("USER/AGENT/IDENTIFICATION");
> > >
> > >
> > > # make request
> > > my $request = HTTP::Request->new(GET => $LoginURI);
> > >
> > > # authenticate
> > > $request->authorization_basic($User, $Passwd);
> > >
> > > # except response
> > > my $response = $ua->request($request);
> > > my $content=$response->content;
> > >
> > > print "\$content :$content:\n";
> > >
> > >
> > >
> > >
> > > # Get Second Page, Index to the data
> > > $request = HTTP::Request->new(GET => $IndexURI);
> > > # except response
> > > $response = $ua->request($request);
> > > $content=$response->content;
> > >
> > > print "\$content :$content:\n";
> > >
> > >
> > >
> > >
> > > The first (LoginURI) works and I get logged in and the view of an
> > > internal page.  The second part (IndexURI) responds:
> > >
> > > $content :<h4>Not logged in</h4><p>You must be logged in to view that
> > > page.</p>:
> > >
> > > So it seems to not be passing auth or a cookie.  I tried putting the
> > > authorization_basic right after the second request command but no
> > > change.
> > >
> > > Thus spake Christopher Fowler (cfowler at outpostsentinel.com):
> > > > This is an example.  Use it to get a cookie that is valid for the
> > > > website then go grab whatever you need
> > > >
> > > > my $ua = LWP::UserAgent->new();
> > > > $ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
> > > > my $response = $ua->post("$proto://$ARGV[0]/cgi-bin/login",
> > > >     { 'user' => "root", 'pass' => "password" });
> > > >
> > > >
> > > > _______________________________________________
> > > > Ale mailing list
> > > > Ale at ale.org
> > > > http://www.ale.org/mailman/listinfo/ale
> > > >
> > > :wq!
> > >
> > > -----------------------------------------------------------------------
> > >---- Robert L. Harris                     | GPG Key ID: E344DA3B
> > >                                          @ x-hkp://pgp.mit.edu
> > > DISCLAIMER:
> > >       These are MY OPINIONS             With Dreams To Be A King,
> > >        ALONE.  I speak for              First One Should Be A Man
> > >        no-one else.                       - Manowar
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://www.ale.org/mailman/listinfo/ale
> >
> :wq!
>
> ---------------------------------------------------------------------------
> Robert L. Harris                     | GPG Key ID: E344DA3B
>                                          @ x-hkp://pgp.mit.edu
> DISCLAIMER:
>       These are MY OPINIONS             With Dreams To Be A King,
>        ALONE.  I speak for              First One Should Be A Man
>        no-one else.                       - Manowar