Jak si stahnout www stranku (vyřešeno)

Dobry den,

chtel bych si stahnout command line prikazem www stranku. Vsude jinde se mi to dari, jen jedna krabicka od Ubiquity odolava.

Nechapu proc je to takhle slozite, ale krabicka vyzaduje zadat Content-Type: multipart/form-data; a to wget neumi. Stary pristup --user --password nepodporuje.

Alespon jsem toto odsnifoval pri prohlizeni z Firefoxu?


POST /login.cgi HTTP/1.1
Host: 10.23.204.242
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.20) Gecko/20081217 
Firefox/2.0.0.20
Accept: 
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,
*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://10.23.204.242/login.cgi?uri=/
Cookie: AIROS_SESSIONID=f4e8fe875111857565e5b487e8c8a5d9; ui_language=en_US
Content-Type: multipart/form-data; boundary=---------------------------129763137732512
Content-Length: 451

-----------------------------129763137732512
Content-Disposition: form-data; name="uri"

/
-----------------------------129763137732512
Content-Disposition: form-data; name="username"

jmeno
-----------------------------129763137732512
Content-Disposition: form-data; name="password"

heslo
-----------------------------129763137732512
Content-Disposition: form-data; name="Submit"

Login
-----------------------------129763137732512--

Nemate nejaky napad, jak ze skriptu stahovat podobne zaheslovane stranky.

Kolacek se vyrobit da:

xxx:/home/f/monitor# wget --load-cookies /tmp/cookies.txt --keep-session-cookies --save-cookies /tmp/cookies.txt --post-file /etc/nakrm.txt "http://10.23.204.242/login.cgi?uri=/stainfo.cgi" -O /tmp/login.cgi

Ale korektne zalogovat se mi nedari.

dekuji Jara

Odpovědi

To je přihlášení přes webový formulář. Ten můžete s wgetem poslat pomocí parametru --post-data nebo --post-file. Pokud ovšem ten server zpracuje i data ve formátu application/x-www-form-urlencoded, protože alespoň podle mé manuálové stránky wget neumí multipart/form-data. Pokud server opravdu vyžaduje multipart/form-data (nechápu proč, není tam upload souboru), pak zkuste třeba curl.

28.3.2010 16:26 JF | skóre: 23
Rozbalit Rozbalit vše Re: Jak si stahnout www stranku

Wgetem jsem take zkousel --post-file. Kdyby se dalo k --pst-file pridat jeste neco jako nepridavej hlavicku, tak by to slo.

Jenze hlavicka z wgetu dopadne takhle spatne. A bohuzel ji zarizeni nesezere.

POST /login.cgi HTTP/1.0
User-Agent: Wget/1.11.4
Accept: */*
Host: 10.23.204.242
Connection: Keep-Alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 1009

Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Referer: http://10.23.204.242/login.cgi?uri=/index.cgi
Accept-Language: cs
Content-Type: multipart/form-data; boundary=---------------------------7da2ce24280476
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) )
Host: 10.23.204.242
Content-Length: 455
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: ui_language=en_US; AIROS_SESSIONID=3d872d807e697be4eaa2e26c2bfbcbcf

-----------------------------7da2ce24280476
Content-Disposition: form-data; name="uri"

/index.cgi
-----------------------------7da2ce24280476
Content-Disposition: form-data; name="username"

jmeno
-----------------------------7da2ce24280476
Content-Disposition: form-data; name="password"

heslo
-----------------------------7da2ce24280476
Content-Disposition: form-data; name="Submit"

Login
-----------------------------7da2ce24280476--

28.3.2010 16:59 Filip Jirsák | skóre: 68 | blog: Fa & Bi
Rozbalit Rozbalit vše Re: Jak si stahnout www stranku

Hlavičky nemůžete mít na začátku souboru, ty musíte poslat parametrem --header.

28.3.2010 18:11 JF | skóre: 23
Rozbalit Rozbalit vše Re: Jak si stahnout www stranku

Dekuji moc, uz se podarilo stranku nacist, ale bohuzel jen z Windows :(. Tam mam wget 1.11.4

wget --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt --header="Content-Type: multipart/form-data; boundary=---------------------------7da2ce24280476" --post-file=nakrm.txt "http://10.23.204.242/login.cgi?/stainfo.cgi"

wget --keep-session-cookies --load-cookies cookies.txt

V Linuxu mam 1.11.1 a ten stranku ne a ne nacist. Ale alespon jsem se dostal o krok dal.

28.3.2010 16:58 JF | skóre: 23
Rozbalit Rozbalit vše Re: Jak si stahnout www stranku

Diky za namet pouzit curl. Ale take mi to nejak nejde.

rodga:/home/fojtik/monitor# curl -b/tmp/cookie.txt --anyauth -F "uri=/;username=jmeno;password=heslo;Submit=Login" "http://10.23.204.242/login.cgi"


< ? xml version="1.0" encoding="iso-8859-1"?>
< ! DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
< html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 < head>
  < title>417 - Expectation Failed< /title>
 < /head>
 < body>
  417 - Expectation Failed
 < /body>
< /html>

Uz jsem si zkompiloval tu samou verzi wget i pro Linux.

Nechapu, proc mi www server odpovida na ten samy prikaz ve Windows 302 Found to je asi spravne a v Linuxu 200 OK.

M:\rodga>wget --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt --header="Content-Type: multipart/fo
m-data; boundary=---------------------------7da2ce24280476" --post-file nakrm_bullet.txt "http://10.23.204.242/login.cgi?/stain
o.cgi" -O login.cgi
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = c:/progra~1/wget/etc/wgetrc
--2010-03-28 21:43:56--  http://10.23.204.242/login.cgi?/stainfo.cgi
Connecting to 10.23.204.242:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /stainfo.cgi [following]
--2010-03-28 21:43:56--  http://10.23.204.242/stainfo.cgi
Connecting to 10.23.204.242:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `login.cgi'
    [ <=> ] 3á129       --.-K/s   in 0s
2010-03-28 21:43:56 (115 MB/s) - `login.cgi' saved [3129]

A tohle je linux:

./wget --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt --header="Content-Type: multipart/form-data; boundary=---------------------------7da2ce24280476" --post-file nakrm_bullet.txt "http://10.23.204.242/login.cgi?/stainfo.cgi" -O login.cgi
--2010-03-28 21:42:00--  http://10.23.204.242/login.cgi?/stainfo.cgi
Connecting to 10.23.204.242:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `login.cgi'
     0K .         31.5M=0s
2010-03-28 21:42:00 (31.5 MB/s) - `login.cgi' saved [1859]

Řešení 1× (JF (tazatel))

28.3.2010 23:13 JF | skóre: 23
Rozbalit Rozbalit vše Re: Mam v Linuxu stejnou verzi wget jako ve Windows a nejde to

No jo, v Linuxu chtela hlavicka CR a ve Windows CR/LF.

Uz to jde.

Dotaz: Jak si stahnout www stranku

Odpovědi

417 - Expectation Failed