Python Coders...Msaada

[Python Newbie]

Ni vizuri uwe na tool inaitwa Postman kama unapenda kuscrape, itakurahisishia sana kumake http requests.

Haya kuhusu swala lako, hiyo site nimejaribu kwenye postman naona tatizo wanahisi wewe ni bot kama jamaa juu hapo alivyosema so wanakupush kwenye captcha page. Kuweza kuvuka hiyo ukiweka "User-Agent" kwenye headers kwanza inagoma coz wanataka more information user-agent peke yake haitoshi. Nilizoweka zikakubali ni hizi

user-agent : Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36

origin : https://www.bloomberg.com

referer: https://www.bloomberg.com/quote/SPX:IND

cookie : <cookie from chrome browser>

[CODE]

Cookie hiyo unaipata kwenye chrome browser, visit hiyo site in the browser, right click nenda inspect, fungua network, refresh page then uclick the first link uangalie Headers walizotuma, kuna cookie pale, include hiyo kwenye header basi, watadetect kua you are human from there. Kazi kwisha problem solved.

Hiyo cookie hujui lini ita-expire, so iki-expire probably ukihitaji kuscrape itagoma, so tafuta jinsi ya kupata cookie jar na kurefresh ujue wanavyoigenerate. Au njia nyingine ndefu unaweza tumia libraries ambazo zinascrape kwa kujifanya browser inakua na implementations zote done for you.

Asante sana kwa hii shule