python - using requests to login to a website that has javascript login form -

let me preface saying have little programming experience. i've learned bunch in last few days trying write program. running python 2.7 on windows 7 using pycharm, requests, beautiful soup, , lxml.

i trying scrape data website relies heavily on javascript. have 2 options:

1) data need populated through javascript , not need login. have not been able figure how @ data. i've live monitored headers live http headers chrome plugin , think i've found javascript i'ts beyond means figure out. long bit of code, i'll post if interested in taking look.

or

2)on 1 of main pages found series of id numbers can use generate url's each of individual items analyzing. problem have logged in see these individual item pages. code follows:

from requests.adapters import httpadapter requests.packages.urllib3.poolmanager import poolmanager beautifulsoup import beautifulsoup import ssl  # request date user udate = "06/22/2015"  # raw_input('enter date mm/dd/yyyy\n')  # open tlsv1 adapter (whataver means) class myadapter(httpadapter):     def init_poolmanager(self, connections, maxsize, block=false):         self.poolmanager = poolmanager(num_pools=connections,                                        maxsize=maxsize,                                        block=block,                                        ssl_version=ssl.protocol_tlsv1)  # begin requests session. every here on out use tlsv1 protocol import requests  payload = {     'logname': 'xxxxxxxx',     'logpass': 'xxxxxxxx' }  s = requests.session() s.mount('https://xxxx.xxx', myadapter())  # login post , request source code main page. log = s.post('loginurl', data=payload) print log.text  result = s.get(url) soup = beautifulsoup(result.content) print soup 

neither post or show me logged in website. logform id's html source code this:

<div id="divlogform">         <label for="badtext"><div id="badtext" class="badtext" style="display:none" tabindex="-2">user name or password invalid</div></label>          <div class="loglabel">             <label for="logname" > user name&nbsp;&nbsp;</label><input tabindex="0" id="logname" class="loginput" value="" />         </div>         <div  class="loglabel">             <label for="logpass" >user password&nbsp;&nbsp;</label><input  tabindex="0"id="logpass" type="password" class="loginput" value="" />         </div> 

so i'm passing logname , logpass post.

there logform.js bit of code

$("#logbutton").click(function()         {   //$('#divlogform').hide();             //$('#divloading').show();               var uname = $("#logname").val();            var upass = $("#logpass").val();            var url = "/index.cfm";            $.post(url, {zaction:'ajax',zmethod:'login',func:'login',username:uname, userpass:upass},                    function(data){if (data.isok =="yes"){location.href="/index.cfm";}                                   else {$('.badtext').show(); $('#badtext').focus();};                                  },"json");         }); 

the loginurl in code taken var url in script. have tried using username & userpass , have tried uname , upass post these didnt work either.

not sure how move forward here. appreciated

the last bit of javascript posted gives clue why login post request isn't working.

according javascript, should sending dictionary looks following login post:

{     'zaction': 'ajax',     'zmethod': 'login',     'func': 'login',     'username': '<enter username>',     'userpass': '<enter password>' },