#StackBounty: #python #python-3.x #web-scraping #python-requests Unable to scrape a piece of static information from a webpage

Bounty: 50

I’ve created a script in python to log in a webpage using credentials and then parse a piece of information SIGN OUT from another link (the script is supposed to get redirected to that link) to make sure I did log in.

Website address

I’ve tried with:

import requests
from bs4 import BeautifulSoup

url = "https://member.angieslist.com/gateway/platform/v1/session/login"
link = "https://member.angieslist.com/"

payload = {"identifier":"usename","token":"password"}

with requests.Session() as s:
    s.post(url,json=payload,headers={
        "User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36",
        "Referer":"https://member.angieslist.com/member/login",
        "content-type":"application/json"
        })
    r = s.get(link,headers={"User-Agent":"Mozilla/5.0"},allow_redirects=True)
    soup = BeautifulSoup(r.text,"lxml")
    login_stat = soup.select_one("button[class*='menu-item--account']").text
    print(login_stat)

When i run the above script, I get AttributeError: 'NoneType' object has no attribute 'text' this error which means I went somewhere wrong in my log in process as the information I wish to parse SIGN OUT is a static content.

How can I parse this SIGN OUT information from that webpage?


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.