How to login into the Fidelity website and navigate within it?

Question

I would love to get some help in logging into the Fidelity website and navigate within it. My attempts so far have not led me to anywhere significant. So here is the code that I have written, after much consultation with answers around the web. The steps are:

Login to Fidelity
Check if response code not 200, but is 302 or 303 and my code passes this test (with a code of 302).
Then I check the number of cookies returned (there were 5) and for each cookie I try to navigate to a different web page within Fidelity (I do this five times, once for each cookie, simply because I do not know which subscript "j" of the variable "cookie" will work).

function loginToFidelity(){
  var url = "https://www.fidelity.com"; 
  var payload = {
    "username":"*********", 
    "password":"*********" 
  }; 
  var opt = {
    "payload":payload,"method":"post","followRedirects" : false
  };
  var response = UrlFetchApp.fetch(encodeURI(url),opt);
  if ( response.getResponseCode() == 200 ) { 
    Logger.log("Couldn't login.");
    return
  } 
  else if (response.getResponseCode() == 303 || response.getResponseCode() == 302) {
    Logger.log("Logged in successfully. " + response.getResponseCode());
    var cookie = response.getAllHeaders()['Set-Cookie']
    for (j = 0; j < cookie.length; j++) {
      var downloadPage = UrlFetchApp.fetch("https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio#activity", 
          {"Cookie" : cookie[j],"method" : "post","followRedirects" : false,"payload":payload});
      Logger.log(downloadPage.getResponseCode())
      Logger.log(downloadPage.getContentText())
    }
  }
}

For each choice of the subscript "j", I get the same answer for the ResponseCode (always 302) as well as the same answer for ContentText. The answer for ContentText is obviously incorrect as it is not what it is supposed to be. The ContentText is shown below:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://login.fidelity.com/ftgw/Fidelity/RtlCust/Login/Init?AuthRedUrl=https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio">here</a>.</p>
</body></html>

Based on this, I have two questions:

Have I logged into the Fidelity site correctly? If not, why do I get a response code of 302 in the login process? What do I need to do differently to login correctly?
Why am I getting such a strange and obviously incorrect answer for my ContentText while getting a perfectly reasonable ResponseCode of 302? What do I need to do differently, so that I can get the password-controlled page within Fidelity, whose url is "https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio#activity"?

NOTE: Some other tests have been done in addition to the one stated above. Results from these tests are provided in the discussion below.

I have tried this code with "followRedirects" : true. It produces a ResponseCode of 200 and ContextText contains the HTML for a page which is useless (title is "Login to Fidelity Investments"). I get the exact same answer for all five values of j. — McSquare, Sep 06 '18 at 20:44
Are there any other headers in the response that seem useful? Are you putting the `cookie` in the correct part of the `UrlFetchApp` options? https://stackoverflow.com/q/10869932/9337071 — tehhowch, Sep 07 '18 at 02:26
Thanks for your interest. I have logged the entire cookie (all 5 elements). These elements are (5 dots equal long string of characters): [JSESSIONID=.....; path=/lpp; httponly][ MC=.....; path=/; domain=.fidelity.com; expires=Sat, 07-Sep-2019 20:28:10 GMT][akaalb_www_binpublic_alb=......; path=/;; HttpOnly][ _abck=.....; expires=Sat, 07 Sep 2019 20:28:10 GMT; max-age=31536000; path=/; domain=.fidelity.com][ bm_sz=.....; Domain=.fidelity.com; Path=/; Expires=Sat, 08 Sep 2018 00:28:10 GMT; Max-Age=14400; HttpOnly]. My feeling is Cookie[0] is the only one that is useful, but I am not sure. — McSquare, Sep 07 '18 at 20:46
For your second question, I tried the method stated in the link you provided (using the "headers" option and I was careful to use their exact method, line for line). For two values of j (0 and 2), I got into an irrelevant login page within Fidelity. The other three values of j (1, 3 and 4) produced an error with a response code of 500. — McSquare, Sep 07 '18 at 21:00
Please be sure your question reflects the effort you have performed, especially if you have a large set of tests performed and results from those tests. — tehhowch, Sep 07 '18 at 21:03

score 0 · Answer 1 · answered Oct 12 '19 at 00:25

Here is something which worked for me. You may have found the solution already, not sure. Remember to fill in your loginid where the XXXX is and the pin number for YYYY.

I understand this is python code, not the google script, but you get the idea about the code flow.

import requests, sys, lxml.html
s = requests.Session()
r = s.get('https://login.fidelity.com')
payload = {

'DEVICE_PRINT' : 'version%3D3.5.2_2%26pm_fpua%3Dmozilla%2F5.0+(x11%3B+linux+x86_64%3B+rv%3A41.0)+gecko%2F20100101+firefox%2F41.0%7C5.0+(X11)%7CLinux+x86_64',
    'SavedIdInd' : 'N',
    'SSN' : 'XXXXX',
    'PIN' : 'YYYYY'
}

r = s.post(login_url, data=payload, headers=dict(referer='https://login.fidelity.com'))
response = s.get('https://oltx.fidelity.com/ftgw/fbc/oftop/portfolio')

print response.content

Results in Bad Request\n\n
Bad Request
\nYour browser sent a request that this server could not understand.
today — Sam T, Nov 03 '20 at 21:34

score 0 · Answer 2 · answered Oct 14 '19 at 16:47

0

mwahal, you left out the critical form action url (your login_url is undefined)

this works (if added to your python code)

login_url = 'https://login.fidelity.com/ftgw/Fas/Fidelity/RtlCust/Login/Response/dj.chf.ra'

btw here's the result of the print after the post showing successful login

{"status": 
    {
        "result": "success", 
        "nextStep": "Finish", 
        "context":  "RtlCust"
    }   
}

or adding some code:

if r.status_code == requests.codes.ok:
    status = r.json().get('status')
    print(status["result"])

gets you "success"

answered Oct 14 '19 at 16:47

Kevin Normoyle

1

Results in: Bad Request\n\n
Bad Request
\nYour browser sent a request that this server could not understand.
– Sam T Nov 03 '20 at 21:33
yes. That method no longer works. I use selenium now to login and then save about 5 cookies that I use for subsequent requests (works okay for non-selenium). – Kevin Normoyle Nov 04 '20 at 22:49

score 0 · Answer 3 · edited Jun 07 '20 at 16:01

Unfortunately the answer from @mwahal doesn't work anymore - I've been trying to figure out why, will update if I do. One issue is that the login page now requires a cookie from the cfa.fidelity.com domain, which only gets set when one of the linked JavaScript files is loaded.

One alternative is to use selenium, if you just want to navigate the site, or seleniumrequests if you want to tap into Fidelity's internal APIs.

There is a hitch with seleniumreqeusts for the transactions API... the API requires Content-Type: application/json and seleniumrequests doesn't seem to support custom headers in requests. So I use selenium to log in, call one of the APIs that doesn't need that header, copy then edit the response's request header, and use regular requests to get the transactions:

from seleniumrequests import Chrome
import requests

# Log into Fidelity
driver = Chrome()
driver.get("https://www.fidelity.com")
driver.find_element_by_id("userId-input").send_keys(username)
driver.find_element_by_name("PIN").send_keys(password)
driver.find_element_by_id("fs-login-button").click()

r = driver.request('GET', 'https://digital.fidelity.com/ftgw/digital/rsc/api/profile-data')
headers = r.request.headers
headers['accept'] = "application/json, text/plain, */*"
headers['content-type'] = "application/json"

payload = '{"acctDetails":[{"acctNum":"<AcctId>"}],"searchCriteriaDetail":{"txnFromDate":1583639342,"txnToDate":1591411742}}'
api = "https://digital.fidelity.com/ftgw/digital/dc-history/api"
r = requests.post(api, headers=headers, data=payload)

transactions = r.json()

How to login into the Fidelity website and navigate within it?

3 Answers3

Bad Request

Bad Request