Solution 1 :

You need to get sessionId and requestId from HTML code and next request this URL:

f"https://www.healthgrades.com/api3/usearch?where=CA&sessionId={sessionId}&requestId={requestId}&sort.provider=bestmatch&source=init&what=Acupuncture&category=provider&cid&debug=false&debugParams=false&isPsr=false&isFsr=false&isFirstRequest=true&pageNum=1&userLocalTime=23%3A55"

After to you parse JSON response you’ll find ["search"]["totalPages"] (and ["search"]["totalCount"]) values.

Problem :

I am trying to get the number of pages on this site, but nothing works.

response.css('._2uKgC p::text').getall()

html:

<nav class="_2uKgC" aria-label="Page navigation" data-qa-target="pagination">
<p class="">Page 2 of 209</p>
<ul>...</ul></nav>

What am I doing wrong?

Comments

Comment posted by Kishan Mehta

This element is not present in html you can check “view page source” to check what HTML you are getting instead of inspect element.

Comment posted by Alexander

I looked, but did not understand how it works. Can you help me?

By