0

I am trying to get a website's (habr.com) page_source using python3 and selenium, but there are some svg tags that contain use tags with xlink:href parameter and #shadow-root (closed) in them. That peace of html looks like this:

html_source

My python code is this:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('https://habr.com/en')
source = driver.page_source

I am doing this using just python interpreter, not from file, so there is enough time to load all the resources. So, the source variable will contain all the html except this #shadow-root (closed) part.

I tried this solution, but I guess it works only with #shadow-root (open).

What should I do in order to obtain the whole html source including those Shadow DOM parts?

UPDATE: The whole point of this is that I want to make a some kind of proxy server that will point to desired website and change all the links on the page to my localhost. In order to make proper tests I wanted to get the source html from target website and compare it tag by tag with my localhost's source html. But I cannot do that until I get this Shadow DOM content. I mean I can do that but it just will not be an objective test.

Artem Ilin
  • 353
  • 2
  • 19
  • go to the think the xlink:href in the use tag points to and load that. Shadow roots (closed) are not directly accessible. – Robert Longson Apr 12 '19 at 10:38
  • @RobertLongson This raises several questions. How can I know that a certain element has this `shadow DOM` in it? How do I load it and then how do I properly add the result to the `html`? If I paste the link from `xlink:href` in browser, it shows an empty page. – Artem Ilin Apr 12 '19 at 10:47
  • elements have shadow DOM, other svg elements don't. You don't add the result to the html file, it's already there but inaccessible to selenium. I don't know what you're pasting or where you're putting it. I don't know why you want to do this or what you're trying to achieve here. – Robert Longson Apr 12 '19 at 10:50
  • @RobertLongson I updated the question explaining the point. – Artem Ilin Apr 12 '19 at 11:27
  • grab what the use points to, put that somewhere and then change the use xlink:href to point to the local location of that content. – Robert Longson Apr 12 '19 at 13:07

0 Answers0