VBscript activeX Internet Explorer object alternative

Question

I need a Linux alternative to browser manipulation. In Windows I would use a VBscript to do this via Internet Explorer object, but in Linux I don't have either of those.

Is there any other script like environment for Linux that can manipulate FireFox or Chrome? I need to emulate a user using the browser and retrieve date from sites the 'user' accesses.

curl is not appropriate, because I need javascript as well. Basically I need the script to see the same thing that a 'real' user would see.

Answer 1

you can use selenium to automate browser interaction

alternatively you can bypass the browser and access the data directly by using web scraping - e.g. in python there is scrapy