There's a lot to it.
For non-interactive web pages, I recommend BeautifulSoup.
Even if normal BeautifulSoup doesn't work, there are lots of ways to pretend to be a browser. Give those a try before you go for browser automation tools.
If you're scraping a site that does require a browser automation tool, I recommend Playwright over Selenium. It's newer and weird for a Python dev, but the documentation is great.