We're looking for a developer with good knowledge of CSS selectors. Your job will be to write the CSS selectors to program a web scraper to gather data from online retailers. You can verify the selector works by running document.querySelector(SELECTOR) or document.querySelectorAll(SELECTOR) in the dev tools console.
1. pageNumberParameter - Paste the URL parameter that determines which page of results to show, e.g. from this link "https://www.very.co.uk/sports-leisure/mens-sports-clothing/outdoor/e/b/119443,2781.end?pageNumber=2" you would find ?pageNumber=2 and replace the page number with $i so it becomes ?pageNumber=$i 2. acceptCookiesSelector - This one is only required if the site has a cookies popup that blocks the page. Paste the CSS selector for a button that accepts the cookies. 3. pageCountSelector - CSS selector for the button or other HTML element where the innerText is ONLY the last page number. E.g. if there are 30 pages of jeans, find an element whose only text is "30" - this is typically located with several page buttons that might look like 1 2 3 4 5 ... 30 4. searchPageGrid - CSS selector on the search page for the container that holds all the links to product pages. 5. searchPageOrigin - CSS selector for the a tags with hrefs linking to product pages, duplicate hrefs don't matter but try to skip any ads. Instead of running document.querySelectorAll, this will be run like searchPageGrid.querySelectorAll so sometimes this can be simple as "a" 6. searchPageTitle - CSS selector for the product title on the search page 7. searchPagePrice - CSS selector for the CURRENT product price on the search page 8. searchPageThumbnail - CSS selector for the product thumbnail on the search page, must have a src or a data-src or a data-srcset attribute with a link to the image. Make sure it works on all the products on the page because sometimes they're lazy-loaded. 9. productTitle - CSS selector for the product title on the product page 10. price - CSS selector for the product price on the product page 11. oldPrice - CSS selector for the original product price on the product page (if it doesn't exist, leave blank, but make sure to check at least one discounted product) 12. image - CSS selector for the first product image on the product page, must have a src or a data-src or a data-srcset attribute with a link to the image We will be running the scraper and checking that each site is scraped correctly. You will be paid for each retailer you complete so make sure there are no errors. It should take about 10 minutes per retailer. If you're unsure how to write the selectors on a site/field, move onto the next one and fill in the text box(es) with the word TODO and we'll complete those ones. Start your proposal with "CSS Selector" so we know you've read it all.
We're looking for someone who can scrape 75 of these sites this week. If you do a good job on this, there may be ongoing work for you as we add more retailers.
We will be able to provide a video example of how to scrape a site. Below is a list of each field that needs completing for each retailer.
