An experienced data engineer is sought to build a product data scraping application. The web scraper will harvest key details like product names, descriptions, images and pricing from various e-commerce websites. The scraped structured data needs to be normalized and stored in a database. Tools like Python and libraries such as BeautifulSoup and Scrapy should be utilized to automate the scrape process. The candidate must have strong knowledge of web scraping techniques and building robust data pipelines. Prior experience developing similar commercial grade web scrapers and handling large scrapped datasets is essential. Companies looking to leverage competitor pricing and listings online will benefit from this data harvesting solution.
