Hello Freelancers, I'm searching for a developer familiar with web scraping and Python to fix an existing web scraper which scrapes product data from products from category links from italian ecommerce-website www.yeppon.it
The script basically works, but it gives an error on certain points when scraping, I think because of a light change in the structure of the website which causes an error when the script tries to scrape a products text description.
Goal of this project is to fix the errors so the script works like it used to again, scraping data of products from given category-URLs from the website and giving out the data in csv-files. I think this won't be too much of an effort because it is basically this one error which needs to be located and fixed, everything else still seems to work fine. Price can be discussed.
Some facts:
1. The web scraper is based on Python with a GUI. It's final version comes as an exe file (therefore I can't attach it in the project description, I will send it in the messages or work stream). 2. It scrapes certain product data (like product name, price, description, image links) by category links which can be entered into the GUI. The GUI also has some input fields, these are just for fixed strings which can be entered into the fields and will be given out in the CSV files the script gives the product data in. 3. The scraper technically still works, however, it gives an error when scraping certain categories. You can check this by running the tool, filling out the given input fields with the data explained in the "Instructions" tab of the tool and then start scraping. It will produce this error (can be found in the log file):
------------------------------------------------------------------------------------------------------------------ 4. The error seems to occur when scraping a products text description. The text description consists of three possible elements: - bulletpoints formatted into an ul element - a text description which is cleaned/has HTML code removed/replaced - scraping data from a table on the website and putting it into a given HTML structure
It was developed by a freelancer from PPH for a colleague of mine, unfortunately I can't reach my colleague for quite some time now to ask for all the details or the freelancers name, so I will post this to the public.
Scraping some categories will result in the error mentioned above, for example: https://www.yeppon.it/c/elettrodomestici/grandi-elettrodomestici/asciugabiancheria or https://www.yeppon.it/c/elettrodomestici/grandi-elettrodomestici/frigoriferi
Others work just fine, like: https://www.yeppon.it/c/telefonia/smartphone/smart-phone
I will attach the files I have about this project from my colleague. As I can't attach exe or rar files, I attached:
- a first version of the Python code (it is a beta version which will give another error which is solved in the final exe file and not the final code, just to give you an impression), as well as the code of the GUI and the requirements. These are async_scraper.txt, gui.txt and requirements.txt
