Looking for a senior web scraper to develop a system that scrapes data from a variety of sources and injects the data into Airtable.
Requirements
- Daily scraping
- 99.99% uptime
- Notifications if scraping fails
- Documentation on system setup
- Clear and efficient escalation path to fix the issue
Sources
- Facebook Jobs
- LinkedIn Jobs
- Indeed
- GlassDoor
- Craigslist
- Google Jobs
Output
- The final destination of the scraped data is Airtable
- The easiest way to get the data into Airtable is Zapier or Google Sheets
What I have today:
- Twitter --> Zapier --> Airtable
- Reddit --> Zapier --> Airtable
- RSS --> Zapier --> Airtable
Opportunities for improvement
I would like to increase data collection and try and grab information within the job post, including company name, URL, email address, company linkedin url, etc
What I DONT want
- One time scraping
- Inconsistent or unreliable scraping
Hourly Range: $20.00-$40.00
Posted On: May 09, 2021 17:20 UTC Category: Data Extraction Skills:API Integration, ETL Pipeline, Web Scraper, Scrapy, Import.io, Selenium, html2text, Beautiful Soup, Python-Goose, Python, PHP, SQL, C#, Extract, Transform and Load, Web Crawling, Data Extraction, Zapier, Airtable
Skills: API Integration, ETL Pipeline, Web Scraper, Scrapy, Import.io, Selenium, html2text, Beautiful Soup, Python-Goose, Python, PHP, SQL, C#, Extract, Transform and Load, Web Crawling, Data Extraction, Zapier, Airtable Country: United States
click to apply
Project ID:
3168599
Project category:
API Integration, ETL Pipeline, Web Scraper, Scrapy, Import.io, Selenium, Html2text, Beautiful Soup, Python Goose, Python, PHP, SQL, C#, Extract, Transform And Load, Web Crawling, Data Extraction, Zapier, Airtable