We have a requirement to build a web scraper for our portal that will scrape job opportunities from selected sites based on keyword and location. The scraped jobs will be republished on our platform in a standardized format. The post has to be redirected to an external link.
This is our current tech stack: CSS Framework: Foundation Front-End Framework: JQuery, React Back-End Framework: Ruby-On-Rails Database: PostgreSql
Features: - Search Bar - Filter jobs by Keyword Search - Location Search - Category Type - Scraping done by period - Google Schema compatible
Category Types: - Graphics & Design - Audio, Music & Audio - Tech and Programming - Digital Marketing - Business - Writing & Translation - Lifestyle - Business & consulting - Admin Support
Job Details to be scrapped - Job Title - Company Name - Company Logo - Job Expiry - Job Description - Job Requirements - External Link (for Apply Button)
We have curated a list of websites and portals where opportunities are posted frequently. Reference sites: https://www.upwork.com/freelance-jobs/ https://www.behance.net/joblist https://jobs.github.com/positions https://stackoverflow.com/jobs https://www.guru.com/d/jobs/
Flow of Data - The scraper will pull the required data from scraping sites on an hourly basis. - The scraped data will be organized and converted into a standard format on our website. Standardized format here refers to linking of the job to the right category -- same font --- same font size, etc. - The job will be saved for approval from our team internally. - Once approved, the job will be live on the job feed. Users will be able to search and apply for postings through our database. Every user is required to create a profile with our site prior to applying for any job.
