Data From Web-Scraping

Estimated reading time: 11 minutes
Data From Web-Scraping

Example inputs CSV to most scrapers:

document

ticker other_con parent oi_ratio code_or_t listed yelp_nam fake_yelp linkedin short_na glassdoor glassdoor glassdoor glassdoor yelp_url yelp_id starting extra_ben website logo spyfu_url fb_names doordash_id
DIN IHOP Yes 0.525074 APPB No Applebee Applebee Applebee Applebee https://w https://w https://w https://w yelp.com/ applebees-neighbor Filtered E https://w applebees.com        
EAT Maggiano' Yes 0.895238 CHIL No Chili's Chili's Gril Chili's Gril Chili's https://w https://w https://w https://w yelp.com/ red-robin-gourmet-burgers https://w chilis.com          
BJRI   No   CPKI No California California California CPK https://w https://w https://w https://w yelp.com/ california-pizza-kitchen https://w cpk.com          
BJRI   No   TGIF No TGI Friday TGI Friday TGI Friday TGIF https://w https://w https://w https://w yelp.com/ tgi-fridays   https://w tgifridays.com        
CAKE   No   CAKE Yes Cheeseca Cheeseca Cheeseca TCF https://w https://w https://w https://w https://w the-chees 2   https://w thechees https://w thechees the-cheesecake-factory
RRGB   No   RRGB Yes Red Robin Red Robin Red Robin Red Robin https://w https://w https://w https://w https://w red-robin-gourmet-burgers https://w redrobin. https://w RedRobin red-robin-gourmet-burgers-and-brews    
BJRI   No   BJRI Yes BJ's Resta BJ's Resta BJ's Resta BJ's https://w https://w https://w https://w https://ye bjs-restau 1 Filtered E https://w bjsrestaur https://w BJsRestau bj-s-restaurant-brewhouse

Doordash:

Inputs to scraper:

You have to obtain the links. To obtain it, I might use something like this from Chris. Otherwise, you can also create a custom search account on Google, to obtain the links.

Links
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-butler-193687/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-farmingville-180029/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-doral-75080/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-elmont-122664/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-patchogue-180028/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-clark-151559/
https://www.doordash.com/store/applebee-s-neighborhood-grill-bar-commack-143840/

Facebook

Scraper:

File

Output:

Example

status_id status_m link_nam status_ty status_lin status_pu num_reac num_com num_shares
849176893 Not that you need a video https://w ######## 634 173 146  
84917689333_101561581442693 photo https://w ######## 0 0 0    
849176893 Why send me a coup photo https://w ######## 0 1 0  
849176893 Cut the co Timeline photo https://w ######## 194 18 40

Glassdoor

Scrapers

Reviews

Interview

Rating

Benefits

Jobs

Output

Reviews

Title Rating Work Life Culture V Career Op Comp Ben Senior Ma Review D Current or Employee Location Recomme Outlook Approves Full-Time Time Emp Pros Cons Advice to Management
"Burnt Do 4   Jan 10, 20 Past Hostess Santa Ros Yes No Neutral Part-time More tha Good foo Burnt dow Sorry about the fire :(        
"Review f 4 4 1 3 1 2 Jan 10, 20 Current Anonymous Employ Yes Yes Yes Part-time   great mon managem treat others how you would like to be treated  
"Server" 5 5 5 5 5 5 Jan 8, 201 Past Food Serv Syracuse, Yes Neutral Neutral Full-time Less than Good tips, Takes a lo Keep up the good work your management team is awesome and you're flexible work schedule is great as well

Interview

Title Interview Date Employee Type Offer Experience Interview Type Application Interview Question
Server/Bartender Jan 6, 2018 Anonymous Employee in New York, NY ['Accepted Offer'] ['Positive Experience'] ['Easy Interview'] ["The process took 2+ weeks. I interviewed at Applebee's (New York, NY)."] ['It was a pretty simple and straightforward interview. I was asked relevant questions about my job history and qualifications .', '0I was offered a job directly after the interview. This was nice because I did not have to wait to find out if I had gotten the job.'] [' How many years experience did I have in the industry? 0 ', 'Answer Question']

Rating

Overall Culture Work Life Senior Management Comp and Benefits Career Opportunities Recommend to Friend CEO approval Positive Business Outlook
Overall Values 2.8 2.8 2.6 3 53% 67% 41%

Benefits

Rating Review D Employee Description
5 14-Jan-18 Current A [' Health insurance available from first day.']
4 3-Jan-18 Current E [" It took me out of my comfort zone, working there you'll meet new people and bond with them, putting smiles on the customers faces, and even make new friends from this experience."]
2 3-Jan-18 Former S [' Did not give enough hours to employees for us to have company contribution for the insurance. Co-pays were higher than previous employer.']
1 2-Jan-18 Current E [" health care, time off, showers, free meals, bad food, beer, can't drink, don't like blondes, don't like tall people"]

Jobs

Title Location Date
Restauran Saint Pete 1 days ago
APPLEBEE' Bradento 3 days ago
Restauran Clearwate 3 days ago
Line Cook Carrollwo 4 days ago

Instagram

Scraper

File 1

File 2

File 3

File 4

File 5

Archive

Outputs

Name Posts Followers Following Category Website Instagram Url
BJ's Restaurant & Brewhouse 1,433 45.1k 119 Creators of the Pizookie. Masters of Pizza. www.bjsrestaurants.com https://www.instagram.com/bjsrestaurants/

Morningstar

Scraper

File 1

File 2

File 3

File 4

File 5

Output

JSON File

Similarweb

Scraper

File

Output

document

Headline Overview Global Ra Country R Category Total Visit Avg Visit Pages Per Bounce R Traffic By Traffic So Referrals Top Refer Top Desti Search Pe Organic K Paid Keyw Top 5 Org Top 5 Paid Social Per Social Ite Display A Top Publi Website C Also Visit Similarity Sites
applebees.com 10,895 1,893 29 4.71M 0:03:27 9.11 30.45% [('United [('Direct', ' 6.88% [('msn.co [('wbipro 64.83% 84.38% 15.62% [('', ''), ('', ' [('', ''), ('', ' 1.26% [('', ''), ('', ' 0.41% ['', '', '', '', '' [('appleb ['', '', '', '', '' ['chilis.com', 'buffalowildwings.com', 'outback.com', 'redlobster.com', 'olivegarden.com', '] buffalowildwings.com

 

Spyfu

Scrapers

File

Outputs

document

MonthlyOrganic KEst MonthEst MonthKeywordsRanking HPaid KeywEst MonthEst MonthAdWordsAdWordsJust MadeJust FeelOrganic CPaid ComShared OrKeywordCore StartWeaknessOrganic EShared PaKeywordCore NichBuy RecoPaid ExcluTop KeywTop KeywInbound LTop Keywords History
https://w11,6656.61M$4.47M1,03211 YEARS
F247551.8k$23.7k011 YEARS2593381, ihop.co1, logansrhttps://w25,9671,0326269,168https://w83900462[('Rank = 1[('Paid Ke  

Yelp

You have to obtain the links. To obtain it, I might use something like this from Chris. Otherwise, you can also create a custom search account on Google, to obtain the links.

Locations

Scrapers

Reviews

Info

Surroundings

Output

document

Day Hours Op Business I Detail Also-Cons Considere Also-View Viewed Link
Mon 11:00 am - Takes Res No   Billings Ba https://www.yelp.com/biz/billings-bald-butcher-covington?page_src=related_bizes  
Tue 11:00 am - Delivery No E3 Paradise https://www.yelp.com/biz/paradise-grill-atoka?page_src=related_bizes  
Wed 11:00 am - Take-out Yes   Old Town https://www.yelp.com/biz/old-town-hall-and-cafe-covington?page_src=related_bizes  
Thu 11:00 am - Accepts C Yes        
Fri 11:00 am - Good for Yes        

document 2 - Click to download

document 3 - Click to download

Linkedin

Scraper

File 1

Output

JSON

Angellist

Scraper File

data, web-scraping, internet, crawler, web, ml