Scrape 1,000 LinkedIn search results into Airtable

Hey guys today I'm going to show you how to use the Search Export in the Growth Hacking module. So what this will do is when you have a search and you want to extract all of the results it will go through on all of the pages. We can do up to a hundred pages and there's ten results per page. That'd be one thousand results. We have also got a separate Sales Navigator one however we'll just be focusing on regular LinkedIn today.

So if you want to extract more than like a thousand profiles you'll need to segment them based on like country and so on or add enough keywords. But in this example we're just going to extract everyone who is in this particular search and then we're going to store them into Airtable. So we'll create a new scenario within Integromat then just choose the Growth Hacking module and we're just going to do Basic Search Export.

Then when you signed up for Growth Hacking you would have got your API token so I'm just gonna use a little test one now. Add that in here. Add in your LinkedIn cookie. And then we're going to come over here and just copy the URL that you want to extract results from. And then in number of profiles, like to start off just to keep this quick we'll just do ten results and we can just run it once to give you a quick example.

Cool so then if you come into result object this is where like a lot of people I think they don't realize it shows all the data here. So we can see all the information here. So here with the first person Alfred Tana we've got the profile URL, first name, last name, connection degree, job title, location. And if we come into the search you can see number one is Alfred Tana and then you can go through all of the results. The second one should be Benjamin Reynolds which is, and so on.

So obviously it's quite pointless having this data in Integromat so we can just go ahead and store it in like Airtable. You could also use Google Sheets or any other tool that you want pretty much. So scroll down to whatever Airtable base you want to store this in. So I'm just using testing for now and Create Table 1. And then let's see like what information do we want to extract, store. So maybe like we want to map the connection, connection degree, location and probably like job title is that's good enough for now.

So because it's pulled ten results, generally Integromat they only work with like one record at a time by default. So what we need to do is add in Iterator and if you put in the result object that is then going to allow us to process all ten records that we have. And then if we go to Create a Record, the Airtable module, and then we just want to go and add in the Airtable that we're using. Cool. And then from the iterator we want to add in like the URL, first name, last name, connection degree, location and job title. Like you can store in all this other information if you want to it just completely depends on your use case.

Cool so let's hit save and then we're going to run it now. Let's finish running. You can see it's just pulling in the data that we've just scraped from LinkedIn and adding it in the Airtable. So like you can use any tool that you want you know you can use Google Sheets you could sync this over directly to your database like whatever tools you prefer to work from. Airtable it's just like a personal preference of mine.

And we can actually then like spread out this flow and instead we can choose to go ahead and return like a thousand results which is going to be all results in that particular search and so it's going to take a little bit longer to do. Cool that has now come through and because the result is so large what we need to do is add a Router. So it's going to return us a file instead of just the plain URLs because it's going to be like a large amount of data. So we need to add a HTTP GET request. So we're going to add in the JSON URL and it's just going to perform a GET request and just click parse response.

And we just need to add some little filtering in here so if JSON URL exists then we're going to route down here. If JSON URL does not exist then route down here. So that means if you're bringing back a large amount of data it's going to route down this part and if you're just taking a small amount it's going to go down the other path. So then the output will be the data from here.

Now just to like speed this up a little and we can enter it here. If we just run that module only. And we can just clone over this Airtable, clone it to the end here and we just need to reset and because it's now coming from this iterator instead of this one. Cool so we're just going to update to this new iterator. So change this back to JSON. We only change that back to get the parameters in the end field. So we'll save that there and we're just going to run again. So it's going to extract 2000 results for us.

So there we go it's now working. A thousand results have been extracted and if we just come down in the Airtable you can see there they're all being added in. So this will just go all the way up to a thousand. I'll just leave it running so you can see it. Cool so there we go all of our records are now extracted into the Airtable base. We've got their location, job title, connection level, first name, last name and LinkedIn profile.

So that is how you extract results from LinkedIn and the same applies for Sales Navigator. Obviously in this example we just did basic LinkedIn. And in the next video I'm going to show you a few different like add-ons that we can add into this flow and then we'll also work on building the actual outreach side of things, sending the connection requests to everyone in this Airtable base and then following up with like messages and so on. So thank you very much for watching. Any questions feel free to leave them in the comments and yeah I hope this helps you set up your automation flows.