Creating a Digital Ocean scraping server
We’re going to use a server - this one in particular is fine for repeating tasks or scraping, but you could also use a high-memory or high-CPU one for data processing tasks.
Step One: Registration and navigation
In order to run a server, you’ll need a Digital Ocean account. That link gives you $10, although you do need to give them $5.
Once you’re registered, visit the Create Droplet page from the top menu.
Step Two: Filling in some blanks
Picking an operating system
First, for our operating system we are going to pick Ubuntu. It’s a common and simple-to-use Linux.
Picking a server size
Then we’re going to pick the smallest size server. We don’t need anything bigger right now.
Block storage
We don’t need any block storage.
Datacenter locations
Datacenter locations don’t matter too much, but feel free to pick one closer to you.
Additional Options
We don’t need any additional options.
SSH Keys
Click New SSH Key if you don’t have one already there to use.
You’ll be confronted with a window! This is where you’re going to put your public key.
Step Three: Creating and using a public key
Instead of a password, we’re going to use a special file that proves who we are.
This is called an SSH key - it has two parts, a public key that we give out to people and web sites, and then we keep a private key to prove that yes, we are who we say we are. That’s the super short way of explaining it.
- Windows, follow this guide to generate an SSH key.
- OS X, you could follow this guide but really, just open up the command line and run the following command.
The command below will ask for a passphrase, but just hit enter to skip it (it’s for extra security - “something you have and something you know”)
ssh-keygen -f ~/.ssh/algorithms_key -t rsa -b 4096
This will create two files, algorithms_key
and algorithms_key.pub
.
- Look at
algorithms_key.pub
, either on the shell or in your text editor (see note below) - Paste the entire file into the box on Digital Ocean
- Name it
algorithms_key
- Click Add SSH Key.
Note: You can also see the file from Terminal by typing
cat ~/.ssh/algorithms_key.pub
.
If you wanted to enter your
.ssh
directory, open up Finder and bring up the “go to folder” option by pressingCommand+Shift+G
, or clicking Go from the top menu, then Go to Folder. Type~/.ssh
and hit enter. You can drag the file to your text editor.
Digital Ocean will automatically select the appropriate checkbox.
Finishing Up
Down at the bottom you can confirm you only want one droplet (aka one server), and you’re able to name it. It doesn’t matter what you name it.
Then click Create to finish
Setting up your server
Now you can move on to setting up your server