10 Gigabit Networking Between Home Server and Computer

Network card

My server is in a huge tower (Fractal Design Define XL R2) with a huge motherboard (Supermicro X9DRH-7F). There is a variant of this motherboard with onboard 10 gigabit ethernet, but the cost was double at the time of purchase, so I opted for this cheaper one.

To get 10 gigabit networking, I will need a 10 gigabit network adapter. I can use one of my six PCIe 3.0 x8 slots, with a max transfer rate of about 8GBps, is plenty fast enough to take a 10 gigabit network adapter card. There could even be two or four 10GbE interfaces on that card and it would still work, since 10 Gb = 1.25 GB and 40 Gb = 5 GB.

Now there are two major types of connectors used for 10 gigabit ethernet: the standard-looking RJ45, and the intimidating SFP+.

Connector type

There are four major methods for home use

• Direct Attach Copper (DAC) cable – cheap but only works for short distances
• LC connector for multi-mode fiber optic cable – better for longer distances across the home
• LC connector for single-mode fiber optics – good for REALLY long distances, but doesn’t seem to make sense for home use
• RJ45 – only really makes sense if you already have at least Cat 6a ethernet cables

I found this video from Craft Computing on YouTube very helpful here. More on this when we talk about the 10 gigabit switch in a few sections.

My server is sitting right underneath my work desk, so I didn’t need long cables. I was going to pick an RJ45 connector until I discovered…

Thunderbolt 3 to 10 Gigabit Adapter

It would have been easy to just buy two of those network cards above and call it day, but my main computer is actually a Lenovo X1 Extreme Gen 1 laptop. I can’t just plug in a traditional PCIe network card.

This laptop is very capable (read: expensive) and comes with Thunderbolt 3, which is EVEN FASTER than 10 GbE, running 40 Gbps through the proper cables. But I need an adapter to get the 10 GbE into the TB 3 port on my laptop.

QNAP makes just such an adapter – one for SFP+ connections, and one for RJ45 connections. The SFP+ adapter was cheaper, so I chose that.

10 Gigabit Switch

I had intended to just plug the two computers together and call it a day, but I decided a much more elegant solution would be to plug each machine into a switch and let the switch facilitate the connections.

The aforementioned YouTube video from Craft Computing detailed a super affordable product: the Mikrotik CRS305-1G-4S+IN. It comes with a power supply, is nearly silent when running, and has one gigabit port to hook up to the main router (if desired) and four 10 gigabit ports to use as a switch.

Summary Purchase List

This is assuming your have a laptop with Thunderbolt 3 and a tower server with open PCIe lanes.

• Mikrotik CRS305-1G-4S+IN - 10 gigabit switch
• Mellanox MNPA19-XTR - 10 gigabit SFP+ network card for server (2-port and 4-port variants)
• QNAP QNA-T310G1S - 10 gigabit to Thunderbolt 3 network adapter for laptop
• (2x) DAC cable

Samba Shares

When I first got my server running, I set up a Samba SMB share to my main storage folder so that I had a new place to move my old files.

That worked pretty quickly. I was happy with that.

So I plugged in all my 10 gigabit hardware and tried transferring files…

I was able to transfer files, but only at a dismal 40-50 MBps. That was not even saturating the write speed on my HDD, which I expected to be at MINIMUM 80 MBps.

The problem was that after setting up my 10 gigabit connections, I kept using that plain ol’ 1 gigabit connection I had already been using. Notice what is highlighted on the left side of the following two screenshots.

Both of those network locations appear the same. When I click one of them in the sidebar on my Windows 10 laptop, it shows the same root folders. Navigating through shows the same folders as well.

The difference is that when connecting, I typed in a different IP address.

Configure IP on Windows Adapter

Right click your networking icon in your Windows task tray and select “Open Network & Internet settings”.

Select “Change Adapter Options”.

Right-click the 10 gigabit network adapter and select “Properties”.

Make sure “Internet Protocol Version 4 (TCP/IPv4)” is enabled and double click it.

Configure it manually to use the same sub-network as your switch. If you bought the Mikrotik one and didn’t change it, the subnet is 192.168.88.xxx. You just have to fill in the last number to give this network adapter it’s own location.

Since I’m not connecting to the internet, I just left the default gateway and DNS server addresses blank because I don’t need them. My Internet traffic is routed through my Gigabit ethernet or my WiFI adapter.

Accept the changes and move to your server.

Configure IP on Ubuntu

Go to network settings.

Click the settings gear icon for your 10 gigabit network card.

In IPv4, add an IP address on the same sub-network you selected earlier. If you went with the default for the Mikrotik switch, then the first three numbers have to be 192.168.88.

Set up a Samba Share on Linux

Now we need to have a folder that we want to share. In my case, I have four 10 TB HDDs that I have mirrored and pooled together. So two serve as a RAID 1 redundant mirror for the other two, which are pooled together to be displayed on my file browser as one folder. It’s pretty neat and ensures I’ll always have my stuff even if a drive fails.

I have mounted this pooled virtual drive in my /mnt folder. The easiest and most visual way to share this folder on my local network is to navigate to it in my Ubuntu Desktop file browser. The default file browser is “Nautilus”.

Right-click the folder and select “Local Network Share”.

Configure your sharing settings. I kept the name of the folder and allowed more access to it so that people on my local network dont need my help getting logged into the folder.

Then hit “Create Share”.

If you run into an error that says Nautilus needs to add some permissions to your folder in order to share it, try to add them automatically (if that is indeed what you want to do).

If you’re hit with another error that says Could not change the permissions of folder, then we need superuser privileges to get things straightened out.

To confirm this is actually the problem, right-click on your desired share folder and select “Properties” then “Permissions”. Here’s a screenshot from another folder of mine that I want to share.

This is showing us that the folder is owned by the user “root”. Since I am logged in as “lance”, I cannot change root’s folder. We’ll need to go to the terminal and tell it “I am root”.

Hit Ctrl + Alt + T to open up the Linux Terminal.

We need superuser privileges to change the permissions on this folder. And we need to know the path of the folder. Mine is /mnt/Intel660P. First we’ll change the owner of the directory and all of its contents.

$ sudo chown --recursive username:username /path/to/folder

sudo gives us superuser privileges (telling the terminal “I am root”)

chown means “change the owner”

--recursive means “do this on all the sub-directories and contained files”

username:username represents the user and the group that owns this folder. Each user has a group that is the same as their username, and that’s what I want to set my folder to.

/path/to/folder designates the file or folder we want to change

Putting it all together, here is what I wrote to change the owner of my folder

$ sudo chown --recursive lance:lance /mnt/Intel660P

And we can check to see if it worked by refreshing our Nautilus file viewing and checking the Properties > Permissions again.

Nice! Now, I need others to be able to access the folder, so we’ll need to change the permissions of this folder and the enclosed files.

Easiest way on Ubuntu desktop is to change the “access” for “group” and for “others”.

Then click on “Change Permissions for Enclosed Files…” and change to the settings you want.

Check out the chmod command if you want to do it in the terminal.

You can get a lot fancier with permissions if you set up samba shares. Click here for the basics of setting up samba shares and click here to get a more advanced samba configuration.

Create New Network Share on Windows

NOW! This step was my big problem. I already had a folder on my Linux server marked as a share. And I already had a link to that share set up on my Windows computer. This gave my Windows computer access to the storage on my Linux server.

The problem was that this link was made BEFORE installing the 10 gigabit network cards. So it was using plain ol’ gigabit (which is honestly fine for my home use, but doesn’t seem as cool so I “needed” to upgrade).

Let’s create a new network share on Windows that now uses this new connection.

On your Windows computer, hit WIN + E to open up Windows Explorer.

Right-click “This PC” in the left sidebar and select “Add a Network Location”. Note: you can also select “Map network drive…” if you want it to be stored as a letter.

This will open the Add Network Location Wizard.

Hit Next to continue.

Select “Choose a custom network location” and select Next.

Now is time to tell Windows where to look. You need your server’s 10 gigabit IPv4 address for this step. This was the last IP address we set just a few steps ago. We will write…

\\IPA.DDR.ESS.XXX\SHARED_FOLDER

Mine looks like this…