Upgrading your HDB Home Network to 10Gbe Project

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
With Wifi-6 gradually coming into our homes, availability of Cat6 trunking in modern HDB set up and 10Gbe switches at prices within reach to most. I hope to share my upcoming upgrade to 10Gbe network in a HDB Home setup, also provide a head start to people who are interested in such set up too.

BEFORE:
zYuq54l.jpg


Existing Network Setup:
1) 1x Netgear GS108PE
2) 2x TPlink SG105E (For living room and kids room)
3) 1x Custom m-itx pc running pfsense 2.4.5
4) 1x Singtel ONR (Bridged at Host side)
5) 1x MR ONT with static IP
6) 2x Ruckus Unleashed R500
7) 1x Ubiquiti EdgeSwitch 10XP (For my home lab)


Project Network Setup:
1) Custom made 5U server rack at 270mm depth
2) 1x Mikrotik CRS312-4C+8XG-RM (On SwOS on WAN side)
3) 1x Mikrotik CRS305-1G-4S+IN (In home Lab)
4) Netgear GS110EMX (In home lab)
5) Transfer custom m-itx build pfsense pc to 1U server chassis
6) 2x Ruckus Unleashed R500
7) 1x Ubiquiti EdgeSwitch 10XP with rack adapter
8) 1x Singtel ONR (Bridged at Host side)
9) 1x MR ONT with static IP
10) 2x TPlink SG105E (For living room and kids room)

All Purchased Gear:
0t9Yp27.jpg

JOAMPuj.jpg

xRBF8HC.jpg

zOUve0t.jpg

IZfDbof.jpg

q3kw9Aa.jpg

DeoPzGT.jpg
 
Last edited:

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
Making it a quieter system:
1) Changing the 4x 40mm fans in the Mikrotik switch to Noctua 40x20 PWM fans.
2) Changing the 2x 40mm fans in 1U-1.5U server chassis fans to Noctua 40x20 PWM fans.
YARaD01.jpg

juAvWdX.jpg


Server Arrangement:
Level 5: 1U chassis running pfsense 2.4.5
Level 4: 1U adapter for Ubiquiti EdgeSwitch 10XP
Level 3: 1U Mikrotic CRS312
Level 2: Breath Gap
Level 1: Base tray for Singtel ONR & MR ONT
2w9FXzl.jpg


Mikrotik CRS305 at homelab connected to 10Gbps port on the Netgear GS110EMX via SFP+ to RJ45 modules and direct to PC through DAC.
FcIK6DD.jpg

t18qegS.jpg
 
Last edited:

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
**UPDATED on Oct 2021**
UOz7MEA.jpg


YZ3xqsO.jpg


Overall, it is a success, but with some ups and downs. The 10G in my lab and FreeNAS in rooms worked as planned. Quite like the cleaner server rack set up. If I do upgrade to 10G internet, will change the NIC in my pfsense box.

Some unfortunate event unfold during deployment.
1) While changing the chassis, the Kingston SSD boot drive died on me. Changed it to an Intel 240GB SSD.
2) Tested all the Cat6 points around the house and found that 2 out of 6 Ethernet point could not run at 10G. Which may be due to wear and tear of connection point or cable degradation. will get a contractor in to restore these data connection point.
 
Last edited:

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
Before:
zYuq54l.jpg


After:
YZ3xqsO.jpg


Full working setup:
f9Cp6F5.png

h80DrJn.png

EejFha1.png
 
Last edited:

zoneguard

Senior Member
Joined
Jun 2, 2000
Messages
1,961
Reaction score
399
What's the specs of the mITX pfsense router? Can keep up with 2x1Gbps WAN? Why don't you use Mikrotik CCR range of routers?

Remember to show some iPerf figures when the whole setup is completed.

What are you running in your home lab to justify the 10Gbe?
 

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
What's the specs of the mITX pfsense router? Can keep up with 2x1Gbps WAN? Why don't you use Mikrotik CCR range of routers?

Remember to show some iPerf figures when the whole setup is completed.

What are you running in your home lab to justify the 10Gbe?

i5-2500K, 8GB RAM, 128GB SSD, GA-HZZN m-itx mobo, 4x 1Gbe Intel NIC. Personal preference geared towards pfsense and more familiar with the GUI.

Both WAN1 and WAN2 are directed to 2 different VLAN (default and VLAN50) within my network. When doing speed test at two VLAN at the same time, I hit close to 1Gbps per WAN. CPU usage will hit 80% when both WAN at peak.
 
Last edited:

zoneguard

Senior Member
Joined
Jun 2, 2000
Messages
1,961
Reaction score
399
I'm running pfSense (virtualized) on i7-4790S with 6 x Intel GBe NICs.

WAN1: M1 1Gbps static IP.
WAN2: Whizcomms 1Gbps.

But severely underutilized the bandwidth so going to decommission 1 WAN soon.
 

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
I'm running pfSense (virtualized) on i7-4790S with 6 x Intel GBe NICs.

WAN1: M1 1Gbps static IP.
WAN2: Whizcomms 1Gbps.

But severely underutilized the bandwidth so going to decommission 1 WAN soon.

Your pfsense box.......much much more powerful than my Yr 2011 cpu box! Sure no sweat!:D

I think yours can still pretty much game very well with a GPU.
 

zoneguard

Senior Member
Joined
Jun 2, 2000
Messages
1,961
Reaction score
399
Your pfsense box.......much much more powerful than my Yr 2011 cpu box! Sure no sweat!:D

I think yours can still pretty much game very well with a GPU.

No lah, it's not bare metal like yours. Got other VMs running on it. And it is a mITX motherboard like yours and the sole PCI-e slot is taken up by the 4 x NICs so no GPU.

I don't have server rack also.

Just curious, why you went with so many different vendors instead of say all Mikrotik outfit or all Ubiquiti outfit which is more common in this forum? Like that, you need to learn and be conversant in everything - is Ruckus very different from the rest?
 

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
No lah, it's not bare metal like yours. Got other VMs running on it. And it is a mITX motherboard like yours and the sole PCI-e slot is taken up by the 4 x NICs so no GPU.

I don't have server rack also.

Just curious, why you went with so many different vendors instead of say all Mikrotik outfit or all Ubiquiti outfit which is more common in this forum? Like that, you need to learn and be conversant in everything - is Ruckus very different from the rest?

My decision to go server rack: wanted to optimize the available cabinet space. 3yrs ago before moving into the unit, always wanted to go with server rack when setting up my smart home. But along the way, with busy work schedule and new baby coming, resort to OTS devices and no-planning setup! Hope this time round, I can do a revamp! :s13:

Hope by sharing how I think about the different brand before I made the decision can provide you with some insight. Brand supporters please dun flame me.

Mikrotik:
Pros: Cheap 10Gbe switches, SwOS is also pretty straight forward. Reliability wise, it just works.
Cons: Don't like their ROS and GUI. Looks ancient. Documentations is not great. If you learning on your own, takes some time to learn the clunky interface and how things work.

Unifi:
Pros: Make good switches other than Netgear. Good APs too, good and simple, nothing more. Good pricing.
Cons: tries to sink you into the whole "Unifi" hole which I am not a fan of. Their UDM and USG is better than most consumer routers out there but compared to firewall/router of the same class, they are still quite far. Lots of on/off setting in their UDM/USG series but you will not know what is going on behind the scene and how they actually work. I will say give it a pass if you want pro features with greater flexibility. There are just much better solution out there.

pfsense:
Pros: Great documentation and community support. Proven reliability and versatility for both consumer and business needs. Stable updates. Fulfill basic needs if you are a novice user. I have been using pfsense since I started learning the basics of firewall, so with that experience, make it hard to switch.
Cons: You probably need to build one yourself. Do take time to understand and learn some of the functions if you are used to consumer routers.

Ruckus:
Pros: reliability, set and forget, proprietary antenna design which performs really well in "noisy" environment, require less APs in a similar environment as compared to other APs (based on experience). Stable firmware updates.
Cons: Expensive unless there is a sale. I managed to use the left-overs from one of my projects, so got it at a discount.

In the end when it comes to decision making I will choose reliability, cost, knowledge and experiences.
 
Last edited:

xiaofan

Arch-Supremacy Member
Joined
Sep 16, 2018
Messages
18,107
Reaction score
2,885
BTW, just wondering what prompts you to upgrade from 1Gbps to 10 Gbps for the home network. Any real use cases at home? Or is it more for future proof?

Is it possible to save some money if setting up an interim 2.5Gbps home network?

I notice that you are still using 2x1Gbps Internet (SingTel and M1). Is it really required as both are pretty reliable?
 

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
BTW, just wondering what prompts you to upgrade from 1Gbps to 10 Gbps for the home network. Any real use cases at home? Or is it more for future proof?

Is it possible to save some money if setting up an interim 2.5Gbps home network?

I notice that you are still using 2x1Gbps Internet (SingTel and M1). Is it really required as both are pretty reliable?

Future proofing, have some fun, transfer large files from PC to 2 NAS library in different rooms (living room and homelab).....more keen towards having some fun to set it up. :D

Maybe it is only me, I always wanted to start with 10Gbe 3 years ago, knowing the unit's internal cabling is Cat6. Going 1Gbe seems a waste to the existing infrastructure, and I rather future proof it.

For my case, the Mikrotik CRS312 does comes with 8x multi-gig, but my NIC only runs at 1Gbe or 10Gbe, so might as well 10Gbe. Not really sure if 2.5Gbe switches will save some money coz never had any chance to compare. I know QNAP will soon launch an affordable unmanaged 2.5Gbe switch soon. For my case I will prefer managed/smart switches which can be tag with VLANs.

Not using M1. I am running MyRepublic 1Gbps as default ISP for all LAN connected devices, Singtel 1Gbps for wifi, CCTV and IOTs. I do have some critical work linked to my workplace system which I could not have any downtime when working/control remotely from home. Having 2x ISPs are part of the BCP, even my network servers are backed with UPS just in case power trip.
 

dodge888

Junior Member
Joined
Apr 29, 2020
Messages
57
Reaction score
5
Very nice. I used to run the following for my home network :

- pfsense 2.4.4 with an old Dell SFF (3rd gen i5) with 4-port Intel gig NIC
- Netgear GS108PE
- Ubiquiti Unifi AC Pro
- VLANs to segment my networks

My gear all 2nd hand except for the Netgear managed switch. pfsense is still my preferred firewall. Could max out my 1gbps Internet using OpenVPN with multi-WAN setup.
 

BradenHeat

Supremacy Member
Joined
Apr 4, 2005
Messages
5,337
Reaction score
621
:spin: :s31: :s22:

Power sia, can feel my wallet screaming at me not to get enticed

Simply epic set up you have man

P.s have you thought of funneling the fans into one way in and one way exhaust?

From the picture it seems it will be all over the place for airflow if you’re trying to keep it cool

Just curious and thinking out loud

Posted from PCWX using iPhone12,5
 

TanKianW

Supremacy Member
Joined
Apr 21, 2005
Messages
5,905
Reaction score
2,220
:spin: :s31: :s22:

Power sia, can feel my wallet screaming at me not to get enticed

Simply epic set up you have man

P.s have you thought of funneling the fans into one way in and one way exhaust?

From the picture it seems it will be all over the place for airflow if you’re trying to keep it cool

Just curious and thinking out loud

Posted from PCWX using iPhone12,5

Yes, will direct all fans to exhaust out of the cabinet in one direction.

The many fans set up you see in the top picture is actually my current set up. Have not upgraded yet.

In this project, all default 40mm fans will all change to Noctua 40mm silent fans. Will be replacing 6 of them, 2 on my 1U pfsense chassis, 4 on Mikrotik CRS312. Will change the direction of the Dynatron 1U CPU blower fan to blow out in same direction too.
 

Repsol_2

High Supremacy Member
Joined
Feb 10, 2001
Messages
39,203
Reaction score
347
I'm running pfSense (virtualized) on i7-4790S with 6 x Intel GBe NICs.

WAN1: M1 1Gbps static IP.
WAN2: Whizcomms 1Gbps.

But severely underutilized the bandwidth so going to decommission 1 WAN soon.

should keep both WAN link and do a Load Balance instead. 1 ISP down still got secondary link. Dual ISP full diversity. Just need to make sure "Source and Destination IP Address Binding" so your internet banking or transaction don't get confused.
 
Last edited:

BradenHeat

Supremacy Member
Joined
Apr 4, 2005
Messages
5,337
Reaction score
621
:s22: :s22: wa Lao eh all the “rich” people 😭😂 now making me gian full setup with NAS and etc

Posted from PCWX using iPhone12,5
 

leechoonhwee

Member
Joined
May 27, 2001
Messages
276
Reaction score
23
I'm also in the midst of upgrading the home network but mine is significantly simple as I only have one NAS and my objective is more towards faster Internet versus faster internal network

So what I got so far is

Pfsense running on a used server from office with 4x sfp+ ports
H3C sfp+ switch
Asus ac5300 ap
10gb Internet from superinternet set as tier 1 gateway
1gb starhub fibre set as tier 2 gateway

Getting an Asus ax89x to connect directly to the pfsense so that hopefully my ax devices can surf past 1gb speed. 😂

Its super overkill setup for home but these hobby setups is really just for prove of concept to bring peace to our own mind and soul 🤣
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ. Forum members and moderators are responsible for their own posts.

Please refer to our Community Guidelines and Standards, Terms of Service and Member T&Cs for more information.
Top