Starting and Sharing your Homelab Projects (Include 10G Network Upgrade)

[Project_Xco]

2) Tested all the Cat6 points around the house and found that 2 out of 6 Ethernet point could not run at 10G. Which may be due to wear and tear of connection point or cable degradation. will get a contractor in to restore these data connection point.

How did you perform this test to find out if they run at 10Gbps?

 

[TanKianW]

How did you perform this test to find out if they run at 10Gbps?

I engaged a network contractor with the 10G testing gear to test for me. (Unless you are lucky enough to get an electrician who has the gear to do it.)

OR

Run 10G test servers at both ends.

 

[TanKianW]

*In Progress* Recent Upgrade for the 10G switch in my Homelab:

Will update on the "under-table" custom mount rack set up, once it has been completed.

From MikroTik CRS305:

To MikroTik CRS309:

 

[theinquirer]

Before:

After:

Full working setup:

10GE on copper!



85 yrs old grandma driving a 850hp ferrari



heng ar

all 10GE is fiber LC cable

 

[TanKianW]

**Update: Under-table mount of open concept rack for 10GbE Switch Upgrade**

Ignore the "before" cable management mess. For those seeking solutions to house their rack mount network appliances/servers:​

(Top) MikroTik CRS309 on RouterOS, (Bottom) *OLD* EdgeSwitch 10XP for 1G connections.

 

[TanKianW]

Some Tips when cable manage your Network Setup:

1. Color code your cables, (Eg. downstream/upstream, LAGG, spare, L2/L3, etc)
2. Group/organize your cables/ports, (Eg. similar or backup cables connected side by side)
3. Use horizontal/vertical cable management devices/rails/trays (for larger racks)
4. Use patch panel
5. Use cable ties or reusable Velcro stripe (preferred)

Simple cable management makes a big difference:
BEFORE:

AFTER:

 

[xiaofan]

Maybe it now makes more sense to go with 10G when new generation of Wifi 6E routers start to offer 10Gbps port.

Ref: TP-Link Archer AXE300 comes with a 10 Gbps WAN/LAN Port and a 10 Gbps SFP+/RJ45 Combo LAN Port.
https://www.tp-link.com/au/home-networking/wifi-router/archer-axe300/
TP-Link Archer AXE200 Omni comes with a 10Gbps port.
https://www.tp-link.com/us/home-networking/wifi-router/archer-axe200-omni/
TP-Link China also offers two Wifi 6E router with dual and single 10Gbps port.
https://www.tp-link.com.cn/product_cattag_2080_107_961_01_02_03.html
Previously very few Wifi 6 home routers come with 10G port, like Asus RT-AX89X and QNAP QHora-301W.
Ref: https://dongknows.com/multi-gigabit-wi-fi-6-routers-to-bring-home-today/

 

[TanKianW]

**UPDATED ON 30 July 2023**

Up my old thread with some updates, since I expect more home users will move to 10G internet if everything goes smoothly next year on the ISP side. Some might therefore start to wonder how they can optimize their home network. Sharing my current 10G-ready set up (since a lot has changed 3yrs back) with forum-ers who are interested or curious to learn more. Who knows, it might even inspire you to build your own home lab!

1) Protection: Starting with the firewall and core switch
2x 10G SFP+ and 3x 1G + DIY firewall appliance running pfsense+ 23.05.1, LAGG to MikroTik CRS312 (ROSv7) 10G core switch. Learn more on DIY pfsense check out HERE

2) Distribution: Downstream (L2) Switches in home lab/office
MikroTik CRS309 10G SFP+ switch and MikroTik CRS326 1G/10G switch (ROSv7). For connecting to storage servers and hypervisors.

3) Servers & Services: Hypervisor systems (in home lab)
2x Nodes hypervisor (2U) systems on XCP-ng+XOA with 2x 10G networking each backup by ZFS storage system

4) Backup/Services: Storage Systems (in home lab)
2x TrueNAS storage systems with 10G networking, for running services/servers and backing up hypervisors. Learn more on DIY NAS like TrueNAS check out HERE

 

[xiaofan]

This thread will gain some traction now that more ISPs are coming out 10Gbps plan.

 

[TanKianW]

**SHARING: Cheap, self-assembled open rack solution for your network gears from taobao.**
Personally, I find it sturdy and good enough for someone building their homelab on a rack. Interested can PM me for link.

 

[TanKianW]

**SHARING: AP Vertical In-Cabinet Support, 10G NICs Recommendations, and TP-LINK POE+ managed switch with 2.5G and 10G Uplink**

For those thinking about how to keep their APs in an upright position without mounting vertically when kept in a cabinet. You can consider using a book stand from Daiso, perfect support at an affordable price! Please don't comment about the coverage, this passed wifey's approval is more important!

Affordable TP-link TL-SE2109PB with 2.5G POE+ (af/at), smart-managed capability, 1x 10G SFP+ uplink which can also do simple VLAN tagging.

Some 10G NICs that worth considering:
From left to right: Intel X540-T2 with NVMe combo card (requires PCIe bifurcation), 10Gtek Intel X550-T2, 10Gtek Intel X520-DA1 (SFP+), (OEM) Intel X520-DA2, Chelsio T520-LL-CR (low latency).

 

[TanKianW]

**For Reference: Common Inter-Connectors Options for your SFP+ 10G devices/Ports (Not Exhaustive)**

1) LC (Lucent Connectors) fibre transceivers Module for SFP+:

When connected to SFP+ Device/Ports:

2) DAC (Direct Attached Copper Twinax) for SFP+,:

When connected to SFP+ Device/Ports:

3) RJ45 transceivers Modules (10GBase-T) for SFP+:

When connected to SFP+ Device/Ports:

 

[Qubicfactor]

I engaged a network contractor with the 10G testing gear to test for me. (Unless you are lucky enough to get an electrician who has the gear to do it.)

OR

Run 10G test servers at both ends.

Can PM me the network contractor? TQIA

 

[TanKianW]

**SHARING: Relocating my 10G (or more) rack servers into storage room (HDB bomb shelter)**
A long coming project planned since last year Oct that procrastinated for quite a long while due to busy work schedules and traveling. The project is both for work production and Homelab purposes (Eg. Home Assistant, PLEX, Nextcloud, Minecraft game servers for my kids, storage for all my family and extended family. Plan is also to run localised AI/ML server when the new RTX5090 launch. Therefore the huge rack for storage and a few U for GPU. Few things I have done over the weekend:​

1. Retrofitted the storage room ventilating hole with mechanical (exhaust) ventilation fan from Panasonic. 3d printed the frame for a better airflow (not yet installed)​
2. 3d printed the rack mount for minisforum MS-01. Likely to run as a hypervisor machine with a low profile GPU for video transcoding. Planning to print some custom rack mount cable management brackets/hooks for routing the network/power cables after upgrading my good old Prusa to MK3.5.​
3. Pulled two Cat6 cables from the top cabinet network appliance rack into the store room through another ventilation hole. At the same time took some time to trunk the cables using cable cover/hides to the prepared location for the server rack.​
4. Pushed in the pre-racked server rack with castor wheels. Took me some time to do so due to the sheer weight from 12x HDD. Lessons learnt: could have pushed the empty rack in before racking the equipment up, but I am lazy to remove one by one after setting it all up before.​
5. Route the Cat 6 cables to the back of the server rack, power the 10G managed switch up. Set it up with LAGG (LACP) to the upstream core switch, connect 2 power cables to the switch for network and power redundancy (though 2x identical switches running on RSTP/MSTP (or iBGP) could have been even better with best route switch redundancy. Plan is one power to one power point, leading to a separate 2200VA APC UPS. One more UPS arriving next week.​
6. Took down my PiKVM.v3 from my network appliance rack and will be using it on the server rack for remote KVM.​

Project in progress: A glimpse on the lighting up of the rack's 10G core managed switch CRS312 from MikroTik running on latest ROS v7.14.3. Tested that there is no signal degradation on the twin 10G LAGG cable using my personal FLUKE meter before commissioning it. Maybe can named as "Starting your Home Data-centre" very soon.

*Will update more along the way when there is progress (kept my fingers crossed)*

 

[hwzlite]

"Starting your Home Data-centre"

Wow....u upped the game make us FOMO liao....
Btw what your ave power consumption/bill$ per month ?

 

[jasonho]

**SHARING: Relocating my 10G (or more) rack servers into storage room (HDB bomb shelter)**
A long coming project planned since last year Oct that procrastinated for quite a long while due to busy work schedules and traveling. The project is both for work production and Homelab purposes (Eg. Home Assistant, PLEX, Nextcloud, Minecraft game servers for my kids, storage for all my family and extended family. Plan is also to run localised AI/ML server when the new RTX5090 launch. Therefore the huge rack for storage and a few U for GPU. Few things I have done over the weekend:​

1. Retrofitted the storage room ventilating hole with mechanical (exhaust) ventilation fan from Panasonic. 3d printed the frame for a better airflow (not yet installed)​
2. 3d printed the rack mount for minisforum MS-01. Likely to run as a hypervisor machine with a low profile GPU for video transcoding. Planning to print some custom rack mount cable management brackets/hooks for routing the network/power cables after upgrading my good old Prusa to MK3.5.​
3. Pulled two Cat6 cables from the top cabinet network appliance rack into the store room through another ventilation hole. At the same time took some time to trunk the cables using cable cover/hides to the prepared location for the server rack.​
4. Pushed in the pre-racked server rack with castor wheels. Took me some time to do so due to the sheer weight from 12x HDD. Lessons learnt: could have pushed the empty rack in before racking the equipment up, but I am lazy to remove one by one after setting it all up before.​
5. Route the Cat 6 cables to the back of the server rack, power the 10G managed switch up. Set it up with LAGG (LACP) to the upstream core switch, connect 2 power cables to the switch for network and power redundancy (though 2x identical switches running on RSTP/MSTP (or iBGP) could have been even better with best route switch redundancy. Plan is one power to one power point, leading to a separate 2200VA APC UPS. One more UPS arriving next week.​
6. Took down my PiKVM.v3 from my network appliance rack and will be using it on the server rack for remote KVM.​

Project in progress: A glimpse on the lighting up of the rack's 10G core managed switch CRS312 from MikroTik running on latest ROS v7.14.3. Tested that there is no signal degradation on the twin 10G LAGG cable using my personal FLUKE meter before commissioning it. Maybe can named as "Starting your Home Data-centre" very soon.

*Will update more along the way when there is progress (kept my fingers crossed)*

No green not nice

 

[TanKianW]

Wow....u upped the game make us FOMO liao....
Btw what your ave power consumption/bill$ per month ?

When all the equipment 点亮 then let you know. Now only a switch and a mini-pc, not yet significant in terms of power consumption. Will take up more power when I light up the TrueNAS Scale storage server and the fully built-up AI/ML server.​

 

[koohansen]

Now you need a 5kVA UPS, Smart PDU

 

[hwzlite]

Now you need a 5kVA UPS, Smart PDU

Psst...Psssst.... His next potential covert project might go solar...

 

[TanKianW]

Psst...Psssst.... His next potential covert project might go solar...

If got the intention to do, will likely not be so haphazard. Not legal for HDB too, due to falling hazards.

Actually got plan to install some, just not for home use, but at my current workplace roof-top to (supplement) power the worklab/production servers. May not need to spend a cent too since the upfront cost paid fully by the vendor (they call "Zero Capital Partnership"). Just purchase the power from them at a standard rate until ROI then the assets become our own.​