Containers for home network

[xiaofan]

librespeed is pretty good, easy to setup.

Wireless speed from my Acer laptop with Intel AX201 card to Asus RT-AX82U router

Wireless speed from my Acer laptop with Intel AX201 card to Singtel Mesh router

 

[xiaofan]

Smokeping also works fine.

Just one day of running, from Singtel network, looks like Google DNS is the best (all around 3ms), then followed by Cloudflare (there is a few spikes), then OpenDNS got more spikes. Quad9 has high delays (>=150ms).

 

[firesong]

Smokeping also works fine.

Just one day of running, from Singtel network, looks like Google DNS is the best (all around 3ms), then followed by Cloudflare (there is a few spikes), then OpenDNS got more spikes. Quad9 has high delays (>=150ms).

Yep. But given the performance over time, CF is not very far from Google DNS. But the functionality improvement for the family tier and the anti-malware DNS sinkhole is worth the slight penalty.

I've been observing it over time, and generally it's just a few miliseconds behind, nothing major.

 

[xiaofan]

Yep. But given the performance over time, CF is not very far from Google DNS. But the functionality improvement for the family tier and the anti-malware DNS sinkhole is worth the slight penalty.

I've been observing it over time, and generally it's just a few miliseconds behind, nothing major.

Yes, I have the same opionion.

On my Asus rotuer side, I am using Pi-hole (On Google Cloud as of now, will shift to local docker instance later) and 1.1.1.3 for my family.

On the OpenWRT router side, I am using 1.1.1.3 as the upstream DNS (DoH), along with openwrt adblock package. Previously when I used pfSense on this network, I was using 1.1.1.3 as the upstream DNS along with pfBlockerNG.

 

[BradenHeat]

Try to get some of the mentioned stuff wrapped around :

- piHole and the mini custom build labs, are they meant for long term security adblock and filtering ?

- since ive been reading the stuff listed, the openWrt QoS settings over those dedicated devices,

did yáll notice very significant improvements for daily driving those setup over say

vs Just one edge router + AP/s setup

vs normal [ asus/ orbi/ mesh setup ]

etc ?


From the security perspective, i could see benefit of setting one up, to filter nasty pages, by pre-loading and filtering off

but for others, i seem not to comprehend on the Docker.


sorry, very noob when it comes to this, as my last known plaything was VM fusion stuff[ 10years back, return to employer liao ], and nothing on network, closest was NGFW but that was for work requirement lmao still dont get most of it brain just drones off


ISP--> fiber router --> pi Hole/ custom server [ docker setup ] -->wireless router
benefits = security and adblocking ?
+ QoS ?

at least thats what as a novice im looking at [ correct right ? lol]

 

[xiaofan]

You can have many variations when it comes to ads blocking. Client side solution like Ublock Origin may still be required other than the following solution. As for Docker, it is just a convenient deployment method for Pi-Hole or AdguardHome, as you can use one Linux server (eg: running on a Raspberry Pi 4/400, or a mini PC) to run multiple containers.

But you do not need to use Docker at all as you can deploy Pi-Home or AdguardHome without Docker, say, on a Raspberry Pi (say 3B+ or 4B).

A. Using consumer router
1) ONT --> main router (Asus/etc) --> wireless AP, with pi Hole [ docker or non-docker ] as the DNS server for the whole network, or selected devices

Benefits = security and adblocking

2) ONT --> main router (Asus/etc) --> wireless AP, with AdguardHome [ docker or non-docker ] as the DNS server for the whole network, or selected devices

Benefits = security and adblocking

Similar to 1. Some people like AdguardHome and some people like Pi-Hole.

B. Using pfSense
1) ONT --> pfSense with pfBlockerNG, pfBlockerNG is similar to Pi-hole but probably a bit more powerful

2) ONT --> pfSense with Pi-hole as DNS server

C. Using OpenWRT
1) ONT --> openwrt with Pi-hole as DNS server

2) ONT --> openwrt with Adblock or Simple Adblock package, similar to Pi-hole but probably a bit less powerful

3) ONT --> openWRT with AdguardHome package, similar to using OpenWRT with Pi-hole, but more integrated

 

[xiaofan]

I do not think it is a good idea to run main OpenWRT router on a docker (maybe not possible either). Rather the idea is to have the OpenWRT docker images as a gateway for selected device, as it is a lightweight Linux based OS with extra packaages to achive different functions, like DoT or DoH, ads blocking, or as a VPN server/clients, etc.

I think QoS should probably implemented on the main router no matter you use consumer router, OpenWRT or pfSense.

Reference in Chinese: I like the idea of running OpenWRT as a gateway but I do not like the image as it is not based on offical OpenWRT but rather a fork, more suitable for Chinese users.
https://hub.docker.com/r/sulinggg/openwrthttps://mlapp.cn/376.html

 

[xiaofan]

BTW, if you want to try out Pi-Hole but do not want to invest in any hardware, just use the free Google Cloud (easier to use but more limitations) or free Oracle cloud (less limitations but IMHO more difficult to use).

AdguardHome is a bit heavier so the free Google Cloud compute engine may be a bit weak to run.

If you want to invest in some cheap HW to run docker, then go with either Raspberry Pi 4/400, or with a low power mini PC, or use your old PC/laptop if you still have them.

https://forums.hardwarezone.com.sg/...ole-blocking-on-google-cloud-compute.6375286/

 

[firesong]

You can have many variations when it comes to ads blocking. Client side solution like Ublock Origin may still be required other than the following solution. As for Docker, it is just a convenient deployment method for Pi-Hole or AdguardHome, as you can use one Linux server (eg: running on a Raspberry Pi 4/400, or a mini PC) to run multiple containers.

But you do not need to use Docker at all as you can deploy Pi-Home or AdguardHome without Docker, say, on a Raspberry Pi (say 3B+ or 4B).

A. Using consumer router
1) ONT --> main router (Asus/etc) --> wireless AP, with pi Hole [ docker or non-docker ] as the DNS server for the whole network, or selected devices

Benefits = security and adblocking

2) ONT --> main router (Asus/etc) --> wireless AP, with AdguardHome [ docker or non-docker ] as the DNS server for the whole network, or selected devices

Benefits = security and adblocking

Similar to 1. Some people like AdguardHome and some people like Pi-Hole.

B. Using pfSense
1) ONT --> pfSense with pfBlockerNG, pfBlockerNG is similar to Pi-hole but probably a bit more powerful

2) ONT --> pfSense with Pi-hole as DNS server

C. Using OpenWRT
1) ONT --> openwrt with Pi-hole as DNS server

2) ONT --> openwrt with Adblock or Simple Adblock package, similar to Pi-hole but probably a bit less powerful

3) ONT --> openWRT with AdguardHome package, similar to using OpenWRT with Pi-hole, but more integrated

A bit more value add.

All the other adblockers like PiHole and the browser addons (uBlock, Adblock Plus, Adguard Home) are DNS based adblockers. A DNS based adblocker works by sinkhole - so it requires a list of DNS resolved names in order to do that. As in, assuming forcesafesearch.google.com is an ad domain, and that it normally points to 216.239.38.120, instead it sinks that by pointing it to something random like 10.0.10.0. So your computer will cannot resolve that ad host. However, more and more ad servers skip using DNS and directly use the IP addresses of ad servers , making all these adblockers redundant. So rather than point to ad.host.domain, they instead point to the direct IP address. These will not be blocked by all these adblockers.

pfBlocker-NG goes one step deeper - it drops the IP addresses at the firewall level. So in the above example, even if you get one of those new hosts that skips "forcesafesearch.google.com" and instead uses 216.239.38.120, the ad will still be blocked because the IP address is blocked at the firewall level. Given that the same one IP address can be the target for many hosts (you can add a few thousand different unique hosts+subdomains off that one IP if you so desire), blocking one single IP address rather than entering tens of thousands of lines of domains in a blocklist is definitely simpler. This is much more effective. And because it works at the firewall level, it does something more than tools like PiHole, AdGuard or browser adblockers can do. Because it is capable of blocking at the IP level, it is also more effective at blocking known malicious hosts better - assuming they are either at the same IP block for dynamic hosts, or are static hosts (which many compromised servers are). This makes it incredibly more effective than PiHole, and why I was happy to move to this solution.

 

[BradenHeat]

ahhh , ok , thanks @xiaofan @firesong basically, the security is given, however disregarding the level of protection from ISP, its still gonna equate more network performance as the blocking is offloaded prior to sending the traffic to end clients [ e.g iphone and laptops ]

so for most, the docker does act as a learning for indepth understanding similar to NGFW, but with certain limitations ?

at least thats what im inferring to, since im very limited to Docker and other homelabs [ apart from PA200 from palo, which still confuses the Fk out of me had to learn for work ]

 

[firesong]

ahhh , ok , thanks @xiaofan @firesong basically, the security is given, however disregarding the level of protection from ISP, its still gonna equate more network performance as the blocking is offloaded prior to sending the traffic to end clients [ e.g iphone and laptops ]

so for most, the docker does act as a learning for indepth understanding similar to NGFW, but with certain limitations ?

at least thats what im inferring to, since im very limited to Docker and other homelabs [ apart from PA200 from palo, which still confuses the Fk out of me had to learn for work ]

It appears you are mixing up some things.

1. Containerisation suites like Docker are used as a means of distributing "pre assembled software" with the necessary configurations/environments so that it's the same across computers. That's really it. Rather than install it bare metal to the computer and having to ensure that all configurations are identical, you just wrap it in a Docker container for distribution.

I can "pre-configure" software in a manner that I know will run the way I set it up on my own development machine with a high degree of predictability. This saves a lot of time. If you've ever developed software and tried to migrate it to other machines, it's a real pain when external dependencies are not included. Microsoft tried to solve this with DLL and .NET packages, and it leads to a lot of unnecessary installations. On *nix, you're expected to set up certain common environments too. There's a reason why things like LAMP/LEMP are mainstays, and how package managers like apt became so popular compared to the old days when we literally had to compile applications each time we install them - the trifecta of configure, make, and make install was a pretty common process before, and the configuration scripts were considered fairly "convenient" then.

2. Adblock solutions are just that - they block ads, and sometimes malicious sites. It's just that it does it via DNS blacklisting. So it blocks them through blocking their resolved domain names. There are pros and cons to this approach, since it does not matter if the backend IP address changes, since the name is static. However, as explained above, since this is the main way all adblockers work, the ad agencies are smart enough to not use domain names, but rather work around it and use static IP addresses.

Simply, using PiHole, Adguard, web browser blockers or others are only half as effective, and losing effectiveness. Why? Because sites depend on ads to earn money. They WANT to workaround all these common blocks and serve ads anyway, and they will do anything and everything to work around it. Since the ad servers are fairly constant, blocking their IPs or IP ranges is more effective. Simply put, a solution like pfBlocker-NG blocks both the DNS and the IP addresses so is more comprehensive and hence effective. Why bother to install an adblocker that's less effective, and losing effectiveness since all agencies want to push their ads to users at all cost? To further complicate matters, there are other ad serving mechanisms too.

Ads are annoying, but the ones that are rather more malicious are trackers and actual malware. Trackers that compile your user profiles simply by what you surf, what you post, and all can form very realistic collections - and this really isn't what we want to have in the hands of other anonymous entities out there that use it for their own profit. The same subversion mechanisms can be used to distribute trojans and other malware, and are definitely cause for concern.

Hope this helps clarify.

 

[xiaofan]

pfBlockerNG is for sure more advanced but it is tied to pfSense. So yes you may want to run pfBlockerNG if you use pfSense.

Pi-hole is probably good enough for majority of the people. And it is more flexible (you can use it along with your consumer grade router, you can run it as a docker container, use a local machine or on the cloud) and the pi-hole interface is arguable much better than pfBlockerNG.

BTW, there are also people who still use pi-hole even when they run pfSense as the main router.
An example: https://www.vikash.nl/moved-from-pfblockerng-to-pi-hole/ (probably his issue is not a real pfBlockerNG issue -- but pi-hole is indeed easier to use).

 

[firesong]

pfBlockerNG is for sure more advanced but it is tied to pfSense. So yes you may want to run pfBlockerNG if you use pfSense.

Pi-hole is probably good enough for majority of the people. And it is more flexible (you can use it along with your consumer grade router, you can run it as a docker container, use a local machine or on the cloud) and the pi-hole interface is arguable much better than pfBlockerNG.

BTW, there are also people who still use pi-hole even when they run pfSense as the main router.
An example: https://www.vikash.nl/moved-from-pfblockerng-to-pi-hole/ (probably his issue is not a real pfBlockerNG issue -- but pi-hole is indeed easier to use).

Yeah. It's likely because they don't want to put in the effort to learn to use pfBlockerNG since there's a learning curve. It is quite a bit more involved since it also involves having to set floating firewall rules. But once you understand it or have a good guide like Lawrence Systems', it's pretty decent.

IMO, Viknash didn't want to work through his Tradfri solution and find or add the necessary exclusions. He is right in that it's a bit more difficult to dig through the DNSBL reports.

 

[xiaofan]

so for most, the docker does act as a learning for indepth understanding similar to NGFW, but with certain limitations ?

at least thats what im inferring to, since im very limited to Docker and other homelabs [ apart from PA200 from palo, which still confuses the Fk out of me had to learn for work ]

If you can learn more advanced stuff from Cisco and Palo Alto Networks (which I have no chance to have access to, other than some managed industrial network switches OEM from Cisco), for sure you should be able to learn Docker or other container related technology. You can say it is very popular now.

Ref:
https://en.wikipedia.org/wiki/Docker_(software)https://en.wikipedia.org/wiki/Kubernetes

 

[BradenHeat]

Yep,

crystal clear haha, thanks man

its just similar to the electron, but machine code like with native performance ? so all apps can just simply "dock"

and that the firewall / adblock etc containers just so happen to be avaliable and easily transferrable

got it though im still not convince on electron lol

It appears you are mixing up some things.

1. Containerisation suites like Docker are used as a means of distributing "pre assembled software" with the necessary configurations/environments so that it's the same across computers. That's really it. Rather than install it bare metal to the computer and having to ensure that all configurations are identical, you just wrap it in a Docker container for distribution.

I can "pre-configure" software in a manner that I know will run the way I set it up on my own development machine with a high degree of predictability. This saves a lot of time. If you've ever developed software and tried to migrate it to other machines, it's a real pain when external dependencies are not included. Microsoft tried to solve this with DLL and .NET packages, and it leads to a lot of unnecessary installations. On *nix, you're expected to set up certain common environments too. There's a reason why things like LAMP/LEMP are mainstays, and how package managers like apt became so popular compared to the old days when we literally had to compile applications each time we install them - the trifecta of configure, make, and make install.

2. Adblock solutions are just that - they block ads, and sometimes malicious sites. It's just that it does it via DNS blacklisting. So it blocks them through blocking their resolved domain names. There are pros and cons to this approach, since it does not matter if the backend IP address changes, since the name is static. However, as explained above, since this is the main way all adblockers work, the ad agencies are smart enough to not use domain names, but rather work around it and use static IP addresses.

Simply, using PiHole, Adguard, web browser blockers or others are only half as effective, and losing effectiveness. Why? Because sites depend on ads to earn money. They WANT to workaround all these common blocks and serve ads anyway, and they will do anything and everything to work around it. Since the ad servers are fairly constant, blocking their IPs or IP ranges is more effective. Simply put, a solution like pfBlocker-NG blocks both the DNS and the IP addresses so is more comprehensive and hence effective. Why bother to install an adblocker that's less effective, and losing effectiveness since all agencies want to push their ads to users at all cost? To further complicate matters, there are other ad serving mechanisms too.

Ads are annoying, but the ones that are rather more malicious are trackers and actual malware. Trackers that compile your user profiles simply by what you surf, what you post, and all can form very realistic collections - and this really isn't what we want to have in the hands of other anonymous entities out there that use it for their own profit. The same subversion mechanisms can be used to distribute trojans and other malware, and are definitely cause for concern.

Hope this helps clarify.

 

[xiaofan]

With the help of macvlan, it is no longer an issue to set up AdguardHome, as it has its own IP address and will not conflict with the Ubuntu docker host, similar to the solution with Pi-hole.

Ref: https://github.com/andrewjmetzger/docker-compose-files/blob/master/adguard-home/docker-compose.yml

 

[xiaofan]

Last time I was using a few Docker applications (Pi-hole and others) using the free VPS (Google and Oracle) without good success.

Today I cleaned up a bit on my three "forever free" VPS virtual instances (all three running Ubuntu 20.04, all three servers located in US). I will keep the free Google cloud virtual instance dedicated for Pi-hole. Then I got wireguard VPN working for the two free Oracle virtual instances (tested using Windows and Android). Previously one of them was not working because of wrong firewall port configuration.

And then I spent some time to get XRay proxy (replacement for V2Ray) working by following a YouTube video using gRPC. It seems to be pretty okay. Windows client using QV2Ray works fine. But somehow I did not get Android client to work (V2RayNG).

Maybe I will try a few light weight docker containers on the two Oracle virtual instance as they have 1GB RAM and more outbound data quota.

 

[firesong]

Yes, I have the same opionion.

On my Asus rotuer side, I am using Pi-hole (On Google Cloud as of now, will shift to local docker instance later) and 1.1.1.3 for my family.

On the OpenWRT router side, I am using 1.1.1.3 as the upstream DNS (DoH), along with openwrt adblock package. Previously when I used pfSense on this network, I was using 1.1.1.3 as the upstream DNS along with pfBlockerNG.

This is the typical reading over time. The delta is usually not significant unless routing issues come up.

 

[xiaofan]

Yes, I have the same opionion.

On my Asus rotuer side, I am using Pi-hole (On Google Cloud as of now, will shift to local docker instance later) and 1.1.1.3 for my family.

On the OpenWRT router side, I am using 1.1.1.3 as the upstream DNS (DoH), along with openwrt adblock package. Previously when I used pfSense on this network, I was using 1.1.1.3 as the upstream DNS along with pfBlockerNG.

Just switched both of my two home networks (Asus RT-AX82U router and Raspberry Pi 400 Openwrt router) to local Pi-hole docker container as the LAN DNS sever. Upstream WAN DNS is still using 1.1.1.3.

Test results are 100% with the following test site.
https://d3ward.github.io/toolz/adblock.html
I encountered a few issues this afternoon, main thing is to fix the Ubuntu host side, Initially I was puzzled why DNS leak test site always show Google as the DNS, and later I found out it was becasue of systemd-resolved settings on the localhost and then Docker Containers inherited it as I specified 127.0.0.1 as the main DNS in the docker-compose.yml file.

As for AdguardHome docker, I will use it as a testing DNS server for selected device. I still like Pi-hole more as I feel the interface is very clean and very easy to use. It is also very easy to backup and replicate the settings.

 

[xiaofan]

So more or less I have finished the first phase of my learning.

1) Portainer (installed),
2) Homer (installed),
3) Pi-hole (installed and deployed two instances to the home networks)
4) AdguardHome (installed and testing now)
5) OpenWRT (installed an unofficial version and do not quite like it as it does not use official repo)
6) Nginx (installed but need to learn how to use the basic)
7) MariaDB (installed but need to learn how to use the basics, not going to learn too much about this)
8) Nextcloud (installed but need to learn the basic)
9) Smokeping (installed and running well)
10) librespeed (installed and running well)
11) netshoot (installed) This seems to be a good troubleshooting tool. https://hub.docker.com/r/nicolaka/netshoot