Work machine saturating network - solutions?

[crombie]

So I ran into an issue this morning where some random update on my work machine saturated our connection. Naturally I went poking around and of course Deco does not have 'true' QoS. Technically I can set devices as priority except for the work device, but our phones randomize MAC and I see them constantly being added as 'new devices'.

I saw suggested the possibility of using SQM through OpenWRT, but all I have are 32 bit machines which will not run SQM as it requires 64bit.

NOW I had thought that OpenWrt being the wi-fi point with only 2.4GHz access would limit the potential draw but I guess the connection is technically 'n'. So in theory faster than our internet connection. (300Mbps down / 10 Mbps up)

So am I basically out of options now? Especially since I have no idea what the culprit was on the work machine, but multiple people were experiencing the same issue.

I mean, I have the same problem with torrents or Steam and all I needed to do was set their download to a reasonable percentage of our connection.

 

[CatBus]

Throw it on ethernet, lock it at 10/100? Not elegant, but sometimes the hammer just works.

 

[evan_s]

You can check your settings on your phones and might be able to disable the randomly changing MAC address behavior. On iOS it looks like you can change Private Wi-FI Address to Fixed or Off instead of rotating to turn off this behavior. Not sure if setting as a priority would help but that should at least allow you to try.

I assume what was happening was probably actually the upload being saturated and that was causing things to fall apart completely as you can't even acknowledge packets received for downloads which causes throttling and other issues on the download. I assume that's cable internet? Maybe you can upgrade to a better package just to get higher upload speed or maybe just a newer modem to support better upload speed?

Can you set a hard cap on your upload speed? Usually setting it to a little below your actual speed helps to eliminate this sort of thing. It's not actually the saturation that's the problem but massive buffers that add major latency on the saturated upload connection. 9mbs upload setting would probably work.

If Deco just doesn't have any options that work and you can't/don't want to increase your internet plan for better upload speed you might just need a different router/mesh setup.

Personally, I don't feel the need for more than 300 down but I couldn't handle 10mbs up. I work from home and 10mbs upload would be a complete pain for all the situations where I end up uploading things to work.

 

[crombie]

Unfortunately there is no option for more than 10Mbps until they get fibre to our area. BUT I figured out why sqm was refusing to install, it is because the OpenWrt laptop was setup in AP mode. I reset it to router, and now all the shaping is in place. I limited the max connection to 150Mbps and 8Mbps which in theory should resolve the issues. The laptop can choke itself for all that I care, I just want the other machines on the network to have the bandwidth too!

 

[Paladin]

You're talking about a normal windows or other OS update taking all 300 megabit download, and/or possibly using up the 10 megabit upload in 'acknowledgement' traffic?

If so, you could set the ethernet/network adapter on the machine to limit its autonegotiation to only 10/100 instead of 10/100/1000 and your downloads will slow down a good bit, which should also free up the upload a bit. If you ever need to download something extra fast later, just put it back to full auto.

Sounds like it might be a device on wifi though, if that is the case, you can have it limited to 2.4 ghz if you want, or like you seem to have done, change the AP to a router mode and use a fair queuing control that should help manage things a bit more automatically, though it sounds like you might now have double routers which can cause double NAT. That sometimes causes issues of its own.

 

[crombie]

You're talking about a normal windows or other OS update taking all 300 megabit download, and/or possibly using up the 10 megabit upload in 'acknowledgement' traffic?

Honestly, it is unknown, if it happens again me and my other team will be sending a ticket in because not everyone has tech knowledge to try to setup this solution. I know Crowdstrike kept pinning out on the CPU, but it was mainly a Windows component that was taking the bulk if the Network in Task Manager.

If so, you could set the ethernet/network adapter on the machine to limit its autonegotiation to only 10/100 instead of 10/100/1000 and your downloads will slow down a good bit, which should also free up the upload a bit. If you ever need to download something extra fast later, just put it back to full auto.

Oooh, that is also a great idea! I will see if I have access to that on the machine. And if not I will have a ticket to have that updated.

Sounds like it might be a device on wifi though, if that is the case, you can have it limited to 2.4 ghz if you want, or like you seem to have done, change the AP to a router mode and use a fair queuing control that should help manage things a bit more automatically, though it sounds like you might now have double routers which can cause double NAT. That sometimes causes issues of its own.

The affected devices were a mix of hard-wired and wifi, the device causing the issues was direct wired to the Deco X60 we have. It is no longer, it is on the 2.4GHz wifi I setup on OpenWrt for the time being. As far as I can tell it isn't double-NAT. The path should be:

Internet -> Deco X60 (DHCP) -> Everything Else -> NB200 (OpenWrt) on one route, and then NB200 -> Wireless on another. I have not seen any of the other devices dropping off of the Deco getting an IP in that range.

 

[gusgizmo]

I would really recommend figuring out what it was exactly that was hammering your link. For example, delivery optimization in which case you could likely block it from uploading altogether.

Breaking the work stuff off into it's own vlan/network is often a good move, then it can easily have it's own policy without having to lean on mac addresses/ip reservations.

What I wouldn't want is to put policies on then teams/zoom start performing poorly.

10mbps is so gross in 2025. Sorry. Cable users are the new dial-up users "everything works fine it's just a little slower" meanwhile fiber users can't soak their pipe if they try.

 

[Andrewcw]

Er so if you can do an ethernet solution. You can get a managed switch to limit bandwidth. I for um reasons. With my Playstation limited it to 300kbps to make a game kick me to my own public yet private server because each console was a host possibility.

 

[wxfisch]

At a guess it is an update component that is misconfigured by whoever runs your client updates. There are some SCCM add-ons that will do distributed updates (i.e. you send the update from the DP to a client and then that client can send it to other clients to lessen the load on the P2P link between offices). Other systems have similar features (BigFix is another common name in the space, Ivanti I think also has this capability). If not configured right to respect VPNs/teleworkers the feature can very quickly eat up bandwidth trying to send multiple gigs of updates/software installers out of a small residential network connection.

I would submit a ticket even if you are able to fix it, as someone that has worked in that space before it is likely that team has no idea they are breaking your connection unless you say something since it will look just fine on their end.

 

[crombie]

Yeah, I am trying not to spend money or too much money on a solution. What is funny is I literally just ditched a box of routers when we moved, and one even had OpenWrt installed. It would have been the perfect solution for this issue with no fiddling. I figured with moving my Plex to Mint that this OpenWrt on the Atom would be cake to setup. It has been anything but, and it is even looking like QoS might not work properly.

NOW that said, over the 2.4GHz connection that is being limited to 40Mbps/s download which my basic networking understanding means it could still be saturated with ACK, but from a raw bandwidth perspective it is a small portion of the total.

Honestly, if this doesn't work out, and if it happens again, I will just turn off wifi and/or pop the Ethernet out of the computer and submit a ticket. If this were my personal machine you 100% know I would hunt the service or app down to the ends of the Earth. For a work computer, that is really a 'them' issue to figure it out.

 

[crombie]

you send the update from the DP to a client and then that client can send it to other clients to lessen the load on the P2P link between offices

You know, this actually seems to align with what was happening. It was like P2P was saturating our connection. Come to think of it they did recently engage with a service that is supposed to automate app updates.

 

[crombie]

Well, would you look at that, last night just past my work hours our connection started to go down again. Pulled the ethernet from the OpenWRT device and a few moments later the network was back to normal.

I researched the app they started using for updates, and it does have a P2P component. I am curious if they just recently decided to turn that on, and that would likely mean it is my upload getting saturated.

Ticket submitted, but my fix is to pull the ethernet from the OpenWrt device if it happens again until or if I manage to get QoS working on that device which I am not holding out much hope that it can be done.

 

[crombie]

Sadly no dice on the ticket, but I got qos scripts working on the OpenWrt device!

 

[Randomizer]

If my work laptop started doing that because IT is really that incompetent, I'd force them to pay for another internet connection. There is no technical reason you can't have a 2nd cable modem at your house billing to another account. Deploying a P2P-based app update platform with no capabilities to throttle traffic is not acceptable at all.

That said, 10Mb upload? In 2026? WTF! Even Starlink gives MUCH more than that very reliably. Heck, most fixed cellular provides give more bandwidth if you have a half-decent signal.

 

[crombie]

I mean, we have two real providers here, Bell (fibre) and Eastlink (cable). All resellers are through Eastlink. And that is limited to the 10Mb upload (except Purple Cow which is doing a very limited fibre rollout).

I was with Bell for years, but could no longer justify paying over $200 a month for Internet even with work paying half of that. Currently work pays for 100% of the internet connection, and if the issue has happened again I have not noticed it since that machine is throttled.

And the side benefit is since it is on it's own subnet it no longer can discover any network shares that I turn on to move files between my home computers.