Any networking experts here?

[dedalos]

Small %  1 or 2 packets here and there until the line gets bz. Clean during bz times. It is acting as if something goes to sleep but what. Everything has been turned off unless the settings are ignored by the os or the nic

[eagl]

Well, it should be off.  Is it possible the Windows is ignoring the setting somehow? 

Your router might have it switched on.  I've also seen competing driver settings be a problem, if you use the basic windows networking tools but also have adaptor specific drivers loaded even if you're not using the adaptor specific management utility.  So you might have QoS turned off in the windows settings but it might be on using the driver toolkit that came with the adaptor.  Or maybe the router has QoS or other traffic shaping turned on.

I have also had problems with jumbo packets, mostly with strange behavior and dropped or malformed packets.

[eagl]

Eliminate any extra hardware, swap to the shortest network cables you can find, etc.  I got a couple of bad cables with some powerline adaptors I bought recently, and the cables can be jiggled out of a good connection with just a slight tug on the cable.  So I threw out those cables and replaced them with better ones that I knew worked fine.

If you have a switch between the computer and the router, try to connect the computer directly to the switch.  Change ports on the switch or router.  All that sort of stuff.  Get rid of fluorescent lightbulbs or other noisy appliances on the same electrical circuit as the computer, switch, router, cable modem, etc.  Even TVs in on, sleep, standby, and off modes can cause huge interference on the power circuits.

[dedalos]

thank you

[Bizman]

Speaking about cables I once stumbled upon a problem where the old computer would work fine with the modem/router, but the new computer wouldn't connect at all. After changing the network cable even the new one could find the net. After some thinking and searching I noticed that the old cable was cat.5 and the new one was cat.6. Apparently the old computer had a built-in 100 Mbps nic and the new a 1000 Mbps one. The modem/router automatically adapted to handle both speeds, but the cat.5 cable wasn't capable for the faster speed. There's also both shielded and unshielded versions of all of the speeds, a shielded cat.6 might be worth trying.

[Vulcan]

Errr what exactly are you using, 1Gbps or 10Gbe?

What kind of switch and what kind of router.

Who setup the network? It's likely to be something around the multicast membership detection and forwarding. Multicast can be a salamander of a thing to troubleshoot depending on what gear you have.

[Trell]

The packet loss every once in a while is what gets me.  Every multicast stream application I have  dealt with has the ability to deal with some missing packets. multicast is not a 100% transmission guarantee t.  It does not deal with error correction, that has to be dealt with within the application.  It has to expect this to happen.  That is just the way it is on a tcp/ip network .   I can see why the network team is not seeing packet loss.  You are not losing large amounts of packets.  It sounds like it is just a standard error rate for collision.

[Vulcan]

That is just the way it is on a tcp/ip network .

Ah huh.

[BoilerDown]

Impossible to make even an educated guess without knowing your hardware and network topology and more specifics about what multicast packets are going missing.  If you actually have a 10Gig link, then you have some higher end gear, and specifics are needed to talk knowledgeably.

About other people's suggestions so far:

Its not a layer 1 issue (network cables, EMF, etc). If that's was the problem, you'd see it in other packets, especially when utilization was high, which you said works fine.
QoS isn't the problem.  During low utilization a QoS scheme's "best effort" will be more than enough.  QoS isn't misconfigured in a way that could cause this to be not true because high utilization periods work fine.
Unless you have a hub on your network, which you don't, its not a collision problem.  Every device will be in its own collision domain.

You haven't said what negative effects this small packet loss causes you.  If nothing, then forget about it.  Network technologies are designed to work, not have zero packet loss.  Having some packet loss is not a problem unless there's things that are actually not working because of it.

[dedalos]

Hey Boiler,

We are a trading firm so you can bet the equipment in high end.  All the servers are in the data center so it is a local network.  The reason I want to stay off the setup discussion is that we have several network engineers look at the problem and also the hardware manufacturer but no one can make it go away.  So I am looking for the crazy out of the box idea or someone that has seen it before.  My guess is that some setting is being ignored by the OS or hardware.

As for it being a problem it really isn't because during the day market updates come in in the 1000s per second.  However, if it is a thinly traded stock and we happen to drop a packet containing an update for it, then it would become an issue.  The real problem is though that given our set up and amount of data we receive we should not have a single drop ever.  That is what is bothering me the most and the fact that I don't understand why they drop.  I've been doing this for over 15 years now and I have never seen anything like it.

[BoilerDown]

Hey Boiler,

We are a trading firm so you can bet the equipment in high end.  All the servers are in the data center so it is a local network.  The reason I want to stay off the setup discussion is that we have several network engineers look at the problem and also the hardware manufacturer but no one can make it go away.  So I am looking for the crazy out of the box idea or someone that has seen it before.  My guess is that some setting is being ignored by the OS or hardware.

As for it being a problem it really isn't because during the day market updates come in in the 1000s per second.  However, if it is a thinly traded stock and we happen to drop a packet containing an update for it, then it would become an issue.  The real problem is though that given our set up and amount of data we receive we should not have a single drop ever.  That is what is bothering me the most and the fact that I don't understand why they drop.  I've been doing this for over 15 years now and I have never seen anything like it.

I don't know what's standard in trading markets, but my first reaction is:

And you're doing multicast IP packets for this?  Multicast IP uses UDP which has no guarantee of delivery.  If you need 100% delivery you need to use TCP and unicast it everywhere it needs to go.  Sometimes it is proper and expected to drop a packet, it can and is done at times by design.  Without more knowledge of your setup, its pointless to speculate what problem you're running in to.  But most likely A) you're making something out of nothing, there's no chance one of these packet drops will ever cause a problem, or B) If that's not true, then you're using multicast for something that should never be done with multicast.

[Bizman]

Just yesterday a customer called and said he had had a lengthy conversation with his ISP because of hiccups in his connection. We already had done all the tricks in his computer, like disabling all energy saving functions in networking.

The trick they suggested was even suggested on their website, so it's not something uncommon. Why I post it here, is that the cable modem is made by Cisco who are a better known brand in professional equipment. Household gear may be smaller and cheaper, but they have to provide equal protection against networms and such as their professional versions. The function in question was ip-flood-detection, disabling of which solved my client's problem. Wouldn't be the first time some safety feature causes trouble...

[dedalos]

I don't know what's standard in trading markets, but my first reaction is:

And you're doing multicast IP packets for this?  Multicast IP uses UDP which has no guarantee of delivery.  If you need 100% delivery you need to use TCP and unicast it everywhere it needs to go.  Sometimes it is proper and expected to drop a packet, it can and is done at times by design.  Without more knowledge of your setup, its pointless to speculate what problem you're running in to.  But most likely A) you're making something out of nothing, there's no chance one of these packet drops will ever cause a problem, or B) If that's not true, then you're using multicast for something that should never be done with multicast.

Boil,

This is exactly why I want to avoid the technical discussion.  It will turn into an argument.  You have to trust me on this one that we have tried all the "standard" trouble shouting and we have engineers that are dealing with these setups all the time looking into it.  We can't find anything that is why I posted here.  I m hoping someone has seen it before or some out of the box ideas. Multicast can and will drop data but only if needed or if there is a problem.  There is no way we should be dropping a single packet under the conditions those packets are dropped. We do have ways of recovering them but now we are getting into a discussion that is not part of the original question.  As crazy as Bizmans ideas sound given what we do and our set up, that is what I am looking for.

As for why we use multicast google OPRA market data.  You will find it interesting but again, the why we use it or should we it should not be part of this discussion  or we will end up with a lot of pages on the wrong subject.

Thank you

[Bizman]

As crazy as Bizmans ideas sound given what we do and our set up, that is what I am looking for.

Thank you for your kind words, sir. As I said in my first input, I'm by no means a networking expert. 

[dedalos]

Thank you for your kind words, sir. As I said in my first input, I'm by no means a networking expert. 

lol, I did not mean it in a bad way. I need ideas that that are out of the box since we looked and still looking inside the box. It was actually meant as a compliment.