Aces High Bulletin Board
General Forums => The O' Club => Topic started by: dedalos on July 17, 2013, 08:51:45 AM
-
Does anyone know why I would be seeing packet loss on a multicast stream only when the utilization is low? For example, I have a 10Gig connection that pulls in market data. At 1% utilization I see packet loss every few seconds. At 3% and above I see none. As long as the network utilization stays above 3% or 4% I don;t see a single drop. Needless to say we have tried every possible setting we and the nic maker could think of with no change on the results. Any ideas? I'd rather not work of the machine specs since I am looking for ideas on what could be causing something like that in general but if you need the specs I ll post them.
Thank you
-
I'm no networking expert, but one thing that comes into mind is that Windows might be shutting your network card down if there's no traffic. Checking the Power Management settings for your nic and unticking the "Let windows shut this down" would be the first step, which you probably already have done with the nic maker. Also, if you have tinkered with Windows Services, could it be possible that you've set some network related service from automatic to manual?
-
I am not sure how much of the network is yours. Are you seeing this on the end server or are you seeing the packet loss on the router connected to your providers wan?
When I used to do more technical work I dealt more on the VoIP side of things. But normally I would put a sniffer as close to the entry point this data is coming into your network or as close to the origination server of this data to make sure it is not an issue father up stream.
-
Hey Biz, yes all that stuff has been checked.
Trell, unfortunately the network is managed by a different company. They have put sniffers and checked the equipment and they claim to see nothing wrong. Everyone thinks it is at the server (receiving end) but I am open to anything really. I am hoping that someone has seen this before and figured it out in the past.
I am looking for the crazy out of the box idea here since we have checked everything we could think of with several people. I think the key is that it only drops when there is low utilization on the line. This is market data so utilization is never at 0%. As long as I can stay above 3 or 4% utilization I don;t see any drops. The higher the better lol.
-
Is there only one or more computers in your end? If only one, how about another network card of another brand just for testing purposes?
-
If you have any control of your own switches you could set up a mirror port and set up a laptop to run a sniffer on just to be sure. Depending on traffic through put you could put a hub in front of the box to do the same thing.
I would second changing out the nic card was well. It is a simple thing to do as a test.. normally when i see packet loss it is on the router or provider side, even if they say it is fine.... Not that i don't trust managed networks but I don't trust Managed networks.
-
Is there only one or more computers in your end? If only one, how about another network card of another brand just for testing purposes?
three servers (same specs) listening on 12 mc channels each. They all show the same behavior at different times.
-
If you have any control of your own switches you could set up a mirror port and set up a laptop to run a sniffer on just to be sure. Depending on traffic through put you could put a hub in front of the box to do the same thing.
I would second changing out the nic card was well. It is a simple thing to do as a test.. normally when i see packet loss it is on the router or provider side, even if they say it is fine.... Not that i don't trust managed networks but I don't trust Managed networks.
Heh, no control what so ever over the network other than calling the network people.
-
Is QoS buffering packets in the hopes that something better will come along?
-
Dedalos, look on Facebook for "Cameraonastick". He's a former friend of mine and is a god in puters and networking. If he can't help you(and it will cost, it's how he makes his living) I truly don't believe anyone can fix it.
-
Is QoS buffering packets in the hopes that something better will come along?
Well, it should be off. Is it possible the Windows is ignoring the setting somehow?
-
Think physical... Perhaps there's a Electromagnetic Field or Radio Frequencies near your equipment that only induces the "noise" when the line is inactive. When utilization begins to ramp up, the minor EMI, RFI fields are negated and you do not see packet loss....
Turn off any HVAC & non-essential Electrical systems and see if you notice a difference.....Just a theory from my experience... :x Had an industrial Air Conditioning unit throw a wicked EMI field around some local wireless networks and found it to be intermittent as hell until we realized the physical proximity of the motors and our access point.....
Hope that helps and not hinders!
-
Here is a question i should have asked before. How are you determining that you are getting packet loss? Is it from the application that is receiving the data or is it a different 3rd party monitoring system that is showing it? I am also wondering what kind of % packet loss you are recieving
-
Sequence numbers from the app, netstat, wireshark
-
Think physical... Perhaps there's a Electromagnetic Field or Radio Frequencies near your equipment that only induces the "noise" when the line is inactive. When utilization begins to ramp up, the minor EMI, RFI fields are negated and you do not see packet loss....
Turn off any HVAC & non-essential Electrical systems and see if you notice a difference.....Just a theory from my experience... :x Had an industrial Air Conditioning unit throw a wicked EMI field around some local wireless networks and found it to be intermittent as hell until we realized the physical proximity of the motors and our access point.....
Hope that helps and not hinders!
lol. Kind of what I m looking for since we have tried the conventional aproaches.
-
Small % 1 or 2 packets here and there until the line gets bz. Clean during bz times. It is acting as if something goes to sleep but what. Everything has been turned off unless the settings are ignored by the os or the nic
-
Well, it should be off. Is it possible the Windows is ignoring the setting somehow?
Your router might have it switched on. I've also seen competing driver settings be a problem, if you use the basic windows networking tools but also have adaptor specific drivers loaded even if you're not using the adaptor specific management utility. So you might have QoS turned off in the windows settings but it might be on using the driver toolkit that came with the adaptor. Or maybe the router has QoS or other traffic shaping turned on.
I have also had problems with jumbo packets, mostly with strange behavior and dropped or malformed packets.
-
Eliminate any extra hardware, swap to the shortest network cables you can find, etc. I got a couple of bad cables with some powerline adaptors I bought recently, and the cables can be jiggled out of a good connection with just a slight tug on the cable. So I threw out those cables and replaced them with better ones that I knew worked fine.
If you have a switch between the computer and the router, try to connect the computer directly to the switch. Change ports on the switch or router. All that sort of stuff. Get rid of fluorescent lightbulbs or other noisy appliances on the same electrical circuit as the computer, switch, router, cable modem, etc. Even TVs in on, sleep, standby, and off modes can cause huge interference on the power circuits.
-
thank you
-
Speaking about cables I once stumbled upon a problem where the old computer would work fine with the modem/router, but the new computer wouldn't connect at all. After changing the network cable even the new one could find the net. After some thinking and searching I noticed that the old cable was cat.5 and the new one was cat.6. Apparently the old computer had a built-in 100 Mbps nic and the new a 1000 Mbps one. The modem/router automatically adapted to handle both speeds, but the cat.5 cable wasn't capable for the faster speed. There's also both shielded and unshielded versions of all of the speeds, a shielded cat.6 might be worth trying.
-
Errr what exactly are you using, 1Gbps or 10Gbe?
What kind of switch and what kind of router.
Who setup the network? It's likely to be something around the multicast membership detection and forwarding. Multicast can be a salamander of a thing to troubleshoot depending on what gear you have.
-
The packet loss every once in a while is what gets me. Every multicast stream application I have dealt with has the ability to deal with some missing packets. multicast is not a 100% transmission guarantee t. It does not deal with error correction, that has to be dealt with within the application. It has to expect this to happen. That is just the way it is on a tcp/ip network . I can see why the network team is not seeing packet loss. You are not losing large amounts of packets. It sounds like it is just a standard error rate for collision.
-
That is just the way it is on a tcp/ip network .
Ah huh.
-
Impossible to make even an educated guess without knowing your hardware and network topology and more specifics about what multicast packets are going missing. If you actually have a 10Gig link, then you have some higher end gear, and specifics are needed to talk knowledgeably.
About other people's suggestions so far:
Its not a layer 1 issue (network cables, EMF, etc). If that's was the problem, you'd see it in other packets, especially when utilization was high, which you said works fine.
QoS isn't the problem. During low utilization a QoS scheme's "best effort" will be more than enough. QoS isn't misconfigured in a way that could cause this to be not true because high utilization periods work fine.
Unless you have a hub on your network, which you don't, its not a collision problem. Every device will be in its own collision domain.
You haven't said what negative effects this small packet loss causes you. If nothing, then forget about it. Network technologies are designed to work, not have zero packet loss. Having some packet loss is not a problem unless there's things that are actually not working because of it.
-
Hey Boiler,
We are a trading firm so you can bet the equipment in high end. All the servers are in the data center so it is a local network. The reason I want to stay off the setup discussion is that we have several network engineers look at the problem and also the hardware manufacturer but no one can make it go away. So I am looking for the crazy out of the box idea or someone that has seen it before. My guess is that some setting is being ignored by the OS or hardware.
As for it being a problem it really isn't because during the day market updates come in in the 1000s per second. However, if it is a thinly traded stock and we happen to drop a packet containing an update for it, then it would become an issue. The real problem is though that given our set up and amount of data we receive we should not have a single drop ever. That is what is bothering me the most and the fact that I don't understand why they drop. I've been doing this for over 15 years now and I have never seen anything like it.
-
Hey Boiler,
We are a trading firm so you can bet the equipment in high end. All the servers are in the data center so it is a local network. The reason I want to stay off the setup discussion is that we have several network engineers look at the problem and also the hardware manufacturer but no one can make it go away. So I am looking for the crazy out of the box idea or someone that has seen it before. My guess is that some setting is being ignored by the OS or hardware.
As for it being a problem it really isn't because during the day market updates come in in the 1000s per second. However, if it is a thinly traded stock and we happen to drop a packet containing an update for it, then it would become an issue. The real problem is though that given our set up and amount of data we receive we should not have a single drop ever. That is what is bothering me the most and the fact that I don't understand why they drop. I've been doing this for over 15 years now and I have never seen anything like it.
I don't know what's standard in trading markets, but my first reaction is:
And you're doing multicast IP packets for this? Multicast IP uses UDP which has no guarantee of delivery. If you need 100% delivery you need to use TCP and unicast it everywhere it needs to go. Sometimes it is proper and expected to drop a packet, it can and is done at times by design (http://www.cathayschool.com/controlling-multicast-scope-with-ttl-a201.html). Without more knowledge of your setup, its pointless to speculate what problem you're running in to. But most likely A) you're making something out of nothing, there's no chance one of these packet drops will ever cause a problem, or B) If that's not true, then you're using multicast for something that should never be done with multicast.
-
Just yesterday a customer called and said he had had a lengthy conversation with his ISP because of hiccups in his connection. We already had done all the tricks in his computer, like disabling all energy saving functions in networking.
The trick they suggested was even suggested on their website, so it's not something uncommon. Why I post it here, is that the cable modem is made by Cisco who are a better known brand in professional equipment. Household gear may be smaller and cheaper, but they have to provide equal protection against networms and such as their professional versions. The function in question was ip-flood-detection, disabling of which solved my client's problem. Wouldn't be the first time some safety feature causes trouble...
-
I don't know what's standard in trading markets, but my first reaction is:
And you're doing multicast IP packets for this? Multicast IP uses UDP which has no guarantee of delivery. If you need 100% delivery you need to use TCP and unicast it everywhere it needs to go. Sometimes it is proper and expected to drop a packet, it can and is done at times by design (http://www.cathayschool.com/controlling-multicast-scope-with-ttl-a201.html). Without more knowledge of your setup, its pointless to speculate what problem you're running in to. But most likely A) you're making something out of nothing, there's no chance one of these packet drops will ever cause a problem, or B) If that's not true, then you're using multicast for something that should never be done with multicast.
Boil,
This is exactly why I want to avoid the technical discussion. It will turn into an argument. You have to trust me on this one that we have tried all the "standard" trouble shouting and we have engineers that are dealing with these setups all the time looking into it. We can't find anything that is why I posted here. I m hoping someone has seen it before or some out of the box ideas. Multicast can and will drop data but only if needed or if there is a problem. There is no way we should be dropping a single packet under the conditions those packets are dropped. We do have ways of recovering them but now we are getting into a discussion that is not part of the original question. As crazy as Bizmans ideas sound given what we do and our set up, that is what I am looking for.
As for why we use multicast google OPRA market data. You will find it interesting but again, the why we use it or should we it should not be part of this discussion or we will end up with a lot of pages on the wrong subject.
Thank you
-
As crazy as Bizmans ideas sound given what we do and our set up, that is what I am looking for.
Thank you for your kind words, sir. As I said in my first input, I'm by no means a networking expert. :salute
-
Thank you for your kind words, sir. As I said in my first input, I'm by no means a networking expert. :salute
lol, I did not mean it in a bad way. I need ideas that that are out of the box since we looked and still looking inside the box. It was actually meant as a compliment.
-
lol, I did not mean it in a bad way. I need ideas that that are out of the box since we looked and still looking inside the box. It was actually meant as a compliment.
I took it as a compliment in the first place. I've learned through this thread that you've got more than enough of expert level advice concerning your problem, that's why I dared offer my less stellar ideas. Hope you get it sorted in one way or another.
Besides, your words really were kind and I like being thanked as much as anyone else.
-
I took it as a compliment in the first place. I've learned through this thread that you've got more than enough of expert level advice concerning your problem, that's why I dared offer my less stellar ideas. Hope you get it sorted in one way or another.
Besides, your words really were kind and I like being thanked as much as anyone else.
lol. Good. Most ideas will be wrong but one of them may turn on the light. You never know.
-
Why I post it here, is that the cable modem is made by Cisco who are a better known brand in professional equipment. Household gear may be smaller and cheaper, but they have to provide equal protection against networms and such as their professional versions. The function in question was ip-flood-detection, disabling of which solved my client's problem. Wouldn't be the first time some safety feature causes trouble...
FWIW Cisco cable modems are usually just linksys gear rebadged, and Cisco security products are awful (yes lots of people buy ASA's, but lots of people buy Big Mac's too).
dedalo's the easiest thing to do would be to put a box in between the switch and the server/device missing packets, and drag through packet by packet. But BoilerDown hit the nail on the head, UDP is a stateless protocol, combined with the nuances of multicast whoever set the system up needs a kick up the backside. I would put money on it being related to membership and timeout requests (the switch needs to maintain and refresh the list of devices partaking in the multicast session). There are going to be tiny periods when 'administrative' tables are updated, and that could lose you packets.
If the data is so important it should have been on tcp or the app should have it's own embedded error checking for the UDP data.
-
dedalo's the easiest thing to do would be to put a box in between the switch and the server/device missing packets, and drag through packet by packet.
A little more info on where you are going with this please. Lets say I did this and we still drop, what would it point at? The problem is all the hardware we own is exactly the same. Lets say I don;t see any drops, what do I check yet?
But BoilerDown hit the nail on the head, UDP is a stateless protocol, combined with the nuances of multicast whoever set the system up needs a kick up the backside.
lol, yes we are familiar with networking 101 but I will put a call in to the exchanges to explain that they need to use TCP instead of multicast.
I would put money on it being related to membership and timeout requests (the switch needs to maintain and refresh the list of devices partaking in the multicast session). There are going to be tiny periods when 'administrative' tables are updated, and that could lose you packets.
Again, a little more info please. In 15 years I have not witnessed this behavior. The server will only drop when the 10Gig line is below 2 or 3% utilization but packets are coming in in the thousands per second even at those levels. Why would the memberships time out and how do I tell if that is what is happening?
If the data is so important it should have been on tcp or the app should have it's own embedded error checking for the UDP data.
It does, but it does not explain why the packets drop, does it?
-
If this was my network, I would be debugging the routers at the various interfaces. I would also have a sniffer set between devices, changing it once I knew that the traffic was making it to that next router or switch.
I would doubt this is on your computer(s), and since your network is managed by somebody else, you may need your team to get on them about debugging their routers and switches, however the topology is designed.
Do you even have access to the last router on your side, or is that still managed by another company?
-
If this was my network, I would be debugging the routers at the various interfaces. I would also have a sniffer set between devices, changing it once I knew that the traffic was making it to that next router or switch.
I would doubt this is on your computer(s), and since your network is managed by somebody else, you may need your team to get on them about debugging their routers and switches, however the topology is designed.
Do you even have access to the last router on your side, or is that still managed by another company?
If we are talking about physical access, no unless I go to the data center. I could if I needed to but I would need to know what I am looking for. Can anything of what you mentioned/thinking be checked remotely from my servers? There are three parties involved in this. Me, the network host and the data provider. I am willing to take their word that they have checked their end and don;t see the drops. Do I trust them 100%? No, but I can only push so much without any proof and they are risking legal action if they are lying. I don;t have access to the data providers equipment. My hope is that it is something on my end because I can fix that once I figure out what it is. If the drops happened during high utilization it would be easy to come up with ideas. But it doesn't. Do you know of any settings on a switch that could cause this? Anything would be helpful no matter how crazy it sounds. I can get the equipment specs on Monday if you think they could help.
-
You wouldn't need physical access. Sorry, I should've been more specific. I just meant access to the configuration of the routers.
There could be a number of reasons why it's happening, but on each router, I'd look at the error logs for the port and see what's there. It would give you an idea of where to start. If you're seeing lost packets, or other signs of network issues on the physical device, you can troubleshoot from there. If the logs are clean, and you're not seeing issues, move on to the next router or switch.
There are various possibilities on layer 2 and layer 3. Just an example from a Cisco site: http://www.cisco.com/en/US/tech/tk828/technologies_tech_note09186a0080094b55.shtml
You can install a free packet sniffer on your servers as well, such as Wireshark, and analyze those logs. However, that's where physical access might come in handy by plugging a sniffer (or laptop with a packet sniffer) into the same physical device to ensure you're capturing all of the traffic.
I would ask the network host to do some debugging. I don't believe it would be on your data provider's side, at least not at first. It could be something regarding MTU sizes. An example from Juniper: http://kb.juniper.net/InfoCenter/index?page=content&id=KB25312
So you know, I am out of 'hands on' for several years now, and I have a staff that would handle these problems, but in managing it forward, I'm just relaying how I would do it on my network, and who I would have my staff contact. I'd absolutely want to debug the interfaces and go from there.
-
You wouldn't need physical access. Sorry, I should've been more specific. I just meant access to the configuration of the routers.
There could be a number of reasons why it's happening, but on each router, I'd look at the error logs for the port and see what's there. It would give you an idea of where to start. If you're seeing lost packets, or other signs of network issues on the physical device, you can troubleshoot from there. If the logs are clean, and you're not seeing issues, move on to the next router or switch.
There are various possibilities on layer 2 and layer 3. Just an example from a Cisco site: http://www.cisco.com/en/US/tech/tk828/technologies_tech_note09186a0080094b55.shtml
You can install a free packet sniffer on your servers as well, such as Wireshark, and analyze those logs. However, that's where physical access might come in handy by plugging a sniffer (or laptop with a packet sniffer) into the same physical device to ensure you're capturing all of the traffic.
I would ask the network host to do some debugging. I don't believe it would be on your data provider's side, at least not at first. It could be something regarding MTU sizes. An example from Juniper: http://kb.juniper.net/InfoCenter/index?page=content&id=KB25312
So you know, I am out of 'hands on' for several years now, and I have a staff that would handle these problems, but in managing it forward, I'm just relaying how I would do it on my network, and who I would have my staff contact. I'd absolutely want to debug the interfaces and go from there.
Cool, thank you. I have remote access to the servers and they all have wireshark installed. I have to find a way to use it without impacting production since the amount of data coming in is ridiculous lol. I would have to ask the admins for the logs since I don't have access to the network. However, they do claim they are clean. Time to look for my self I guess.
Thank you
-
FWIW Cisco cable modems are usually just linksys gear rebadged, and Cisco security products are awful
I've read about Cisco owning Linksys, my point was merely a guess that security related solutions might be similar in all size systems.
@dedalos: Yet another crazy idea: Is it possible that your network host has got new gear, which might be slightly incompatible with yours? I used to suffer from Internet problems twice, when my ISP refurnished. The first time they had to update the firmware of their new gadgets, the second time I thought getting a new modem would be the most viable solution.
-
Cool, thank you. I have remote access to the servers and they all have wireshark installed. I have to find a way to use it without impacting production since the amount of data coming in is ridiculous lol. I would have to ask the admins for the logs since I don't have access to the network. However, they do claim they are clean. Time to look for my self I guess.
Thank you
Not a problem at all. It would take a few minutes for you or the network admins to log into each device and get a status on the interfaces that are used. Can you sit next to the network admins while they troubleshoot?
-
Not a problem at all. It would take a few minutes for you or the network admins to log into each device and get a status on the interfaces that are used. Can you sit next to the network admins while they troubleshoot?
The equipment is in Chicago, the admins in NY (although they have staff that can go into the data center if they need to) and I am in Texas lol
-
The equipment is in Chicago, the admins in NY (although they have staff that can go into the data center if they need to) and I am in Texas lol
That is funny. :)
Perhaps ask if you can view their desktop while they debug, so you can ask questions in real time.
-
A little more info on where you are going with this please. Lets say I did this and we still drop, what would it point at? The problem is all the hardware we own is exactly the same. Lets say I don;t see any drops, what do I check yet?
lol, yes we are familiar with networking 101 but I will put a call in to the exchanges to explain that they need to use TCP instead of multicast.
Again, a little more info please. In 15 years I have not witnessed this behavior. The server will only drop when the 10Gig line is below 2 or 3% utilization but packets are coming in in the thousands per second even at those levels. Why would the memberships time out and how do I tell if that is what is happening?
It does, but it does not explain why the packets drop, does it?
Well, you have unicast broadcast and multicast right. Unitcast is point to point right where the send specifies the address of the destination. Broadcast goes to everyone. For multicast you have to tell the switch you're connected too that you want too partake in a multicast session, that switch in turn may need to tell upstream devices. IIRC the multicast table refreshes every 60 - 120 seconds. Hell in low traffic scenarios where there is no activitiy it could even be something as simple or weird as an ARP timeout.
-
Well, you have unicast broadcast and multicast right. Unitcast is point to point right where the send specifies the address of the destination. Broadcast goes to everyone. For multicast you have to tell the switch you're connected too that you want too partake in a multicast session, that switch in turn may need to tell upstream devices. IIRC the multicast table refreshes every 60 - 120 seconds. Hell in low traffic scenarios where there is no activitiy it could even be something as simple or weird as an ARP timeout.
Its worth taking a look at although low traffic for me is probably high traffic for other systems but it may give the network guys some ideas. Thank you
-
Possibly a wild goose chase, but is there a chance that an upstream host is traffic shaping you to another pipe when your bandwidth in use drops below a set level?
-
Possibly a wild goose chase, but is there a chance that an upstream host is traffic shaping you to another pipe when your bandwidth in use drops below a set level?
Everything is possible I guess. It should be a direct connection from one data-center to the other but . . .
-
Everything is possible I guess. It should be a direct connection from one data-center to the other but . . .
Simple route tracing over time might work unless they're using virtual servers or routers or just throttling without changing IP addy.
Or, maybe when bandwidth drops some automated housekeeping starts which sucks cpu time resulting in dropped packets? Like the idiots who assume everyone goes to lunch at noon so they force start the corporate antivirus scans at noon bringing the entire IT infrastructure to its knees for an hour as every hard drive on the network starts thrashing?
-
Simple route tracing over time might work unless they're using virtual servers or routers or just throttling without changing IP addy.
Or, maybe when bandwidth drops some automated housekeeping starts which sucks cpu time resulting in dropped packets? Like the idiots who assume everyone goes to lunch at noon so they force start the corporate antivirus scans at noon bringing the entire IT infrastructure to its knees for an hour as every hard drive on the network starts thrashing?
To be honest, I think that is what is happening. Not a person but the servers or the nics are ignoring some setting or something else is telling the OS to turn them back on. This is the card I am using: Solarstorm SFN5162F SFP+ Server Adapter. When this thing gets bz it runs flawlessly so I know the hardware can take the load. Something is putting something to sleep, but what or how do I find it if when looking at the settings everything has been turned off?
-
Still strange.
My old laptop had pci-e bus power savings options, but the settings were visible only from the ATI video card drivers. There may be a data bus power savings mode kicking in. That could be pretty deep in the drivers. Might need to get some serious OS support going if you need custom drivers. What OS? Do the machines have a BIOS you can access, to try turning off all power savings settings?
-
Along eagl's lines - what OS are the servers running, and what make and model are the servers?
-
Windows Server 2008 R2 Standard SP1 dual Xeon 2.3Ghz
-
Think physical... Perhaps there's a Electromagnetic Field or Radio Frequencies near your equipment that only induces the "noise" when the line is inactive. When utilization begins to ramp up, the minor EMI, RFI fields are negated and you do not see packet loss....
Turn off any HVAC & non-essential Electrical systems and see if you notice a difference.....Just a theory from my experience... :x Had an industrial Air Conditioning unit throw a wicked EMI field around some local wireless networks and found it to be intermittent as hell until we realized the physical proximity of the motors and our access point.....
Hope that helps and not hinders!
One of our offices recently had a problem with comms between a computer and a check scanner that was ultimately traced to a nearby powered stapler. D'oh! <facepalm!>
(http://electricstapler.org/wp-content/uploads/2011/08/Electric-Stapler.jpg)
-
Dedalos, another angle could be: if you mix 1G and 10G Ethernet (say your servers have 1G interfaces, and the distribution switches use 10GE) this can produce packet loss by even the most modest spikes in traffic.
(There is a long winded explanation of how this can happen, it needs a speed mismatch between some of the interfaces in your traffic flow - if that's not your setup disregard this post - if it is i can explain it :))
-
Dedalos, another angle could be: if you mix 1G and 10G Ethernet (say your servers have 1G interfaces, and the distribution switches use 10GE) this can produce packet loss by even the most modest spikes in traffic.
(There is a long winded explanation of how this can happen, it needs a speed mismatch between some of the interfaces in your traffic flow - if that's not your setup disregard this post - if it is i can explain it :))
its the opposite remember? High load = no problems.
-
ah, my fault, only read half of your post apparently :bhead
-
Lol. Although, there is a 1gig interface also. 10gig input and 1 gig out put on different nics.
-
Disable any kind of jumbo frames :) Maybe some nics are combining packets when they have the processing time overhead and there is a compatibility issue somewhere.
Best way on my home network to induce instability is to enable jumbo frames on everything.
-
Disable any kind of jumbo frames :) Maybe some nics are combining packets when they have the processing time overhead and there is a compatibility issue somewhere.
Best way on my home network to induce instability is to enable jumbo frames on everything.
no jumbo frames used or allowed, ever lol.
-
I found this http://support.microsoft.com/kb/2639824 (http://support.microsoft.com/kb/2639824) I will install it and see what happens. Not holding my breath though lol.
-
Silly question. Have you opened up a ticket with Microsoft on this issue yet? their support is normally good. We have went to them on some weird problems before. The cost is next to nothing.
-
One of our offices recently had a problem with comms between a computer and a check scanner that was ultimately traced to a nearby powered stapler. D'oh! <facepalm!>
(http://electricstapler.org/wp-content/uploads/2011/08/Electric-Stapler.jpg)
I used to sell the Swingline electric stapler that was of the same type as the one used in some photocopiers, so you might to want to check them too...