Ask HN: My mom called me, but I got someone else. How can I report this bug?
34 comments
Sounds like frame slips on a TDM circuit. It could be a DS3 or DS1, doesn't really matter, the principle is the same. For example, on an ISDN PRI (T1):
2 pairs of copper wires using Time Division Multiplexing create 24 digital channels of 64Kbps: 23B channels (call bearer audio) + 1 D channel (call signalling).
This is very time sensitive, so if the clocks are incorrectly set (both in free run / not synchronized), this will happen all the time; in other cases defective equipment or noise on the line (noise is on the copper circuit, not the digital channel)(a technician connecting a test set or toner on a T1 is a common cause), it can cause all of the channels to shift one or more channels. So the audio on channel 1 is now connected to channel 2 at the opposite end, etc. This state will last for a short while until the D channel becomes angry as it is now receiving jibberish and disable the circuit with a yellow alarm as both sides think the remote is at fault. If more circuits are available, your second call should proceed fine and the technicians at both ends will have an alarm and will need to reset the circuit if there isn't an automatic mechanism to do so. So nothing to worry about unless it happens all of the time, but this will definitely generate log events.
This is very time sensitive, so if the clocks are incorrectly set (both in free run / not synchronized), this will happen all the time; in other cases defective equipment or noise on the line (noise is on the copper circuit, not the digital channel)(a technician connecting a test set or toner on a T1 is a common cause), it can cause all of the channels to shift one or more channels. So the audio on channel 1 is now connected to channel 2 at the opposite end, etc. This state will last for a short while until the D channel becomes angry as it is now receiving jibberish and disable the circuit with a yellow alarm as both sides think the remote is at fault. If more circuits are available, your second call should proceed fine and the technicians at both ends will have an alarm and will need to reset the circuit if there isn't an automatic mechanism to do so. So nothing to worry about unless it happens all of the time, but this will definitely generate log events.
No offense, but it sounds like you're just throwing around terms you don't really understand.
Frame slips won't cause an audio misconnect like this, because... well, lots of reasons. The channel layout (23B, 1D) you describe isn't TDM, it's PRI (primary rate interface). Slips occur at a lower layer; the T-carrier.
All T-carrier based telecommunications systems transmit at a fixed frequency. Even when the circuit is idle, a stream of bits are transmitted across the line, so that both ends remain in sync. A loss in synchronization is called a slip. That much you got correct.
These circuits are digital though. When a slip occurs, it's not as if whole frames jump from one to the next. What happens when you perform a binary shift on a chunk of data? The outcome is wildly unpredictable. The protocols at higher layers (like PRI) have mitigation strategies for dealing with slips. A re-sync occurs almost instantaneously. A T-carrier circuit that slips regularly will not be reliable. Based on my experience, persistent frame slips result in either a dropped call (B-channel reset), or garbled audio (due to shifted bits in the media [1]). If you've ever been on a call, and the audio suddenly becomes a garbled mess that sounds like a mix of modem communication and alien speech, that's what bit-shifted media sound slike.
I have never seen a misconnect caused by a slip. I suppose it's possible, but the likelihood would seem extremely remote. Imagine if you Base64 encoded something, performed a single binary shift, and the Base64 decoded result turned out to be the lyrics of the Star Spangled Banner. Unlikely.
To top it all off PRI is virtually non-existent within carriers. PRI is an end-user interface. You only see it in the last mile. Carriers use protocols like SS7, which you'll never see on end-user equipment. Even SS7 is less common these days, because the vast majority of telephone calls aren't TDM over the long haul any more. In the early 2000's, over half of the long distance calls in the US were VoIP. I'm not in the business any more, but I'm certain that percentage is much higher today. Ever since the long haul went fiber, carriers have been moving away from TDM.
Sorry to be such a pedant, but this is a real rabbit hole of a discussion that won't be solved with a basic understanding of DS1/PRI circuits.
[1]: Phone calls are broken down in to two parts. Media is what telecom people call the audio that is transmitted. It is distinct from signaling, which is the information used to set up and tear down calls.
Frame slips won't cause an audio misconnect like this, because... well, lots of reasons. The channel layout (23B, 1D) you describe isn't TDM, it's PRI (primary rate interface). Slips occur at a lower layer; the T-carrier.
All T-carrier based telecommunications systems transmit at a fixed frequency. Even when the circuit is idle, a stream of bits are transmitted across the line, so that both ends remain in sync. A loss in synchronization is called a slip. That much you got correct.
These circuits are digital though. When a slip occurs, it's not as if whole frames jump from one to the next. What happens when you perform a binary shift on a chunk of data? The outcome is wildly unpredictable. The protocols at higher layers (like PRI) have mitigation strategies for dealing with slips. A re-sync occurs almost instantaneously. A T-carrier circuit that slips regularly will not be reliable. Based on my experience, persistent frame slips result in either a dropped call (B-channel reset), or garbled audio (due to shifted bits in the media [1]). If you've ever been on a call, and the audio suddenly becomes a garbled mess that sounds like a mix of modem communication and alien speech, that's what bit-shifted media sound slike.
I have never seen a misconnect caused by a slip. I suppose it's possible, but the likelihood would seem extremely remote. Imagine if you Base64 encoded something, performed a single binary shift, and the Base64 decoded result turned out to be the lyrics of the Star Spangled Banner. Unlikely.
To top it all off PRI is virtually non-existent within carriers. PRI is an end-user interface. You only see it in the last mile. Carriers use protocols like SS7, which you'll never see on end-user equipment. Even SS7 is less common these days, because the vast majority of telephone calls aren't TDM over the long haul any more. In the early 2000's, over half of the long distance calls in the US were VoIP. I'm not in the business any more, but I'm certain that percentage is much higher today. Ever since the long haul went fiber, carriers have been moving away from TDM.
Sorry to be such a pedant, but this is a real rabbit hole of a discussion that won't be solved with a basic understanding of DS1/PRI circuits.
[1]: Phone calls are broken down in to two parts. Media is what telecom people call the audio that is transmitted. It is distinct from signaling, which is the information used to set up and tear down calls.
DS1 clocks are not required to be synchronized, they are interleaved into DS3 with lossless bit-stuffing.
When there is a loss of frame sync, signalling channels are inhibited. There's no transient synchronization recovery condition where channels are misaligned, the frame sync must be recovered first, and when it's recovered, all channels are correctly aligned. I used to design this equipment. I'm afraid this explanation is not correct.
When there is a loss of frame sync, signalling channels are inhibited. There's no transient synchronization recovery condition where channels are misaligned, the frame sync must be recovered first, and when it's recovered, all channels are correctly aligned. I used to design this equipment. I'm afraid this explanation is not correct.
It's more likely a software issue in the protocol stack that negotiates and control the switching fabric.
I work in telecoms and specialized in development and testing of TDM for almost a decade. I never saw clock slip manifest itself as a whole channel shift. The drift occurs at bit level, not in chunks of 8 bits. The elastic buffers in the phy protect against behavior and if they overflow due to a persistent drift it only results in a click in the channel.
I work in telecoms and specialized in development and testing of TDM for almost a decade. I never saw clock slip manifest itself as a whole channel shift. The drift occurs at bit level, not in chunks of 8 bits. The elastic buffers in the phy protect against behavior and if they overflow due to a persistent drift it only results in a click in the channel.
cool info, thanks.
wow
You could report it with your phone carrier, and have your mom do the same. Carriers keep something called CDRs that will have some level of debugging information associated with it. They might turn up something obvious right away. The truth is, they're not going to do anything about it unless it's recurring. And then, you'd better hope it is reproducible, or the techs will keep closing the tickets and moving on.
I've done a lot of telecom work for intermediate sized carriers. I've dredged through many gee-bees (that's gigabytes, folks) of SIP logs, cross reference my ass off, set up intermediate traces with IXCs, you name it, all in search of answers to problems just like the one you described. The problem is that the system doesn't stay the same for long. The telephone network in the US is more varied and complex than the Internet, because it has an unholy mix of legacy and new technologies glued together in ways that will make you lose sleep at night. At some point, you develop an appreciation for the fact that it works at all.
I've done a lot of telecom work for intermediate sized carriers. I've dredged through many gee-bees (that's gigabytes, folks) of SIP logs, cross reference my ass off, set up intermediate traces with IXCs, you name it, all in search of answers to problems just like the one you described. The problem is that the system doesn't stay the same for long. The telephone network in the US is more varied and complex than the Internet, because it has an unholy mix of legacy and new technologies glued together in ways that will make you lose sleep at night. At some point, you develop an appreciation for the fact that it works at all.
When I was a kid we had ISDN bonded for an insanely fast 128k to the internet. Unfortunately our ISP was in another city and the telco had only configured 8 of the 256 available trunk channels, so sometimes channel B would get routed to nowhere giving only 64k, or worse both/channel A would get that treatment and we'd have no access at all. Sometimes it would take 30 minutes of repeated redialing to get a connection.
No amount of filing trouble tickets or contacting SBC (now ATT) support would get anyone to pay attention. They all assumed it was a problem with our equipment or the ISP and couldn't have cared any less.
In the end, I drove up to the switching center building and knocked on the unmarked door (small town obviously, and I was 16 so no fear of being arrested) and managed to explain the problem to an actual engineer. They brushed me off but I guess someone paid attention because two weeks later that engineer called me and explained the misconfigured trunk and apologized for the poor service. Never had another problem after that.
So the answer to your question is you can either try to get in touch with one of the telco engineers directly in-person, or you can give up.
No amount of filing trouble tickets or contacting SBC (now ATT) support would get anyone to pay attention. They all assumed it was a problem with our equipment or the ISP and couldn't have cared any less.
In the end, I drove up to the switching center building and knocked on the unmarked door (small town obviously, and I was 16 so no fear of being arrested) and managed to explain the problem to an actual engineer. They brushed me off but I guess someone paid attention because two weeks later that engineer called me and explained the misconfigured trunk and apologized for the poor service. Never had another problem after that.
So the answer to your question is you can either try to get in touch with one of the telco engineers directly in-person, or you can give up.
Yes. Some construction crew put a backhoe through one of my circuits several years ago. You can call hicap support every hour politely requesting status updates, but that's no substitute for driving around the neighborhood until you find the poor engineer rewiring everything and bring him a coffee / iced tea.
Unfortunately, this works better in your small town or my small town than in San Francisco.
Unfortunately, this works better in your small town or my small town than in San Francisco.
My family used to have a hilarious issue where callers hanging up in the moment between the last ring and the answering machine would get called back by the answering machine. Thus every so often we would get messages that were the sound of a phone ringing followed by "hello! ... hello? ... ". Never really figured out why; it wasn't a feature of the machine as far as we could tell and it simply went away after a few years.
On topic, I think the phone system's routing is much less perfect than people think. But I can't imagine how you would go about reporting this; I have a hard time finding anyone at my telecom's customer service that even understands the most basic tethering questions or what plan I'm on.
On topic, I think the phone system's routing is much less perfect than people think. But I can't imagine how you would go about reporting this; I have a hard time finding anyone at my telecom's customer service that even understands the most basic tethering questions or what plan I'm on.
Three way calling (conference calling):
Subscriber A (who has 3 way calling) calls Subscriber B and an answering machine or modem answers the call; Subscriber A hangs up and quickly picks up the phone to make another call or the receiver bounces when they hung up (hookflash).
The phone system thinks Sub A tried to conference in another caller so it put subscriber B on hold (instead of disconnect), but since the three way call failed, it attempts to reconnect Sub A to the call on hold with Sub B. To do this, it rings back Subscriber A.
Subscriber A's answers their ringing phone, but hear nothing. Subscriber B's answering machine records the system calling Subscriber A back (ringback tones) and the confused "hello?, HELLO?!?" from Subscriber A who hears nothing as the answering machine is silent.
This also sometimes happened when a friend of mine tried to setup a modem game with me for DOOM at 2am. If the connection failed for some reason and did a hookflash (and my modem was still trying to connect), it attempted to reconnect him by ringing his line back and waking up his entire family. When his mom answered the call to my screeching modem that was still trying to connect: BUSTED.
So lesson learned: disable 3 way calling. You probably don't use it and it often results in this absurd but sometimes hilarious behaviour. Playing DOOM at 2am will result in terrible grades in high school.
Subscriber A (who has 3 way calling) calls Subscriber B and an answering machine or modem answers the call; Subscriber A hangs up and quickly picks up the phone to make another call or the receiver bounces when they hung up (hookflash).
The phone system thinks Sub A tried to conference in another caller so it put subscriber B on hold (instead of disconnect), but since the three way call failed, it attempts to reconnect Sub A to the call on hold with Sub B. To do this, it rings back Subscriber A.
Subscriber A's answers their ringing phone, but hear nothing. Subscriber B's answering machine records the system calling Subscriber A back (ringback tones) and the confused "hello?, HELLO?!?" from Subscriber A who hears nothing as the answering machine is silent.
This also sometimes happened when a friend of mine tried to setup a modem game with me for DOOM at 2am. If the connection failed for some reason and did a hookflash (and my modem was still trying to connect), it attempted to reconnect him by ringing his line back and waking up his entire family. When his mom answered the call to my screeching modem that was still trying to connect: BUSTED.
So lesson learned: disable 3 way calling. You probably don't use it and it often results in this absurd but sometimes hilarious behaviour. Playing DOOM at 2am will result in terrible grades in high school.
We used to have the same problem. Would love to know the reason why it happened.
That or OP has a Macbook, uses TOR, and reads Boing Boing.
When I worked in VoIP dpmnt. of some IM provider users were reporting issues like this. Sometimes caller and callee were connected to other party in a random manner.
It turned out this strange behavior was caused by bug in PBX software- TDM channels were linked incorrectly under heavy load on E1 links. AFAIK this bug is known for years and it is still not fixed.
But... I guess it's not related to your case, since vendor of this PBX is Polish.
These things happen quite a bit in India and we call it a cross connection.
I guess its just a switching problem between two providers.
Similar things have happened with my cell a few times. Once, my sister called and left a "happy birthday" voice mail. She got a return call from a confused woman whose device showed a missed call from my sister, and who had received the voicemail in my stead. Weird.
I thought perhaps it could be incorrectly entered MSIDs because the calls always seem to be mis-routed to the (theoretically) correct geographical location, but I know nothing beyond my personal experience.
I thought perhaps it could be incorrectly entered MSIDs because the calls always seem to be mis-routed to the (theoretically) correct geographical location, but I know nothing beyond my personal experience.
Both phone-companies have extensive logs of all calls. Using them to troubleshoot problems however always means manually reading every step. Because of this labor intensity and the privacy aspect (do you want every helpdesk-agent to look at the real source of anonymous calls?) I doubt you can convince them to research this, unless it's something that occurs frequently.
This happened to me once. My parents were in the Netherlands and I was in Germany. I tried to call them and had a confused dutch person on the other side and my parents also talked to some dutch guy at the same time
again the parents, hmm. But what did that call cost?
This reminds me of a weird calling issue I came across, but mine is obviously unrelated. I needed to make calls from America to Australia (inquiring about apartments from the Australian equivalent of Craiglist), so I was using a calling card that I typically use for calls around North America.
I make the first call to Australia, and the apartment was suppose to be owned by a woman looking for a roommate. Anyway, a man picks up the line and says hello. I can hear a woman in the background talking, and dogs barking. I ask for the woman by name, and the man says to hold on one second. Then he starts arguing with the woman for a minute, returns to the line and asks if I'm still there, and then says to hold one moment, and now you hear the dogs barking louder, and it sounds like he's taking them outside because the woman is complaining. There's also a TV or radio in the background. I think he returns again a couple of minutes later, apologizes and asks me to wait longer, but I can't be certain. I remember after about 5 or 10 minutes I decided to hang up because I'm calling long distance, the man sounded rude, and it didn't seem like a place I'd like to live any way.
Now, I call a few other places throughout the week, and get a few people on the line, and others with weird connection errors saying the numbers can't be reached.
Then, I'm making a call to a number in a completely different city a few thousand kilometers away, and the same man from before answers. He asks me to hold one second, and I don't recognize the voice at first. Suddenly, I hear the TV, radio and the dogs barking again, and get this weird sense of deja vu as things start to repeat. I hang up. I now realize it's a recording, and I never talked to a real person the first time around. But... I'm calling a completely unrelated number, so how did I get this message again? I call the number back, and now it goes through to the actual person I was trying to reach.
This would continue to happen on future calls, but by then I knew the routine and would hang up and redial.
Any idea what was happening here? I used these calling cards all the time and never had an issue. Somehow my calls were getting sent to this recorded message when calling Australia, regardless of the number I was dialing. The recording was trying to keep me on the line for an extended period of time.
My only conclusion is that these cards are getting rerouted through another country or less than trustworthy business. They then forward a certain percentage of calls to this message in order to get longer call times, and higher profits.
I make the first call to Australia, and the apartment was suppose to be owned by a woman looking for a roommate. Anyway, a man picks up the line and says hello. I can hear a woman in the background talking, and dogs barking. I ask for the woman by name, and the man says to hold on one second. Then he starts arguing with the woman for a minute, returns to the line and asks if I'm still there, and then says to hold one moment, and now you hear the dogs barking louder, and it sounds like he's taking them outside because the woman is complaining. There's also a TV or radio in the background. I think he returns again a couple of minutes later, apologizes and asks me to wait longer, but I can't be certain. I remember after about 5 or 10 minutes I decided to hang up because I'm calling long distance, the man sounded rude, and it didn't seem like a place I'd like to live any way.
Now, I call a few other places throughout the week, and get a few people on the line, and others with weird connection errors saying the numbers can't be reached.
Then, I'm making a call to a number in a completely different city a few thousand kilometers away, and the same man from before answers. He asks me to hold one second, and I don't recognize the voice at first. Suddenly, I hear the TV, radio and the dogs barking again, and get this weird sense of deja vu as things start to repeat. I hang up. I now realize it's a recording, and I never talked to a real person the first time around. But... I'm calling a completely unrelated number, so how did I get this message again? I call the number back, and now it goes through to the actual person I was trying to reach.
This would continue to happen on future calls, but by then I knew the routine and would hang up and redial.
Any idea what was happening here? I used these calling cards all the time and never had an issue. Somehow my calls were getting sent to this recorded message when calling Australia, regardless of the number I was dialing. The recording was trying to keep me on the line for an extended period of time.
My only conclusion is that these cards are getting rerouted through another country or less than trustworthy business. They then forward a certain percentage of calls to this message in order to get longer call times, and higher profits.
I've had this happen a few times, only it's the caller that ends up getting someone else when they dial my number.
I"m on T-Mobile.
I used to have Verizon, and before that AT&T and have kept the same number. I've suspected it was something to do with me porting my number.
I"m on T-Mobile.
I used to have Verizon, and before that AT&T and have kept the same number. I've suspected it was something to do with me porting my number.
This happened to me couple of times in a different country, and again in a yet another country. All while calling on local numbers from local numbers. I suspect it is error that happens due to design, not a bug. It happens in other countries' systems too.
I wish I knew. I've been getting text message conversations delivered to my Google Voice account from some other person, but appearing to come from me. No idea how to report it.
I had a similar problem when I changed phone numbers. My GV account was still setup for the old number so my calls and texts would go to the guy who got assigned my old number... He was not appreciative.
Oh, wow, that's exactly what it is. I had a temporary number for a while that I added to GV and forgot to remove later on. Thanks for the hint!
I wonder how often these kinds of errors happen? I've never heard of that happening to anyone. I thought automated switchboards were supposed to solve problems like this.
Consider that the software that switches and connects phone calls is updated as new hardware goes into place and as IP routing takes over. I'm sure there are lulls in the occurrence of these bugs as they're fixed on the dominant devices and then before new devices or technologies take over.
right - now that software's in charge, nothing should ever go wrong!
These stories happened a lot in my reading up on Phreaking history. This is just a networking error, I wouldn't worry about it unless it keeps happening.
Is it reproducible? You (or her) can call 611 for repair service from your phone company.
That’s pretty wild. Sounds like your calls got flipped on the routing board somehow.
wow, and I thought that only happened to me!
Some weird american voice was on the other side and it sounded a bit like a comedy show, but interestingly the voice was not made out of pieces and put together into sentences, but actually coherent and responsive to my questions. Living in Germany, this was actually weird enough to think that someone is in control of our cell-phones.
But why did we both have the caller-id of our mom's?
But why did we both have the caller-id of our mom's?
cosmic ray bit flip?
maybe you could ask the NSA about it... they seem to keep those records. </sarcasm>
What happened:
First off: I live in SF, work on the peninsula, and have a Utah (801) area code. My mom lives in New Mexico. She has Verizon, I have AT&T.
Yesterday, my mom shows up on my caller ID. I leave my office and walk down the hall. I answer.
On the other end is a strange man. We are both confused. He asks if I'm so-and-so from Modesto, California. I say no, he is dumbfounded, "Really?! Wow." I ask him what number he is calling, and he starts with "209", but the call drops out before he finishes.
I call my mom back. She picks up. I ask her if she just called me, and she says yes, she left a voicemail, but she was confused because she got a voicemail from some "lady who sounded weird". She left a message, asking if I was in trouble, and telling me to call back.
I think when she hung up after leaving the short voicemail, it may have terminated the call I had with this guy.