home    builders    Search

builders ➜ Debug A Poor Link

Debug A Poor Link

Pardon the lack of organization here. This page will need polishing.

How to characterize a link

The neighbor on the other side of the link should show on the routes nodes list. Run the r r command.
See also G8BPQ R command.
SPHEN:KA4QRT-2} Routes
> 1 K1QSY-2   200  14! 442  236  33% 0 0 19:41  6 200
> 2 WD1QRS-2  200  14! 221   17   7% 0 0 19:41  0 200
> 3 K2CC-2    200  12! 562    5   1% 0 0 19:42  1 200
  4 K7QTH-2   200   0!  55    0   0% 0 0 21:02  0 156
In the example above, port 1 K1QSY-2 has a bad retry rate. Averaged over the last 442 information frames (messages with text) the node has had to transmit 236 extra copies of the information frame in order to complete the message. That has resulted in a 33% figure. We like to keep the percentage number down to about 9% in an acceptable link. 3% is considered excellent. The higher the percentage number, the slower the link will seem.
Also important in K1QSY's line is the 2nd to last figure. The 6 in this column means there are 6 messages waiting to go across this link. We'd like to see that figure at 0, or nearly 0, meaning little or no traffic waiting.

Further down in the r r result, port 4 really has a problem. The column before the ! shows 0 nodes sourced. That means the local node, SPHEN, is not hearing the routing table "nodes" broadcast from K7QTH-2. Note that the percentage rate has been excellent. My guess in looking at this is that K7QTH-2 suffered a catastrophic failure, like somebody turned off the node entirely.

Your next move to solve K7QTH-2 bad link problem would be to see if K7QTH-2 show on the MHEARD 4 list and how long ago K7QGH-2 was heard.
See also G8BPQ M command.

Looking back at K1QSY-2. Connect across the link and see what the r r table on the other end shows.

MIKEY:K1QSY-2} Routes
> 1 KA4QRT-2  200   8! 806    4  2% 0 0 19:41  0 200
> 2 N8DDK-2   200   4!1221  170  7% 0 0 19:41  0 200
This shows us that the failure is directional. Either the transmission from SPHEN:KA4QRT-2 is too weak, has bad audio, or maybe MIKEY:K1QSY-2 has an incorrectly set receive volume level?

Even if the r r table shows a good link to one of your neighbors, you should connect across to the neighbor once in a while and see what they show looking back at you. In our current environment we can't tell the entire story about a link without looking at both ends, though if you have a radio and antenna that can hear both ends, you can do a pretty good analysis just listening to a busy channel.

Required for a good link

  1. 20db quieting - the stations must be strong to each other with no buzzing or clicking
  2. transmit modulation just below saturation of the transmitter
  3. transmitted signal must match the bandwidth of the receiver (commercial radio wide vs narrow mode setting can screw this up)
  4. adequate but not over-loud receive level to the modem in the TNC
  5. audio must be relatively flat. The high tone can actually be a very different volume than the low town, but only if the other characteristics are spectacular
  6. During any packet transmission, the modem tones must be audible at the receive TNC's modem before the important bits of the packet start. This can be foiled by a slow squelch or too-short TXdelay
  7. Radio should have low audio distortion. Some radios will distort the data enough that 15% is the best rate we'll get with big packets

See also Configure TNC-PI.
See also TNC and Radio Adjust and Test

Things that can hurt the link

  1. Weak signal
  2. too long a path or too much in the way
  3. Local interference to the receiver on one end or the other
  4. Bad radio on one end or both ends
  5. In-band desense from another local transmitter
  6. multipath
  7. TxDelay is too short
  8. Receive audio level from the radio to the TNC is too high or too low
  9. Transmit level from TNC to radio is too high or too low
  10. RFI (from one of your transmitters) affecting the modem
  11. Receive squelch taking too long to open
  12. Radio shutting off its receiver to save power
  13. Automatic Gain Control messing up the modem (this was a problem affecting DireWolf)
  14. radio off frequency
  15. Wiring error
  16. Not enough voltage during transmit
  17. Radio turned off
  18. Temperature affecting something
  19. Hidden transmitter causing collisions
  20. Hidden transmitter causing your receiver to start receiving a packet before the desired link starts
  21. Some radios will distort the data enough that 15% is the best rate we'll get with big packets

The way packet radio links over FM behave at the receiver is disconcerting in that the digital link can be on the threshold of failure, yet to the node the link appears to behave perfectly. Then when they cross the threshold into failure, the link appears to fail completely, only to recover suddenly shortly after. This can happen on a several‑second cycle; or minutes; or hours; or entire seasons.
To a human brain this is akin to having a great big suspension bridge across a river be there one day, missing completely for a couple of hours the next day, then suddenly re-appear after lunch.
Well, we're talking about radio here. If this was voice radio, we'd be able to understand signals that were rough to hear but were still useable. They'd dip into being really rough once in a while. Being gone suddenly and completely for a short period is just not natural.
So.. when debugging a packet radio link, falling back to listening to the link with our ears is a good plan.

Steps to debug a link

  1. Verify the routes and port numbers on each side.
  2. With the receiver driving a speaker on one side, have the other transmit a signal. It should be full quieting. Repeat with the other direction.
  3. Set up a monitor receiver to listen to both ends at the same time. The receive signal from the distant station need not be full quieting to the monitor receiver, you just have to be able to hear the entire packets from the other end.
  4. Verify that the TXDELAY leaves an audible lead-in tone from each station to your monitor receiver. Set the TXDELAY long, just to make sure.
  5. Run BPQterm on each end.
  6. Set the transmit audio on each end such that the audio is not distorted on a receiver at the transmit site. Do this with tarpn linktest <portnum>    while listening on a local receiver. To set the transmitter, turn on a receiver on that frequency and set the volume on the HT so you can hear it. Adjust the transmit level across its entire range and verify that you can tell the changes in transmit level on the HT. Set the transmitter TNC-PI so it's transmit level is just below the maximum (from the HT's receive perspective) that you can set it to. There is a point in the transmit audio level control, where the level will top out and everything beyond that is "overdriving" which will just be distorted and useless at the receive end. Set the transmitter audio level control to just below the point where the level tops out.
  7. Send packets from the transmit end, using Linux command tarpn linktest <portnum>   . Alternatively you can connect over the radio to the other end and request stat. If you are configured for TNC-PI with audio monitoring switches, set the switch to monitor for the one TNC-PI serving the link you are debugging. Observe the oscilloscope during receive packets and adjust the receive level for 0.8v Peak to Peak coming from the radio. In the display shown below, that's four squares of 0.2v each. Notice that for this link two tones are not the same volume so you get different Peak to Peak readings depending on the data. Anywhere from 0.6v to 1.4v (3 to 7 graticule squares) is adequate I think.
    Refer also to Using an Oscilloscope for measuring Rx Audio Level to TNC-PI
    If you do not have an oscilloscope, do tarpn linktest <portnum>    to send test packets from the transmit end. Adjust the receive audio level to receive all of the numbered packets.
  8. Or, connect to the other end and request stat. Lacking access to the linktest utility means you'll have to be able to listen to the channel to know when packets come across. Observe in TARPN-HOME (node tab, monitor checked) and try to arrange that all audible packets are decoded and displayed.
  9. Repeat on the other end
  10. No scope, no receiver method: Observe the BPQterm monitoring the link. On the other end of the link start a transmit cycle connecting to a non-existant callsign, as above, but now observe the timing of the transmits. At the receive end watch for packets which do not show on the BPQterm by looking for gaps in the expected traffic. You should be able to listen to the other end transmit and know if there are any changes in the period between packets. Fine tune the receiver's volume knob to optimize until no packets are missed.
  11. Repeat in the other direction
  12. Connect to both ends of the link and grab the current numbers from r r for the link you are working on.
  13. Now connect to the far end, connect back, connect out, back, out, back. Now do a stat command.
  14. Listen to the exchanges and verify that there are no times of dead air until the stat is fully received at your end
  15. Use BPQterm's monitor window to verify that every transmit has a response. See whose messages are being sent but not being received at the other end.
  16. Observe the INFO-frames sent value vs the INFO-frames-retried value. Compare against the starting value. You can use these values from both ends and identify which side is having difficulty sending to the other.
  17. There are still several things that can go wrong, including multi-path distortion. The clue that there is multipath is that the received signal from the other end of the link sounds distorted, but when listened to at the transmit end, is not distorted. Multipath can be caused by having a reflection which is as good as the direct path. It can be exacerbated by badly aimed directional antennas, or a major RF attenuator or hill directly in line between the two sites. Horizontal directional antennas are a good thing to combat multipath. If the antennas can't be moved or improved, the best thing to do is to change frequencies or even bands between the sites.
  18. If you do get it to work well, set the TXDELAY down to where it the lead-in is noticeable but not finicky short and not tedious.

to be continued

© Tadd Torborg, 2016↝2019 -- all rights reserved