Category Archives: Tech

Technical stuff

Bonjour, Goodbye: A Strange iTerm2 Networking Bug

From time to time, I experienced strange behaviour in my programs and scripts running on my Mac. They became unable to connect to some hosts on my local network: only some, not all. This caused failures in my photo-processing scripts and data backups, and it seemed to happen after something was triggered on the system.

Most of the error messages were network-related, such as “No route to host”.

Connecting to my local API server using curl (or a Python script) didn’t work, but it worked in my browser.

Tired of my scripts and backups failing randomly, I investigated what might be happening. I had a workaround, but it required running those scripts after opening an SSH session to localhost (or running everything as root…).

And it happened again, so I went down the rabbit hole. (Spoiler: a self-upgrade likely triggered it.)

The mystery: two hosts, one subnet, different outcomes

I couldn’t directly connect to a host named pdp17 except when running it via a local SSH session, while I could connect to one called cosmos.

pineapple:~ kewl$ ping -c 1 pdp17 ; ssh localhost ping -c 1 pdp17
PING pdp17.in.eo.lu (10.31.33.23): 56 data bytes

--- pdp17.in.eo.lu ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
(kewl@localhost) Password:
PING pdp17.in.eo.lu (10.31.33.23): 56 data bytes
64 bytes from 10.31.33.23: icmp_seq=0 ttl=64 time=0.498 ms

--- pdp17.in.eo.lu ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.498/0.498/0.498/0.000 ms
pineapple:~ kewl$ arp -an | grep 10.31.33.23
? (10.31.33.23) at bd:14:41:f1:42:42 on en11 ifscope [ethernet]
pineapple:~ kewl$ ping -c 1 cosmos
PING cosmos.in.eo.lu (10.31.33.5): 56 data bytes
64 bytes from 10.31.33.5: icmp_seq=0 ttl=64 time=0.356 ms

--- cosmos.in.eo.lu ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.356/0.356/0.356/0.000 ms

Eliminating the usual suspects

Maybe it’s related to the system environment, so I compared the environments in the SSH and local sessions. Nothing relevant.

DNS resolution? Nope.

ARP, routing and interface selection? Nope.

Running tcpdump while pinging revealed that the echo request was sent and its reply received, despite the packets being reported as lost.

The permissions paradox

But in the same shell, running sudo ping pdp17 worked.

I opened Terminal.app (instead of iTerm2), and I could ping pdp17 there.

Maybe that macOS Privacy and Security feature for the Local Network? That could explain why root can connect, and also why it works when run via SSH, since the SSH daemon’s parent is launchd. More information here; the relevant notes are:

macOS maintains separate local network privacy state for each user account.

macOS automatically allows local network access by:

  • Any daemon started by `launchd`
  • Any program running as root
  • Command-line tools run from Terminal or over SSH, including any child processes they spawn

important: The exception for `launchd` daemons doesn’t apply to `launchd` agents. If you’re building a `launchd` agent, see the discussion below.

But why would it work on some hosts on the same network? Anyway, it was permitted in System Preferences. Same verification with the firewall rules: there were none.

The breakthrough: Unified Logging

I checked Unified Logging while trying to connect in another session:

pineapple:~ kewl$ sudo log stream --style compact --info --debug  --predicate 'subsystem == "com.apple.networkextension"'
...
2026-11-18 14:03:37.774 I UserEventAgent[115:24bd5f] [com.apple.networkextension:] Got local network blocked notification: pid: 1711, uuid: 44250719-9DD6-396A-B5CD-51BFAD462FFE, bundle_id: (null)  
2026-11-18 14:03:37.774 E UserEventAgent[115:24bd5f] [com.apple.networkextension:] Failed to get the signing identifier for 1711: No such process  
2026-11-18 14:03:37.774 Db UserEventAgent[115:24bd5f] [com.apple.networkextension:] Signing identifier cache miss for 44250719-9DD6-396A-B5CD-51BFAD462FFE  
2026-11-18 14:03:37.774 I UserEventAgent[115:24bd5f] [com.apple.networkextension:] LocalNetwork: did not find bundle ID for UUID 44250719-9DD6-396A-B5CD-51BFAD462FFE  
2026-11-18 14:03:37.774 E UserEventAgent[115:24bd5f] [com.apple.networkextension:] Failed to find bundle ID, ignoring

PID 1711 no longer existed when I checked, and macOS couldn’t determine its signing identifier or associate it with an application bundle. Could a process have died? So it’s a process that isn’t allowed to connect, and it’s not iTerm2.

I later wanted to investigate what this PID could be, and my assumption that it wasn’t iTerm2 was both right and wrong: it was the PID of iTerm2 before it self-updated, and ended up with helper processes from versions 3.7.6 and 3.7.7 running simultaneously. Also, the bundle 44250719-9DD6-396A-B5CD-51BFAD462FFE?

I investigated the UUID reported in the Local Network blocked notification: 44250719-9DD6-396A-B5CD-51BFAD462FFE, and after discovering that components from both iTerm2 3.7.6 and 3.7.7 were still running, I checked the old application’s executable in my backup using dwarfdump –uuid, and it was a match!

When different tools disagree

Establishing the connection with netcat worked, but not with telnet:

pineapple:~ kewl$ nc -v pdp17 22
Connection to pdp17 port 22 [tcp/ssh] succeeded!
SSH-2.0-OpenSSH_10.0p2
^C
pineapple:~ kewl$ telnet pdp17 22
Trying 10.31.33.23...
telnet: connect to address 10.31.33.23: No route to host
telnet: Unable to connect to remote host

telnet is installed from Homebrew, but nc comes with the OS. I also inspected their code signature, which was irrelevant (the command used was: codesign -dv --verbose=4 "$(command -v nc)" 2>&1 ; replace nc with telnet and ssh)

Another interesting pattern was when forcing IPv4: ssh -4 pdp17 led to a stalled socket that didn’t time out for several minutes, but running ssh -4 10.31.33.23 connected to the host.

 SSH itself had another strange error; while most tools returned “No route to host”, SSH had its own message, which is quite obscure.

pineapple:~ kewl$ ssh pdp17 id
ssh: connect to host pdp17 port 22: Undefined error: 0

 But where it got interesting is when I added an argument to ssh, it connected:

pineapple:~ kewl$ ssh -o ConnectTimeout=3 pdp17 id
uid=1000(kewl) gid=1000(kewl) groups=1000(kewl),27(sudo),100(users)

 OpenSSH’s options affect the connection behaviour. It may change the connection to use non-blocking sockets.

And I found this web page where someone had a similar problem, where they mention:

So the evidence points to the blocking connect() to these mDNS-resolved IPv6 addresses failing on current macOS build, while the non-blocking path works. This could be a bug in this macOS build’s handling of mDNS-resolved scoped addresses, or something in the Local Network privacy layer.

The unexpected fix: disabling Bonjour in iTerm2

That blog post invited me to look towards mDNS. So I tried disabling Bonjour in iTerm2’s Magick settings. Then I opened a new tab, and it just connected.

I found a workaround which doesn’t require me to change my process, but it’s not the root cause.

One final observation

Disabling Bonjour in iTerm2 restored connectivity, but that doesn’t necessarily mean Bonjour was the problem.

After the troubleshooting session, I looked deeper into Unified Logging and found the UUID in macOS’s Local Network blocked notification matched the executable from the previous version of iTerm2, which I had upgraded the day before. Components from both versions were still running.

I suspect that the update left some stale application identity or network permission state, possibly interacting with iTerm2’s Bonjour integration, but it’s only a hypothesis.

So, maybe Bonjour was the trigger, the workaround reset something, or the two observations are related in another way.

One other detail I remembered after I wrote this text: I also disabled Wi-Fi to use my wired Gigabit Ethernet connection. That may have triggered a network-interface change while iTerm2 was running, possibly affecting Bonjour discovery or macOS’s Local Network privacy state. So the self-upgrade wasn’t necessarily the only trigger.

Below are the versions I used:

  • iTerm2: Build 3.7.7, but components of 3.7.6 and and 3.7.7 ended up running simultaneously.
  • macOS: Darwin pineapple 25.6.0 Darwin Kernel Version 25.6.0: Mon Oct 12 00:33:09 PDT 2026; root:xnu-12377.161.15.701.48~2/RELEASE_X86_64 x86_64
  • OpenSSH: OpenSSH_10.5p1, LibreSSL 3.3.6

The investigation was done on November 18, 2026.

What this taught me

The Network is the Computer, and the terminal emulator has opinions about both!

enfuse examples

Last month, I took a time lapse session of New York skyline during dusk. I wanted to merge them and turned to Enfuse. As I didn’t find too many examples I wrote a small script to generate some series of images out of a sample of the time lapse pictures. The result is quite amazing and looks promising. Below is the average image of 201 photos taken during 40 minutes.

all-default

The shell script can be downloaded here (attention! it’s not idiot proof 😉 ). Just copy your image files to a directory with the script and check it out. It runs on both Linux and Mac systems and requires to have Enfuse installed and the command enfuse in the system path.

Note that each picture can be opened in full size, using Open Image in New Tab. For the tests, I have reduced the number of images to 10 of them during the twilight.

Rendered gray projectors

Various gray-projectors in soft-mask, using --gray-projector=value, where value is below:

average normal-average
lightness normal-lightness
l-star normal-l-star
luminance normal-luminance
pl-star normal-pl-star
value normal-value

The difference is not obvious; however if we enable the flag --hard-mask, then the difference becomes evident:

average hardmask-average
lightness hardmask-lightness
l-star hardmask-l-star
luminance hardmask-luminance
pl-star hardmask-pl-star
value hardmask-value

Sigma and Mu variation

The following images have been rendered using the average gray projector.

Sigma

value soft-mask hard-mask
0.00 soft mask sigma0.00 hard mask sigma0.00
0.10 soft mask sigma0.10 hard mask sigma0.10
0.20 soft mask sigma0.20 hard mask sigma0.20
0.30 soft mask sigma0.30 hard mask sigma0.30
0.40 soft mask sigma0.40 hard mask sigma0.40
0.50 soft mask sigma0.50 hard mask sigma0.50
0.60 soft mask sigma0.60 hard mask sigma0.60
0.70 soft mask sigma0.70 hard mask sigma0.70
0.80 soft mask sigma0.80 hard mask sigma0.80
0.90 soft mask sigma0.90 hard mask sigma0.90
1.00 soft mask sigma1.00 hard mask sigma1.00

Mu

value soft-mask hard-mask
0.00 soft mask mu0.00 hard mask mu0.00
0.10 soft mask mu0.10 hard mask mu0.10
0.20 soft mask mu0.20 hard mask mu0.20
0.30 soft mask mu0.30 hard mask mu0.30
0.40 soft mask mu0.40 hard mask mu0.40
0.50 soft mask mu0.50 hard mask mu0.50
0.60 soft mask mu0.60 hard mask mu0.60
0.70 soft mask mu0.70 hard mask mu0.70
0.80 soft mask mu0.80 hard mask mu0.80
0.90 soft mask mu0.90 hard mask mu0.90
1.00 soft mask mu1.00 hard mask mu1.00

Mu and Sigma matrix

Only using the hard mask rendering where Mu values are horizontal and Sigma vertical:

  0.00 0.25 0.50 0.75 1.00
0.00 hard mask mu 0, sigma 0 hard mask mu 25, sigma 0 hard mask mu 50, sigma 0 hard mask mu 75, sigma 0 hard mask mu 100, sigma 0
0.25 hard mask mu 0, sigma 25 hard mask mu 25, sigma 25 hard mask mu 50, sigma 25 hard mask mu 75, sigma 25 hard mask mu 100, sigma 25
0.50 hard mask mu 0, sigma 50 hard mask mu 25, sigma 50 hard mask mu 50, sigma 50 hard mask mu 75, sigma 50 hard mask mu 100, sigma 50
0.75 hard mask mu 0, sigma 75 hard mask mu 25, sigma 75 hard mask mu 50, sigma 75 hard mask mu 75, sigma 75 hard mask mu 100, sigma 75
1.00 hard mask mu 0, sigma 100 hard mask mu 25, sigma 100 hard mask mu 50, sigma 100 hard mask mu 75, sigma 100 hard mask mu 100, sigma 100

Looks like the fun has just begun 🙂

Unix redirection fail

This morning the / file system was full on one of our AIX servers:

root@host[/root] > df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 524288 0 100% 5311 89% /

I didn’t take more than a few seconds to realise where this came from:


root@host[/root] > cd /
root@host[/] > du -x -k | sort -n | tail -5
14348 ./etc/perf
15328 ./lpp/bos
30900 ./etc
180880 ./dev
258580 .
root@host[/] > cd dev
root@nim1[/dev] > ls -lSr | tail -1
-rw-r--r-- 1 root system 185118720 Sep 2 15:26 null 2>&1

Interesting redirection… And the file ‘null 2>&1’ contained some interesting stuff, which helped identifying what process wrote to this file. Just deleted it, modified the script, and the incident was solved.

The Effect of Time

This is an unexpected effect of time:
PING CRITICAL – Packet loss = -400%, RTA = 1029419237376.00 ms

The absurd Round-Trip Average (RTA)

1029419237376.00 milliseconds ~ 32.64 years.

This happened because the system time went backwards during the ping test.

It works by sending a small data packet (an ICMP echo request) and waiting for a reply, while stamping the “send time” and “receive time” using the system clock.

If the clock suddenly jumps back, the “receive time” can appear to be earlier than the “send time.”

This creates a negative round-trip time, which is interpreted as a large number because the system stores the value as an “unsigned” (only positive) integer; a small negative value is displayed as a large positive number.

But why did packet loss become -400%?

Packet loss is calculated as a percentage of packets that were expected but never received. The formula is something like:
Packet Loss % = ((Packets Sent - Packets Received) / Packets Sent) * 100

A negative packet loss means the system believes it received more packets than it sent. But how could that happen?

When the system clock moves backwards, the ping tool’s internal accounting gets confused. It can miscount the replies, often because duplicate replies are suddenly counted as new, separate packets. If it sent 5 packets but, due to the glitch, counted 25 replies, the math becomes: ((5 - 25) / 5) * 100 = -400 (%).

Challenge

Can you determine the time shift that occurred while the check was running, and what are the odds that the system time changes backwards during the time interval between an echo request and echo reply?

Non-technical explanation

Imagine you’re timing a race with a stopwatch. You click ‘start’ when the runner leaves, but the clock fails and resets to midnight; when the runner finishes, after you click ‘stop’, the watch shows a negative time. If the stopwatch wasn’t designed to display negative numbers, it will display a huge, meaningless number instead.

Nice Firefox extension: Web Developer

Hey there,

I found a great Firefox extension that is worth it!

Have a look at this partial screenshot:

partial screenshot

For example, it allows CSS edit on the fly and see changes directly: no reload/upload necessary! Adjust your style, then save and put live, isn’t that wonderful?

You can also play with privacy, display image information, div, frames, validate Links on the page, and many other interesting things.

Looks like I found a new friend 😀