Incredible. Every Single Take on AI is Wrong.

For the past 6 months, I've been reading on-and-off the flurry of blog posts and articles written by very experienced software developers on AI, and the overwhelming feeling I get is, that they are all wrong.

I don't have the inclination to write a treatise countering all the things people are saying, but I do have three articles for which I want to simply document agree/disagree on the various points being made. If only to document my own beliefs, so I can check a year or so in the future and check how badly I was off.

My AI Skeptic Friends Are All Nuts

This article is divided into sections, which is very convenient.

1. level setting — agree, you have to re-evaluate because LLMs have drastically improved this year
2. the positive case — agree, this is probably the strongest argument for LLM-assisted coding
3. but you have no idea what the code is — mostly agree, you must read all the code
4. but hallucination — idiotic argument
5. but the code is shitty, like that of a junior developer — idiotic argument #2
6. but it’s bad at rust — what even is this saying? wrong.
7. but the craft — applies only in some situations, and author doesn't see that
8. but the mediocrity — same as 7
9. but it’ll never be AGI — agree
10. but they take-rr jerbs — presented annoyingly, but directionally correct. there are always more things for people to do.
11. but the plagiarism ­— offtopic, and the author should not have delved into this at all
12. positive case redux — ah now your motte becomes clear, and it's bullshit. social pressure does not an argument make.
13. but i’m tired of hearing about it — oh please just... no

Overall, garbage article. Don't read it, except maybe sections 1, 2, 3, 9.

Contra Ptacek's Terrible Article On AI

 This is a response to the previous article. As you might guess, I disagree with this one too.

1. Immediate Red Flags — disagree, over-generalizing, nit-picking
2. Trash-Tier Ethics — agree! but also tbf, copyright has nothing to do with ethics. it is not a natural law.
3. Why The Appeals To Random Friends? — too long, but yeah same argument as me
4. Is AI Getting The Right Level Of Attention? — TOO LONG, but agreed
5. These Executives Are Grifting Or Incompetent — over-generalized again, but this is true of some execs, yes
6. Killing Strawmen — agree, same argument as me stated differently
7. Why The Half-Hearted Defense Of Artists? — offtopic, i will ignore again

Overall, fine, more like a rant than anything else, and doesn't acknowledge the correct parts of the first article, which is a shame. But I guess most people on the internet write when they feel strongly about something, and that cannot lead to good analysis.

AI Mania Is Eviscerating Global Decision-Making

Same author again. Same problem of writing only when he feels strongly on a subject, and making too many generalizations. Let's <ol> again:

1. AI Investments Are Generally Total Failures — agree, although many people will correctly disagree because it does not apply to their AI investments because it generalizes too much
2. Heretics Will Be Shot — only at some high-profile large companies, again generalizing from his own experience without trying to understand the shape of his filter-bubble
3. AI Demos Are The Mind-Killer — agree on the facts, but get off your high-horse. your refusal to take money just tells me you're living in luxury and nothing else.
4. Executives, Game Theory, and The Emperor’s Clothes — good, concise, and correct. this is how every single hype cycle in big tech functions. burn this in your memory, and see it happen every decade.
5. You Must Be This AI-Native To Ride —  only applicable to big tech. this sort of dumb thinking is actually good for the rest of us. if big tech didn't have dozens of achilles' heels like this, they would be unstoppable.
6. Navigating AI Mania — obviously correct, unless some architectural change happens in LLMs that drastically improves their real-world capabilities, not just in benchmarks

Overall a good article, you should read it, and skim when it gets too verbose and makes sweeping generalizations.

I just recalled, there is a good article on AI-assisted coding that I think everyone should read. And it's this one by danluu: https://danluu.com/ai-coding/ . I generally agree with everything listed in it, with reservations around the "no code review" parts.

You might notice that I didn't talk about any articles that are frothing-at-the-mouth regarding AI (in either direction). Those people aren't worth reading at all.

Well that's all I had folks. Hope you enjoyed this. 

An Esoteric Type of Memory "Leak"

A little while ago, my colleague Sebastian started complaining about OOMs caused by Evolution taking up tens of gigabytes of memory. We discussed using sysprof to debug it, but it was too busy a time for Sebastian to set aside a few hours to do that.

Funnily enough, the most efficient fix at the time was to buy more RAM, since rust-analyzer was also causing OOM issues.

A few weeks went by. Restarting Evolution had become a daily ritual for Sebastian. 

Then, on a whim, I decided investigating this might be a good test for an LLM.

I updated my Evolution git repo, built it, and started up Claude Code in the source root. This was the only prompt I supplied: 

Find memory leaks in Evolution, current sourcedir. Particularly leaks that could accumulate over several hours. A colleague has a leak that slowly accumulates memory usage to several GB over the course of a day, requiring a restart of Evolution. That is the main focus, but we can fix other leaks in the process.

I wish I was lying, but that was all Claude Code needed to find the problem: Evolution just needed to call malloc_trim(0) from time to time.

I refused to believe it at first. I was only convinced when we saw the memory drop after running gdb -p $(pidof evolution) -batch -ex "call malloc_trim(0)" -ex detach

This seems absurd! Doesn't glibc reclaim freed memory from time to time?

Yes, it does. It calls sbrk() to do that. However, sbrk() can only reclaim free memory at the top of the heap, since it simply moves the program break downward to do so. malloc_trim(0) calls sbrk() and then also calls madvise(..., MADV_DONTNEED) on the free pages, which allows the kernel to reclaim them.

So if you have 10GB of unused memory followed by 4 bytes allocated at the top of the heap, your RSS is >10GB, even if you're using a few hundred megs. Till you call malloc_trim(0).

Note that you can only get into this situation if you have hundreds of thousands of small allocs/deallocs happening repeatedly. If your alloc is >128KB, mmap() is used for the allocation, and none of this applies.

Coincidentally, GLib's use of GSlice for GObject allocations was masking this issue in the past, but GSlice has been a no-op for some time now (for good reasons). Ideally, Evolution should not be using GObject for such ephemeral objects.

Lesson learned: if you have memory usage issues and you suspect fragmentation, try malloc_trim(0) before you go thinking about fancy allocators.

A strange game. The only winning move is not to play.

That's a reference to the 1983 film “WarGames”. A film that has had incredible influence on not just the social milieu, but also cyber security and defence. It has a lot of lessons that need re-learning every couple of generations, and I think the time for that has come again.

Human beings are very interesting creatures. Tribalism and warfare are wired in our minds in such a visceral way, that we lose the ability to think more than one or two steps forward when we're trying to defend our tribe in anger.

Most people get that this is what makes warfare conducted with nuclear weapons particularly dangerous, but I think not enough words have been written about how this same tendency also makes warfare conducted with Social Media dangerous.

You cannot win a war on Social Media. You can only mire yourself in it more and more deeply, harming yourself, the people around you, and the potential of what you could've been doing instead of fighting that war. The more you throw yourself in it, the more catharsis you will feel, followed by more attacks, more retaliation, and more catharsis.

A Just War is addictive, and a Just War without loss of life is the most addictive of all.

The only winning move is not to play.

The Internet in general and Social Media in particular are very good at bringing close to you all kinds of strange and messed-up people. For a project like GNOME, it is almost unavoidable that the project and hence the people in it will encounter such people. Many of these people live for hate, and wish to see the GNOME project fail.

Engaging them and hence spending your energy on them is the easiest way to help them achieve their goals. You cannot bully them off the internet. Your angry-posting and epic blogs slamming them into the ground aren't going to make them stop. The best outcome is that they get bored and go annoy someone else.

The only winning move is not to play.

When dealing with abusive ex-partners or ex-family members, a critical piece of advice is given to victims: all they want is a reaction. Everything they're doing is in pursuit of control, and once you leave them, the only control they have left is over your emotional state.

When they throw a stone at you, don't lob it back at them. Catch it and drop it on the ground, as if it doesn't matter. In the beginning, they will intensify their attack, saying increasingly mean and cutting things in an attempt to evoke a response. You have to not care. Eventually they will get bored and leave you alone.

This is really REALLY hard to do, because the other person knows all your trigger points. They know you inside out. But it's the only way out.

The only winning move is not to play.

Wars that cannot be won, should not be fought. Simply because war has costs, and for the people working on GNOME, the cost is time and energy that they could've spent on creating the future that they want to see.

In my 20s and early 30s I made this same youthful mistake, and what got me out of it was my drive to always distil decisions through two questions: What is my core purpose? Is this helping me achieve my purpose?

This is such a powerful guiding and enabling force, that I would urge all readers to imbue it. It will change your life.

The Substance

I have many thoughts about this film, and I don't really have the time to put it all together into one coherent whole, but an incoherent whole is also what the film also gave us, so I don't mind mirroring it.

This film is three movies in a trench coat—one per act.

The First Act

The first act was about beauty standards. The entertainment industry. The youth-consuming machine. The brutality of every content-creation business. It makes you viscerally associate disgust and revulsion with the circumstances that The Protagonist finds herself in.

It's all true, it's all widely known, it's all been done before.

This act was also a trojan horse, because the next act shifts from social commentary to...

The Second Act

... commentary on the Self.

It's about a human, whose relationship with their many selves each with its own desires, is cause for a constant tug-of-war. They make this abundantly clear with a scene making clear that all shooting schedules will be accommodated around Sue's every-alternate-week availability. Any lack of balance is entirely Self-Inflicted.

It's about the Self and the Self alone.

It's about her.

It's about you.

The you who wants to be the best version of yourself, and the you that sabotages your body, your soul, your mind, and what you can be (or want to be).

If you struggle with alcohol, it's about the alcoholic in you, and the tearful regret the next morning.

If you have an eating disorder, it's about the gorger (or starver) in you, and the you that aches to do better.

If you are a revenge procrastinator, it's about the late nights, and the next day, when you curse yourself for not doing better.

Every day that you do not stay active and take care of your body, you take a day from yourself in old age.

What's been taken can't be given back.

Both those people are you. Tonight and tomorrow morning. In youth and in old age. You and she/he/they are the same. And yet, you sabotage yourself.



A little bit of temporary suffering today could save you a lot of permanent damage.



You could stop The Substance Abuse, but you can't. You choose to make the You of tomorrow suffer instead. Till you die...

... which is how this film could've ended, but they had room for another movie in there.

The Third Act

Avid cinema-goers know that the third act (more generally in media, the ending) is where a lot of stories fall apart. It's a natural consequence of the creation process.

Creation often begins with an evocative idea, and growing from there is natural and progressive. But a satisfying ending requires you to have two evocative ideas with a convincing progression that connects the two seamlessly.

It's very, very difficult.

Some of the best stories are those that have their genesis in an ending, and they grow backwards from there to a beginning. People are a lot more forgiving about beginnings. All's well that ends well.

This wasn't one of those stories.

The ending was The Thing, Planet Terror, Kill Bill, The Colour Out of Space. It was Camp. It was an entirely different genre of story.

It was, at least to me, pretty clearly about taking you on a roller-coaster ride that requires you to suspend disbelief so it can take you to an ending that happens to wrap back to the cold open with the Walk of Fame Star, which in retrospect was obviously construed entirely for the sake of the ending. It had little other purpose.

Personally, I enjoyed it, and I think I was meant to. It was an interesting way to side-step the problem of "how to have a satisfying ending". Like ending a three-course meal with a joyride for dessert.

It was a fun film, but I will never watch it again.

What is WebRTC? Why does it need ‘Signalling’?

If you’re new to WebRTC, you must’ve heard that it’s a way to do video calls in your browser without needing to install an app. It’s pretty great!

However, it uses a bunch of really arcane terminology because it builds upon older technologies such as RTP, RTCP, SDP, ICE, STUN, etc. To understand what WebRTC Signalling is, you must first understand these foundational technologies.

Readers who are well-versed in this subject might find some of the explanations annoyingly simplistic to read. They will also notice that I am omitting a lot of of detail, leading to potentially misleading statements.

I apologize in advance to these people. I am merely trying to avoid turning this post into a book. If you find a sub-heading too simplistic, please feel free to skip it. :-)

RTP

Real-time Transport Protocol is a standardized way of taking video or audio data (media) and chopping it up into “packets” (you can literally think of them as packets / parcels) that are sent over the internet using UDP. The purpose is to try and deliver them to the destination as quickly as possible.

UDP (user datagram protocol) is a packet-based alternative to TCP (transmission control protocol), which is connection-based. So when you send something to a destination (IP address + port number), it will be delivered if possible but you have no protocol-level mechanism for finding out if it was received, unlike, say, TCP ACKs.

You can think of this as chucking parcels over a wall towards someone whom you can’t see or hear. A bunch of them will probably be lost, and you have no straightforward way to know how many were actually received.

UDP is used instead of TCP for a number of reasons, but the most important ones are:

1. TCP is designed for perfect delivery of all data, so networks will often try too hard to do that and use ridiculous amounts of buffering (sometimes 30 seconds or more!), which leads to latencies that are too large for two people to be able to talk over a call.

2. UDP doesn’t have that problem, but the trade-off is that it gives no guarantees of delivery at all!

   You’d be right to wonder why nothing new has been created to be a mid-way point between these two extremes. The reason is that new transport protocols don’t get any uptake because existing systems on the Internet (operating systems, routers, switches, etc) don’t (want to) support them. This is called Protocol ossification, and it's a big problem for the Internet.

   Due to this, new protocols are just built on top of UDP and try to add mechanisms to detect packet loss and such. One such mechanism is…

RTCP

RTP Control Protocol refers to standardized messages (closely related to RTP) that are sent by a media sender to all receivers, and also messages that are sent back by the receiver to the sender (feedback). As you might imagine, this message-passing system has been extended to do a lot of things, but the most important are:

1. Receivers use this to send feedback to the sender about how many packets were actually received, what the latency was, etc.
2. Senders send information about the stream to receivers using this, for instance to synchronize audio and video streams (also known as lipsync), to tell receivers that the stream has ended (a BYE message), etc.

Similar to RTP, these messages are also sent over UDP. You might ask “what if these are lost too”? Good question!

RTCP packets are sent at regular intervals, so you’d know if you missed one, and network routers and switches will prioritize RTCP packets over other data, so you’re unlikely to lose too many in a row unless there was a complete loss of connectivity.

Peer

WebRTC is often called a “peer-to-peer” (P2P) protocol. You might’ve heard that phrase in a different context: P2P file transfer, such as Bittorrent.

The word “peer” contrasts with “server-client” architectures, in which “client” computers can only talk to (or via) “server” computers, not directly to each other.

We can contrast server-client architecture with peer-to-peer using a real-world example:

• If you send a letter to your friend using a postal service, that’s a server-client architecture.
• If you leave the letter in your friend’s mailbox yourself, that’s peer-to-peer.

But what if you don’t know what kind of messages the recipient can receive or understand? For that we have…

SDP

Stands for Session Description Protocol which is a standardized message format to tell the other side the following:

• Whether you want to send and/or receive, audio and/or video
• How many streams of audio and/or video you want to send / receive
• What formats you can send or receive, for audio and/or video

This is called an “offer”. Then the other peer uses the same message format to reply with the same information, which is called an “answer”.

This constitutes media “negotiation”, also called “SDP exchange”. One side sends an “offer” SDP, the other side replies with an “answer” SDP, and now both sides know what to do.

As you might expect, there’s a bunch of other technical details here, and you can know all about them at this excellent page that explains every little detail. It even explains the format for ICE messages! Which is…

ICE

Interactive Connectivity Establishment is a standardized mechanism for peers to tell each other how to transmit and receive UDP packets. The simplest way to think of it is that it’s just a list of IP address and port pairs.

Once both sides have successfully sent each other (“exchanged”) ICE messages, both sides know how to send RTP and RTCP packets to each other.

Why do we need IP address + port pairs to know how to send and receive packets? For that you need to understand…

How The Internet Works

If you’re connected to the internet, you always have an IP address. That’s usually something like 192.168.1.150 – a private address that is specific to your local (home) network and has no meaning outside of that. Having someone’s private IP address is basically like having just their house number but no other parts of their address, like the street or the city. Useful if you're living in the same building, but not otherwise.

Most personal devices (computer or phone or whatever) with access to the Internet don’t actually have a public IP address. Picking up the analogy from earlier, a public IP address is the internet equivalent of a full address with a house number, street address, pin code, country.

When you want to connect to (visit) a website, your device actually talks to an ISP (internet service provider) router, which will then talk to the web server on your behalf and ask it for the data (website in this case) that you requested. This process of packet-hopping is called “routing” of network packets.

This ISP router with a public address is called a NAT (Network Address Translator). Like the name suggests, its job is to translate the addresses embedded in packets sent to it from public to private and vice-versa.

Let’s say you want to send a UDP packet to www.google.com. Your browser will resolve that domain to an IP address, say 142.250.205.228. Next, it needs a port to send that packet to, and both sides have to pre-agree on that port. Let’s pick 16789 for now.

Your device will then allocate a port on your device from which to send this packet, let’s say 11111. So the packet header looks a bit like this:

From	To
192.168.1.150:11111	142.250.205.228:16789

Your ISP’s NAT will intercept this packet, and it will replace your private address and port in the From field in the packet header to its own public address, say 169.13.42.111, and it will allocate a new sender port, say 22222:

From	To
169.13.42.111:22222	142.250.205.228:16789

Due to this, the web server never sees your private address, and all it can see is the public address of the NAT.

When the server wants to reply, it can send data back to the From address, and it can use the same port that it received the packet on:

From	To
142.250.205.228:16789	169.13.42.111:22222

The NAT remembers that this port 22222 was recently used for your From address, and it will do the reverse of what it did before:

From	To
142.250.205.228:16789	192.168.1.150:11111

And that’s how packets are send and received by your phone, computer, tablet, whatever when talking to a server.

Since at least one side needs to have a public IP address for this to work, how can your phone send messages to your friend’s phone? Both only have private addresses.

Solution 1: Just Use A Server As A Relay

The simplest solution is to have a server in the middle that relays your messages. This is how all text messaging apps such as iMessage, WhatsApp, Instagram, Telegram, etc work.

You will need to buy a server with a public address, but that’s relatively cheap if you want to send small messages.

For sending RTP (video and audio) this is accomplished with a TURN (Traversal Using Relays around NAT) server.

Bandwidth can get expensive very quickly, so you don’t want to always use a TURN server. But this is a fool-proof method to transmit data, so it’s used a backup.

Solution 2: STUN The NAT Into Doing What You Want

STUN stands for “Simple Traversal of UDP through NATs”, and it works due to a fun trick we can do with most NATs.

Previously we saw how the NAT will remember the mapping between a “port on its public address” and “your device’s private address and port”. With many NATs, this actually works for any packet sent on that public port by anyone.

This means if a public server can be used to create such mappings on the NATs of both peers, then the two can send messages to each other from NAT-to-NAT without a relay server!

Let’s dig into this, and let’s substitute hard-to-follow IP addresses with simple names: AlicePhone, AliceNAT, BobPhone, BobNAT, and finally STUNServer:19302.

First, AlicePhone follows this sequence:

1. AlicePhone sends a STUN packet intended for STUNServer:19302 using UDP

   From	To
   AlicePhone:11111	STUNServer:19302

2. AliceNAT will intercept this and convert it to:

   From	To
   AliceNAT:22222	STUNServer:19302

3. When STUNServer receives this packet, it will know that if someone wants to send a packet to AlicePhone:11111, they could use AliceNAT:22222 as the To address. This is an example of an ICE candidate.

4. STUNServer will then send a packet back to AlicePhone with this information.

Next, BobPhone does the same sequence and discovers that if someone wants to send a packet to BobPhone:33333 they can use BobNAT:44444 as the To address. This is BobPhone’s ICE candidate.

Now, AlicePhone and BobPhone must exchange these ICE candidates.

How do they do this? They have no idea how to talk to each other yet.

The answer is… they Just Use A Server As A Relay! The server used for this purpose is called a Signalling Server.

Note that these called “candidates” because this mechanism won’t work if one of the two NATs changes the public port also based on the public To address, not just the private From address. This is called a Symmetric NAT, and in these (and other) cases, you have to fallback to TURN.

Signalling Server

Signalling is a technical term that simply means: “a way to pass small messages between peers”. In this case, it’s a way for peers to exchange SDP and ICE candidates.

Once these small messages have been exchanged, the peers know how to send data to each other over the internet without needing a relay.

Now open your mind: you could use literally any out of band-mechanism for this. You can use Amazon Kinesis Video Signalling Channels. You can use a custom websocket server or a ProtoBuf server.

Heck, Alice and Bob can copy/paste these messages into iMessage on both ends. In theory, you can even use carrier pigeons — it’ll just take a very long time to exchange messages 😉

That’s it, this is what Signalling means in a WebRTC context, and why it’s necessary for a successful connection!

What a Signalling Server gives you on top of this is state management: checking whether a peer is allowed to send messages to another peer, whether a peer is allowed to join a call, can be invited to a call, which peers are in a call right now, etc.

Based on your use-case, this part can be really easy to implement or really difficult and heavy in corner-cases. Most people can get away with a really simple protocol, just by adding authorization to this multi-party protocol I wrote for the GStreamer WebRTC multiparty send-receive examples. More complex setups require a more bespoke solution, where all peers aren’t equal.