Say the TCP stream sends packets of 1500 bytes, an optimal size for most part of most part of the journey of the packet
At one point the MTU is smaller, a router have to fragment it in two, say one fragment of 1260 bytes and the last fragment contains the remaining 240 bytes.
So, a lot of segments gets fragmented into one 1260 byte fragment and one 240 byte fragment. That 240 byte fragment is far from optimal, the overhead to payload ratio is a lot worse than for the 1260 fragment.
A much more optimal approach would be for the endpoint to adjust the segment size, and send all packets as 1260 bytes. (which is what will happen if PMTU works)
Path MTU discovery is supposed to ferret out the fact that the TCP segment size should be reduced.
IP fragmentation is just a best effort mechanism that is better than the datagrams being dropped due to their excessive size; it allows connections to be established.
You don't want to be doing bulk transfers with fragmentation.
IP fragmentation won't help with latency, because it is redundant. The TCP window is already a form of "fragmentation". TCP already fragments the data into segments that can be received out of order, individually re-transmitted and re-assembled.
For large transfers over fast, high latency networks, TCP supports window scaling. Larger individual segments (and thus IP fragmentation) is not required for the sake of having more data "in the air".
This neglects the fact that IP imposes a fixed per-packet overhead. More packets means lower efficiency.
Using larger packets reduces the overhead as a percentage of traffic, so is desirable regardless of the TCP window size.
Higher efficiency means higher bandwidth, which means lower latency for small transfers. Waiting for two 10-byte packets yields latency equal to the higher of the two latencies. Waiting for one 20-byte packet is, on average, lower latency (because you take one sample instead of worst-of-two).
Of course, we want to be using the largest packets that the underlying network allows.
My comment is about using larger packets specifically while relying on fragmentation to deliver them. Fragmentation re-introduces the overhead since fragments carry IP headers.
If a protocol already segments data into datagrams and reassembles them into a stream, it derives no advantage from IP fragmentation.
IP fragmentation basically provides the possibility of operating in the face of misconfiguration (hopefully temporarily), with reduced performance (better than no performance at all).
(Of course I'm not saying that since TCP chops the data into pieces and has a sliding window that can scale, there is no downside to making the pieces small just for the heck of it, like 50 bytes of payload!)
Why does it matter how efficient each individual packet is, in terms of overhead vs payload? If I send N bytes in M packets the total transfer overhead and efficiency are the same regardless of how uniform the packet sizes are.
The only problem I see is that the larger fragments are more likely to get refragmented, while uniform fragments are more likely to pass through subsequent MTU bottlenecks.
I don't quite understand what you are trying to convey. If you send 100 packets with 10 bytes payload(or TCP segments if you will), I hope it is obvious that there are more overhead (Ethernet/IP/TCP header of 100 packets) compared to sending 1 packet with a 1000 byte payload.
Normally this matters as it is desirable for systems to use less resources, not more. (e.g. we'd want to lessen the load on the network, we'd want applications to have low latency)
At one point the MTU is smaller, a router have to fragment it in two, say one fragment of 1260 bytes and the last fragment contains the remaining 240 bytes.
So, a lot of segments gets fragmented into one 1260 byte fragment and one 240 byte fragment. That 240 byte fragment is far from optimal, the overhead to payload ratio is a lot worse than for the 1260 fragment.
A much more optimal approach would be for the endpoint to adjust the segment size, and send all packets as 1260 bytes. (which is what will happen if PMTU works)