Dissecting the Humanization Pipeline for AI Text: A 6-Step Ablation Study

The scores are good. But what's actually working? In the previous article, I built a pipeline to transform AI text to feel more human-like and reported benchmark results of Mean Alignment 0.945 and Distribution Alignment 0.864. Not bad. But I had my own doubts. Out of the six transformation steps, which ones are truly effective, and which are just noise? High scores alone don't inform design decisions. So I conducted an Ablation Study (removal experiment). I disabled one step at a time and observed what happened. To cut to the chase, there were two surprises and one failure. Method Using a held-out test set of 500 samples (80/20 split), I re-evaluated the pipeline by disabling each of the 6 steps one at a time. Metric Meaning Mean Alignment How close the average feature vector of the pipeline output is to human text Distribution Alignment Overall distribution similarity based on Wasserstein distance Results: The Most Critical Step and the Completely Useless One Disabled Step Mean Align