Yes, I'm aware. Like I said before, each data point in this plot was calculated with both the all pass and ERB centered at the frequency plotted. In other words, at 300Hz, both the all pass and ERB are centered at 300Hz. At 1kHz, both are centered at 1kHz. At 3kHz, both are centered at 3kHz. (And so on...)If the crossover frequency is not 1 kHz, then the Phase vs. Frequency will be the same shape, except that it will shift left or right so that the inflection point is at the new crossover frequency.
As you say, the bandwidth in octaves changes with frequency (lower frequencies being wider), which is why the phase rotation over the ERB decreases with increasing frequency.
Here's another plot which shows the (instantaneous) phase rotation per ERB for some individual crossover frequencies:
You can see that the peaks approximately match the curve shown in the other plot.